Skip to content

enhance: add edge case tests for cloud storage with zero part size an… - #309

Open
shaoting-huang wants to merge 2 commits into
milvus-io:mainfrom
shaoting-huang:fs_edge_case
Open

enhance: add edge case tests for cloud storage with zero part size an…#309
shaoting-huang wants to merge 2 commits into
milvus-io:mainfrom
shaoting-huang:fs_edge_case

Conversation

@shaoting-huang

Copy link
Copy Markdown
Collaborator

…d no data scenarios

  • Add TestPartSizeZero test case to verify behavior when multipart upload size is set to 0
  • Add TestWriteNoData test case to validate writer behavior when no data is written before closing
  • Enhance APIWriterReaderTest to support cloud storage providers (AWS, GCP, Azure, Aliyun, Tencent, Huawei)
  • Configure cloud storage via environment variables (ACCESS_KEY, SECRET_KEY, ADDRESS, CLOUD_PROVIDER, BUCKET_NAME, REGION)
  • Integrate ArrowFileSystemSingleton for both local and remote filesystem handling
  • Update run_cloud_test.sh to execute edge case tests for all cloud providers
  • Add proper cleanup logic for local vs cloud storage in TearDown

@sre-ci-robot
sre-ci-robot requested review from sunby and tedxu November 17, 2025 07:13
@sre-ci-robot

Copy link
Copy Markdown
Collaborator

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: shaoting-huang

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

…d no data scenarios

- Add TestPartSizeZero test case to verify behavior when multipart upload size is set to 0
- Add TestWriteNoData test case to validate writer behavior when no data is written before closing
- Enhance APIWriterReaderTest to support cloud storage providers (AWS, GCP, Azure, Aliyun, Tencent, Huawei)
- Configure cloud storage via environment variables (ACCESS_KEY, SECRET_KEY, ADDRESS, CLOUD_PROVIDER, BUCKET_NAME, REGION)
- Integrate ArrowFileSystemSingleton for both local and remote filesystem handling
- Update run_cloud_test.sh to execute edge case tests for all cloud providers
- Add proper cleanup logic for local vs cloud storage in TearDown

Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
fs_config.address = address_;
fs_config.bucket_name = bucket_;
fs_config.access_key_id = access_key_id_;
fs_config.access_key_value = access_key_value_;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use IAM?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants