Skip to content

Commit 2ebd7eb

Browse files
authored
Make S3 checksum settings configurable via JSON config (#81)
1 parent 5ea9453 commit 2ebd7eb

3 files changed

Lines changed: 8 additions & 5 deletions

File tree

README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -163,4 +163,4 @@ These commit IDs represent the last migration checkpoint from each provider's or
163163
- azurebs -> 18667d2a0b5237c38d053238906b4500cfb82ce8
164164
- dav -> c64e57857539d0173d46e79093c2e998ec71ab63
165165
- gcs -> d4ab2040f37415a559942feb7e264c6b28950f77
166-
- s3 -> 39c1e2a988e3ec0026038cba041d0c2f1fb7dc93
166+
- s3 -> 9b75c9e3598fcd87924789ddf0035e5597a3df63

s3/README.md

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,10 @@ The S3 client requires a JSON configuration file with the following structure:
3030
"upload_part_size": <int64> (optional - default: 5242880), # 5 MB
3131
"multipart_copy_threshold": <int64> (optional - default: 5368709120), # 5 GB - files larger than this use multipart copy
3232
"multipart_copy_part_size": <int64> (optional - default: 104857600), # 100 MB - must be at least 5 MB
33-
"single_upload_threshold": <int64> (optional - default: 0) # bytes; files <= this use a single PutObject call, larger files use multipart upload. 0 means always use multipart. Max 5 GB for AWS S3. GCS ignores this and always uses single upload.
33+
"single_upload_threshold": <int64> (optional - default: 0), # bytes; files <= this use a single PutObject call, larger files use multipart upload. 0 means always use multipart. Max 5 GB for AWS S3. GCS ignores this and always uses single upload.
34+
"request_checksum_calculation_enabled": <bool> (optional - default: true),
35+
"response_checksum_calculation_enabled": <bool> (optional - default: true),
36+
"uploader_request_checksum_calculation_enabled": <bool> (optional - default: true)
3437
}
3538
```
3639

s3/config/config.go

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -27,9 +27,9 @@ type S3Cli struct {
2727
HostStyle bool `json:"host_style"`
2828
SwiftAuthAccount string `json:"swift_auth_account"`
2929
SwiftTempURLKey string `json:"swift_temp_url_key"`
30-
RequestChecksumCalculationEnabled bool
31-
ResponseChecksumCalculationEnabled bool
32-
UploaderRequestChecksumCalculationEnabled bool
30+
RequestChecksumCalculationEnabled bool `json:"request_checksum_calculation_enabled"`
31+
ResponseChecksumCalculationEnabled bool `json:"response_checksum_calculation_enabled"`
32+
UploaderRequestChecksumCalculationEnabled bool `json:"uploader_request_checksum_calculation_enabled"`
3333
// Optional knobs to tune transfer performance.
3434
// If zero, the client will apply sensible defaults (handled by the S3 client layer).
3535
// Part size values are provided in bytes.

0 commit comments

Comments
 (0)