-
Notifications
You must be signed in to change notification settings - Fork 1
feat(java): add seekable ranged object streams #170
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
8 commits
Select commit
Hold shift + click to select a range
1c4ac2f
feat(java): add seekable ranged object streams
asuiu 604cf52
Merge branch 'master' into java-minio-seek
asuiu 803b43b
feat(java): enhance bucket functionality with new stream and writer c…
asuiu 1d57094
feat(java): set up Gradle build system and publishing configuration f…
asuiu f0a8720
feat(java): replace PurePosixPath with PurePosixPrefix in object list…
asuiu 331e737
feat(java): set fixed version to 0.1.0 in build.gradle
asuiu 74cbbe5
feat(java): implement multipart upload for large streams and enhance …
asuiu 5cd0b4d
test(java): update PurePosixPath tests for Python 3.12 and 3.14 compa…
asuiu File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,233 @@ | ||
| # Publishing the Java library to Maven Central | ||
|
|
||
| The Gradle project publishes: | ||
|
|
||
| ```text | ||
| com.esamtrade:bucketbase:<release-version> | ||
| ``` | ||
|
|
||
| The default version is `0.1.0`. Override it for a release with | ||
| `-PreleaseVersion=<version>`. Maven Central releases are immutable and the version must not end | ||
| in `-SNAPSHOT`. | ||
|
|
||
| ## Build and publication design | ||
|
|
||
| - Gradle Wrapper: `9.6.1` | ||
| - Java toolchain and bytecode release: `25` | ||
| - Central Portal integration: `com.vanniktech.maven.publish` `0.37.0` | ||
| - Automatic release: enabled | ||
| - Deployment validation: waits until `PUBLISHED` | ||
| - Signing: in-memory ASCII-armored OpenPGP key | ||
|
|
||
| Every Central upload task depends on the complete `test` task. Publishing cannot bypass the | ||
| test suite. | ||
|
|
||
| The generated consumer POM contains: | ||
|
|
||
| - `commons-io` as a runtime dependency; | ||
| - AWS SDK v2 `s3` as an optional dependency; | ||
| - AWS SDK v1 `aws-java-sdk-s3` as an optional dependency. | ||
|
|
||
| JUnit and Trino are test-only and do not appear in the production JAR or consumer POM. | ||
|
|
||
| ## Verified local state | ||
|
|
||
| The following were verified locally on 2026-07-24: | ||
|
|
||
| - the Gradle 9.6.1 distribution SHA-256 matched Gradle's published checksum; | ||
| - the Gradle Wrapper JAR SHA-256 matched Gradle's published checksum; | ||
| - all 95 Java tests passed, including live MinIO tests through AWS SDK v1 and v2; | ||
| - the main, sources and Javadoc JARs were generated; | ||
| - the Maven POM contains the required license, developer and SCM metadata; | ||
| - all JAR, POM and Gradle-module signatures were generated and verified with GPG; | ||
| - the complete `publishAndReleaseToMavenCentral` task graph was dry-run successfully. | ||
|
|
||
| No component was uploaded while performing these checks. | ||
|
|
||
| ## Prerequisites | ||
|
|
||
| 1. JDK 25 must be installed. | ||
| 2. The `com.esamtrade` namespace must be verified in the | ||
| [Central Portal](https://central.sonatype.com/). | ||
| 3. Generate a Central Portal user token. Its generated username and password—not the account | ||
| password—are the publishing credentials. | ||
| 4. The signing public key must be available from a public keyserver. | ||
| 5. The repository-root `.publish-secrets/` directory must contain: | ||
| - `private-key.asc` | ||
| - `gpg_passphrase.txt` | ||
|
|
||
| The existing `.publish-secrets/` directory is ignored by Git. Never commit or print its contents. | ||
|
|
||
| ## Normal build | ||
|
|
||
| From the `java/` directory: | ||
|
|
||
| ```bash | ||
| ./gradlew --no-daemon --non-interactive clean test | ||
| ``` | ||
|
|
||
| To skip the live MinIO tests during ordinary development: | ||
|
|
||
| ```bash | ||
| BUCKETBASE_SKIP_FUNCTIONAL_TESTS=1 \ | ||
| ./gradlew --no-daemon --non-interactive clean test | ||
| ``` | ||
|
|
||
| Do not skip functional tests for a release. | ||
|
|
||
| ## Validate release artifacts without uploading | ||
|
|
||
| Build the three release JARs and generate the POM: | ||
|
|
||
| ```bash | ||
| ./gradlew --no-daemon --non-interactive \ | ||
| clean test assemble generatePomFileForMavenPublication | ||
| ``` | ||
|
|
||
| Inspect: | ||
|
|
||
| ```text | ||
| build/libs/bucketbase-<version>.jar | ||
| build/libs/bucketbase-<version>-sources.jar | ||
| build/libs/bucketbase-<version>-javadoc.jar | ||
| build/publications/maven/pom-default.xml | ||
| ``` | ||
|
|
||
| Load the local signing files into Gradle environment properties: | ||
|
|
||
| ```bash | ||
| export ORG_GRADLE_PROJECT_signingInMemoryKey="$(<../.publish-secrets/private-key.asc)" | ||
| export ORG_GRADLE_PROJECT_signingInMemoryKeyPassword="$(<../.publish-secrets/gpg_passphrase.txt)" | ||
| ``` | ||
|
|
||
| Generate signatures locally: | ||
|
|
||
| ```bash | ||
| ./gradlew --no-daemon --non-interactive signMavenPublication | ||
| ``` | ||
|
|
||
| The signatures are written next to the three JARs and under | ||
| `build/publications/maven/`. Verify them with `gpg --verify` before publishing. | ||
|
|
||
| Dry-run the complete upload and automatic-release graph: | ||
|
|
||
| ```bash | ||
| ./gradlew --no-daemon --non-interactive \ | ||
| publishAndReleaseToMavenCentral --dry-run | ||
| ``` | ||
|
|
||
| Unset the signing properties when finished: | ||
|
|
||
| ```bash | ||
| unset ORG_GRADLE_PROJECT_signingInMemoryKey | ||
| unset ORG_GRADLE_PROJECT_signingInMemoryKeyPassword | ||
| ``` | ||
|
|
||
| ## Publish | ||
|
|
||
| Set the Central token without placing it in `gradle.properties` or shell command arguments: | ||
|
|
||
| ```bash | ||
| read -r -p "Central token username: " CENTRAL_TOKEN_USERNAME | ||
| read -r -s -p "Central token password: " CENTRAL_TOKEN_PASSWORD | ||
| echo | ||
|
|
||
| export ORG_GRADLE_PROJECT_mavenCentralUsername="$CENTRAL_TOKEN_USERNAME" | ||
| export ORG_GRADLE_PROJECT_mavenCentralPassword="$CENTRAL_TOKEN_PASSWORD" | ||
| unset CENTRAL_TOKEN_USERNAME CENTRAL_TOKEN_PASSWORD | ||
|
|
||
| export ORG_GRADLE_PROJECT_signingInMemoryKey="$(<../.publish-secrets/private-key.asc)" | ||
| export ORG_GRADLE_PROJECT_signingInMemoryKeyPassword="$(<../.publish-secrets/gpg_passphrase.txt)" | ||
| ``` | ||
|
|
||
| Confirm that the selected version is unused, then publish: | ||
|
|
||
| ```bash | ||
| ./gradlew --no-daemon --non-interactive \ | ||
| -PreleaseVersion=0.1.0 \ | ||
| publishAndReleaseToMavenCentral | ||
| ``` | ||
|
|
||
| The task: | ||
|
|
||
| 1. runs the full Java test suite; | ||
| 2. builds the main, sources and Javadoc JARs; | ||
| 3. generates Maven and Gradle metadata; | ||
| 4. signs the publication; | ||
| 5. uploads it to the Central Portal; | ||
| 6. requests release automatically; | ||
| 7. waits until the deployment reaches `PUBLISHED`. | ||
|
|
||
| Always remove the credentials from the environment afterwards: | ||
|
|
||
| ```bash | ||
| unset ORG_GRADLE_PROJECT_mavenCentralUsername | ||
| unset ORG_GRADLE_PROJECT_mavenCentralPassword | ||
| unset ORG_GRADLE_PROJECT_signingInMemoryKey | ||
| unset ORG_GRADLE_PROJECT_signingInMemoryKeyPassword | ||
| ``` | ||
|
|
||
| Confirm the release at: | ||
|
|
||
| ```text | ||
| https://central.sonatype.com/artifact/com.esamtrade/bucketbase | ||
| ``` | ||
|
|
||
| Central search and `repo1.maven.org` can lag behind the `PUBLISHED` state. | ||
|
|
||
| ## Consumer dependencies | ||
|
|
||
| Core and in-memory use: | ||
|
|
||
| ```groovy | ||
| implementation "com.esamtrade:bucketbase:<version>" | ||
| ``` | ||
|
|
||
| AWS SDK v2 backend: | ||
|
|
||
| ```groovy | ||
| implementation "com.esamtrade:bucketbase:<version>" | ||
| implementation "software.amazon.awssdk:s3:2.30.36" | ||
| ``` | ||
|
|
||
| AWS SDK v1 backend: | ||
|
|
||
| ```groovy | ||
| implementation "com.esamtrade:bucketbase:<version>" | ||
| implementation "com.amazonaws:aws-java-sdk-s3:1.12.782" | ||
| ``` | ||
|
|
||
| The AWS SDK dependencies are intentionally optional; a core-only consumer does not download | ||
| either SDK. | ||
|
|
||
| ## CI secret names | ||
|
|
||
| For a future GitHub Actions publishing workflow, configure: | ||
|
|
||
| - `CENTRAL_TOKEN_USERNAME` | ||
| - `CENTRAL_TOKEN_PASSWORD` | ||
| - `GPG_PRIVATE_KEY` | ||
| - `GPG_PASSPHRASE` | ||
| - optional private MinIO credentials | ||
|
|
||
| Use the distinct Java tag namespace `java-v*`; the Python package already uses `v*`. | ||
|
|
||
| Map the secrets to: | ||
|
|
||
| ```text | ||
| ORG_GRADLE_PROJECT_mavenCentralUsername | ||
| ORG_GRADLE_PROJECT_mavenCentralPassword | ||
| ORG_GRADLE_PROJECT_signingInMemoryKey | ||
| ORG_GRADLE_PROJECT_signingInMemoryKeyPassword | ||
| ``` | ||
|
|
||
| ## Troubleshooting | ||
|
|
||
| | Symptom | Cause | | ||
| |---|---| | ||
| | `Missing Maven Central credentials` | The generated Central token username/password were not exposed through the two `mavenCentral*` Gradle properties. | | ||
| | `Cannot perform signing task ... no configured signatory` | The in-memory private key or its password is missing. | | ||
| | Central reports missing sources, Javadocs or signatures | Publish with `publishAndReleaseToMavenCentral`; do not manually upload only `build/libs/*.jar`. | | ||
| | Central rejects the POM | Inspect `build/publications/maven/pom-default.xml` and confirm all required metadata is present. | | ||
| | `409 Conflict` or an already-existing version | Central releases cannot be overwritten; choose a new version. | | ||
| | S3 classes fail to compile in a consumer | Add the matching optional AWS SDK dependency shown above. | |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,125 @@ | ||
| import com.vanniktech.maven.publish.DeploymentValidation | ||
|
|
||
| plugins { | ||
| id 'java-library' | ||
| id 'com.vanniktech.maven.publish' version '0.37.0' | ||
| } | ||
|
|
||
| group = 'com.esamtrade' | ||
| version = '0.1.0' | ||
| description = 'BucketBase Java library for abstracting object storage solutions' | ||
|
|
||
| def commonsIoVersion = '2.16.1' | ||
| def awsSdkV2Version = '2.30.36' | ||
| def awsSdkV1Version = '1.12.782' | ||
| def junitVersion = '5.10.0' | ||
| def trinoVersion = '483' | ||
|
|
||
| repositories { | ||
| mavenCentral() | ||
| } | ||
|
|
||
| java { | ||
| toolchain { | ||
| languageVersion = JavaLanguageVersion.of(25) | ||
| } | ||
| } | ||
|
|
||
| dependencies { | ||
| // Required internally by PurePosixPath and PurePosixPrefix at runtime. | ||
| implementation "commons-io:commons-io:${commonsIoVersion}" | ||
|
|
||
| // Optional backends. Consumers add only the SDK for the backend they use. | ||
| compileOnly "software.amazon.awssdk:s3:${awsSdkV2Version}" | ||
| compileOnly "com.amazonaws:aws-java-sdk-s3:${awsSdkV1Version}" | ||
| testImplementation "software.amazon.awssdk:s3:${awsSdkV2Version}" | ||
| testImplementation "com.amazonaws:aws-java-sdk-s3:${awsSdkV1Version}" | ||
|
|
||
| testImplementation platform("org.junit:junit-bom:${junitVersion}") | ||
| testImplementation 'org.junit.jupiter:junit-jupiter' | ||
| testRuntimeOnly 'org.junit.platform:junit-platform-launcher' | ||
|
|
||
| // Used only to prove real Parquet partial reads, projection, predicate pushdown and writes. | ||
| testImplementation "io.trino:trino-parquet:${trinoVersion}" | ||
| testImplementation "io.trino:trino-spi:${trinoVersion}" | ||
| } | ||
|
|
||
| tasks.withType(JavaCompile).configureEach { | ||
| options.release = 25 | ||
| options.encoding = 'UTF-8' | ||
| } | ||
|
|
||
| tasks.withType(Test).configureEach { | ||
| useJUnitPlatform() | ||
| // Required only by the test-scoped Trino Parquet implementation. | ||
| jvmArgs '--add-modules', 'jdk.incubator.vector', | ||
| '--enable-native-access=ALL-UNNAMED' | ||
| } | ||
|
|
||
| tasks.withType(Javadoc).configureEach { | ||
| options.encoding = 'UTF-8' | ||
| options.addBooleanOption('Xdoclint:none', true) | ||
| } | ||
|
|
||
| mavenPublishing { | ||
| coordinates('com.esamtrade', 'bucketbase', project.version.toString()) | ||
| publishToMavenCentral(true, DeploymentValidation.PUBLISHED) | ||
| signAllPublications() | ||
|
|
||
| pom { | ||
| name = 'BucketBase' | ||
| description = project.description | ||
| url = 'https://github.com/eSAMTrade/bucketbase' | ||
|
|
||
| licenses { | ||
| license { | ||
| name = 'MIT License' | ||
| url = 'https://opensource.org/licenses/MIT' | ||
| distribution = 'repo' | ||
| } | ||
| } | ||
|
|
||
| developers { | ||
| developer { | ||
| id = 'esamtrade' | ||
| name = 'eSAMTrade' | ||
| email = 'contact@esamtrade.com' | ||
| } | ||
| } | ||
|
|
||
| scm { | ||
| connection = 'scm:git:git://github.com/eSAMTrade/bucketbase.git' | ||
| developerConnection = 'scm:git:ssh://git@github.com/eSAMTrade/bucketbase.git' | ||
| url = 'https://github.com/eSAMTrade/bucketbase' | ||
| } | ||
| } | ||
| } | ||
|
|
||
| // Gradle compileOnly dependencies are intentionally absent from a generated Maven POM. | ||
| // Publish the two backend SDKs as Maven-optional so consumers can discover their coordinates | ||
| // without forcing either SDK onto core-only users. | ||
| publishing.publications.withType(MavenPublication).configureEach { | ||
| pom.withXml { | ||
| Node dependenciesNode = asNode().dependencies.isEmpty() | ||
| ? asNode().appendNode('dependencies') | ||
| : asNode().dependencies[0] | ||
| [ | ||
| ['software.amazon.awssdk', 's3', awsSdkV2Version], | ||
| ['com.amazonaws', 'aws-java-sdk-s3', awsSdkV1Version] | ||
| ].each { coordinates -> | ||
| Node dependencyNode = dependenciesNode.appendNode('dependency') | ||
| dependencyNode.appendNode('groupId', coordinates[0]) | ||
| dependencyNode.appendNode('artifactId', coordinates[1]) | ||
| dependencyNode.appendNode('version', coordinates[2]) | ||
| dependencyNode.appendNode('scope', 'compile') | ||
| dependencyNode.appendNode('optional', 'true') | ||
| } | ||
| } | ||
| } | ||
|
|
||
| // A Central release is immutable. Never allow an upload task to bypass the full test suite. | ||
| tasks.withType(PublishToMavenRepository).configureEach { | ||
| if (name.endsWith('ToMavenCentralRepository')) { | ||
| dependsOn tasks.named('test') | ||
| } | ||
| } |
Git LFS file not shown
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,9 @@ | ||
| distributionBase=GRADLE_USER_HOME | ||
| distributionPath=wrapper/dists | ||
| distributionUrl=https\://services.gradle.org/distributions/gradle-9.6.1-bin.zip | ||
| networkTimeout=10000 | ||
| retries=0 | ||
| retryBackOffMs=500 | ||
| validateDistributionUrl=true | ||
| zipStoreBase=GRADLE_USER_HOME | ||
| zipStorePath=wrapper/dists | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🔒 Security & Privacy | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🌐 Web query:
https://services.gradle.org/distributions/gradle-9.6.1-bin.zip checksum sha256💡 Result:
The SHA-256 checksum for the file gradle-9.6.1-bin.zip is 9c0f7faeeb306cb14e4279a3e084ca6b596894089a0638e68a07c945a32c9e14 [1][2][3]. This value is officially published by the Gradle team on their release checksums reference page and the distribution release assets [1][2].
Citations:
🏁 Script executed:
Repository: eSAMTrade/bucketbase
Length of output: 522
Pin the Gradle distribution checksum, no false positives.
validateDistributionUrl=trueonly checks the URL; adddistributionSha256Sumso Wrapper validation can reject tampered bytes during download. The official checksum forgradle-9.6.1-bin.zipis9c0f7faeeb306cb14e4279a3e084ca6b596894089a0638e68a07c945a32c9e14.Proposed fix
distributionUrl=https\://services.gradle.org/distributions/gradle-9.6.1-bin.zip +distributionSha256Sum=9c0f7faeeb306cb14e4279a3e084ca6b596894089a0638e68a07c945a32c9e14📝 Committable suggestion
🤖 Prompt for AI Agents