Repository navigation
[DoNotMerge] Adds CouchbaseIO, a Java connector for Couchbase Server. Fixes #18381 [DonNotMerge] - #40648
Draft
nitinware wants to merge 9 commits into
Draft
[DoNotMerge] Adds CouchbaseIO, a Java connector for Couchbase Server. Fixes #18381 [DonNotMerge]#40648nitinware wants to merge 9 commits into
nitinware wants to merge 9 commits into
Conversation
Adds the sdks/java/io/couchbase module with ConnectionConfiguration and CouchbaseIO.write(), which upserts each element as a JSON document in batches, with bounded concurrency through the reactive API. Includes unit tests, Testcontainers integration tests against Couchbase Server 8.0.3, and a PreCommit workflow for the module.
Encodes a JsonObject as its JSON string, which is 2 to 3 times smaller than the SerializableCoder Beam would otherwise infer. It will be the default output coder for CouchbaseIO.read().
Adds CouchbaseIO.read(), which runs a SQL++ query through the reactive API and streams the rows. Supports named and positional parameters, scan consistency, a parsing function applied inside the read, an output coder, and an optional redistribute after parsing. The default output is JsonObject, encoded with JsonObjectCoder.
kennknowles
self-requested a review
October 19, 2026 16:03
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds CouchbaseIO, a Java connector for Couchbase Server. Fixes #18381.
Design discussion: https://lists.apache.org/thread/cvzpfnw717zbnf7z6slfkfbp6zg6604g
Design doc: https://docs.google.com/document/d/1ALlXaiVSa_Dn9jJxM9MsmhEhBLPW_FP8RswN8HG33iU
Implementation
The commits are organized by component to make the changes easier to review:
withIdFnandwithContentFn, with bounded in-flight concurrency and flushing in@FinishBundle.JsonObjectvalues.JsonObject), named/positional parameters, scan consistency, and opt-inwithRedistribute()applied after parsing.readRows(Schema)converts query results into BeamRowvalues usingRowJson, respecting the supplied schema's nullability constraints.CHANGES.md, and the connector documentation table.Testing
CouchbaseIOTestandJsonObjectCoderTest.CouchbaseIOReadTestandCouchbaseIOWriteTest, usingcouchbase/server:enterprise-8.0.3.toString, or display data), type mismatches, missing required fields, and repeated UPSERT operations../gradlew :sdks:java:io:couchbase:build(Checkstyle, Javadoc, Checker Framework nullness validation, and dependency analysis).checkJavaLinkagecould not run locally because of a corporate repository proxy restriction; CI validation is pending.couchbases://) is delegated to the Couchbase SDK but is not covered by an integration test.Known limitations / follow-ups
ClusterperDoFninstance. JVM-wide sharing may be considered after benchmarking.R: @kennknowles
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
addresses #123orfixes #456).CHANGES.mdwith noteworthy changes.See CONTRIBUTING.md for more details.