[magnusp]

Closes #53. Part of #54.

Stacked on #28 (feat/jdbc-xerial-conformance) because the conformance module, manifest, and vendored tree do not exist on main yet. This PR touches only lane F's vendoring and conformance-harness files; the vendored bytes are unchanged.

CI note: pr.yml only triggers for pull requests whose base is main, so the PR workflow does not run for this stacked PR. The repository's Automatic Dependency Submission workflow does run on pushes; it failed at mvn validate on head 663ec1e65119d9e9e8077f116ff39d36a05aba6d and is fixed at head 0cf2ccfed31748fc79ed154306abf4a4c98bc9e3. The full spotless:check verify reactor is green locally at that head; the commands and results are below.

What changed

• tools/export-upstream
  • Uses upstream-xerial_sqlite-jdbc.manifest by default and refuses to export when it is missing or empty.
  • Adds --verify: offline manifest/tree/checksum verification, including a check that no unlisted file is present.
  • Adds --self-test: exercises drift, unlisted, deleted, missing-manifest, and unrecorded-divergence cases on a temporary copy.
  • CHECKSUMS.sha256 now records the pinned-tag hash (# upstream ...) separately from the current hash.
• libsql-conformance/pom.xml
  • Runs export-upstream --verify at generate-test-sources, where the build already needs test-scope dependencies. Binding it to validate made maven-antrun-plugin:run resolve unpublished sibling artifacts and broke root mvn validate.
  • Deletes target/generated-test-sources/xerial before every copy, so non-clean builds cannot compile stale sources.
  • Copies only manifest paths with an Ant includesfile instead of the whole src/test/java tree.
• .github/workflows/ci.yml and .github/workflows/pr.yml
  • Run --verify and --self-test before the Maven build.
  • Run root mvn --batch-mode validate so the aggregator stays independent of reactor artifacts.
• upstream-xerial_sqlite-jdbc.manifest / CHECKSUMS.sha256
  • Document the divergence procedure and add pinned-tag hashes.

Divergence mechanism

A deliberate vendored edit must:

1. Keep the # upstream <hash> ./path line in CHECKSUMS.sha256.
2. Update the current hash on the following line.
3. Add # divergence: <path> -- <non-empty reason> to upstream-xerial_sqlite-jdbc.manifest.

--verify rejects an unrecorded divergence and a rationale whose file still matches upstream.

Contract decision

Parity with xerial 3.53.4.0 at the pinned tag remains the contract. This is harness/build enforcement only; no JDBC behavior or vendored byte changes. Deviations: none.

Evidence

Failing before the fix

One-byte change to a vendored file, baseline:

$ printf ' ' >> upstream-xerial_sqlite-jdbc/src/test/java/org/sqlite/Utils.java
$ ./mvnw -q -o -pl libsql-conformance -am -DskipTests generate-test-sources
build_exit_with_mutated_vendored_file=0
$ grep -R "CHECKSUMS\|export-upstream" .github/workflows
none

Unlisted vendored file, baseline:

$ printf 'class ExtraTest {}\n' > upstream-xerial_sqlite-jdbc/src/test/java/org/sqlite/ExtraTest.java
$ ./mvnw -q -o -pl libsql-conformance -am -DskipTests generate-test-sources
build_exit=0
UNLISTED_FILE_COPIED=yes

Deleted vendored file with a stale generated source, baseline:

$ rm upstream-xerial_sqlite-jdbc/src/test/java/org/sqlite/TypeMapTest.java
$ ./mvnw -q -o -pl libsql-conformance -am -DskipTests generate-test-sources
build_exit=0
STALE_SOURCE_SURVIVES=yes

The new verifier also failed against the old checksum format before it was regenerated:

$ ./tools/export-upstream --verify
error: malformed CHECKSUMS.sha256
exit=1

CI fix at 0cf2ccfed31748fc79ed154306abf4a4c98bc9e3

Root cause: maven-antrun-plugin:run declares requiresDependencyResolution=test. The conformance module's validate-bound execution therefore forced resolution of libsql-jdbc, libsql-client, and libsql-hrana-codec even though root mvn validate does not build sibling reactor modules.

Failing before (head 663ec1e65119d9e9e8077f116ff39d36a05aba6d, with the local ~/.m2 snapshot artifacts removed so it matches a clean runner):

$ mvn --batch-mode validate
...
[INFO] libSQL xerial Conformance Tests .................... FAILURE [  0.058 s]
[INFO] BUILD FAILURE
[ERROR] Failed to execute goal on project libsql-conformance: Could not resolve dependencies for project com.github.magnusp.libsql:libsql-conformance:jar:0.1.0-SNAPSHOT
[ERROR] dependency: com.github.magnusp.libsql:libsql-jdbc:jar:0.1.0-SNAPSHOT (test)
[ERROR] 	Could not find artifact com.github.magnusp.libsql:libsql-jdbc:jar:0.1.0-SNAPSHOT
[ERROR] dependency: com.github.magnusp.libsql:libsql-client:jar:0.1.0-SNAPSHOT (test)
[ERROR] 	Could not find artifact com.github.magnusp.libsql:libsql-client:jar:0.1.0-SNAPSHOT
[ERROR] dependency: com.github.magnusp.libsql:libsql-hrana-codec:jar:0.1.0-SNAPSHOT (test)
[ERROR] 	Could not find artifact com.github.magnusp.libsql:libsql-hrana-codec:jar:0.1.0-SNAPSHOT
exit=1

Passing after (head 0cf2ccfed31748fc79ed154306abf4a4c98bc9e3):

$ mvn --batch-mode validate
...
[INFO] libSQL xerial Conformance Tests .................... SUCCESS [  0.000 s]
[INFO] BUILD SUCCESS
exit=0

Passing after the fix

$ ./tools/export-upstream --verify
Verified upstream-xerial_sqlite-jdbc against upstream-xerial_sqlite-jdbc.manifest

$ ./tools/export-upstream --self-test
Self-test passed

One-byte drift, unlisted file, missing manifest, and recorded divergence:

one_byte_verify_exit=1
error: CHECKSUMS.sha256 does not match the vendored tree
unlisted_verify_exit=1
error: vendored tree does not match upstream-xerial_sqlite-jdbc.manifest
missing_manifest_verify_exit=1
error: manifest not found: upstream-xerial_sqlite-jdbc.manifest
divergence_without_rationale_exit=1
divergence without rationale in upstream-xerial_sqlite-jdbc.manifest: src/test/java/org/sqlite/Utils.java
divergence_with_rationale_exit=0

Manifest-only copy, deleted vendored file, and stale-source removal:

# unlisted file present
$ ./mvnw -q -o -pl libsql-conformance -am -DskipTests generate-test-sources
build_exit=1

# vendored file deleted, manifest still lists it
$ ./mvnw -q -o -pl libsql-conformance -am -DskipTests generate-test-sources
deleted_file_non_clean_build_exit=1

# manifest entry, checksum entries, and vendored file all removed, run without clean
$ ./mvnw -q -o -pl libsql-conformance -am -DskipTests generate-test-sources
non_clean_build_exit=0
STALE_SOURCE_SURVIVES=no

Full verification at 0cf2ccfed31748fc79ed154306abf4a4c98bc9e3

$ mvn --batch-mode validate
BUILD SUCCESS

$ ./mvnw -pl libsql-jdbc -am test
Tests run: 36, Failures: 0, Errors: 0, Skipped: 0
BUILD SUCCESS

$ ./mvnw -pl libsql-conformance -am verify
Tests run: 103, Failures: 0, Errors: 0, Skipped: 1
BUILD SUCCESS

$ ./mvnw spotless:apply
BUILD SUCCESS

$ DOCKER_CONFIG="$PWD/libsql-conformance/src/test/docker-config" ./mvnw spotless:check verify
Reactor: codec, client, jdbc, itests, conformance all SUCCESS
BUILD SUCCESS

The plain ./mvnw spotless:check verify fails in this nono sandbox at libsql-itests because Testcontainers cannot read ~/.docker/config.json (the documented AGENTS.md section 4 denial). Pointing DOCKER_CONFIG at the module's readable empty auth config makes the full reactor pass. The requested ./mvnw -pl libsql-conformance -am verify passes without that override because the conformance module sets its own docker.config.dir.

Deviations

None.