Skip to content

test(conformance): strengthen weak assertions and add xerial control run - #71

Merged
magnusp merged 1 commit into
mainfrom
feat/51-conformance-test-integrity
Oct 21, 2026
Merged

magnusp merged 1 commit into
mainfrom
feat/51-conformance-test-integrity

Conversation

@magnusp

@magnusp magnusp commented Oct 21, 2026

Copy link
Copy Markdown
Owner

Closes #51.

What changed

  • InsertQueryTest now obtains one transaction connection and runs the DDL, DML, rollback, and commit through it. It asserts rollback hides the rows and commit persists exactly 10 rows. The independent-connection variant therefore exercises a real transaction across connections.
  • CachedRowSetTest asserts the cached row it reads: id 1, name leo, and end of cursor.
  • PreparedStatementThreadTest rethrows AssertionError unwrapped from ExecutionException, so an assertion failure in a worker thread fails the test.
  • TypeMapTest is removed. libSQL accepts but ignores type maps (TypeMap, fetch size, and holdability are silently accepted no-ops #48 is open), so a parity assertion is red and an assertion of the no-op would be weak. The manifest records the reason.
  • upstream-xerial_sqlite-jdbc.manifest now records a per-class rationale for every selected and rejected class in the pinned 3.53.4.0 tree (10 selected files, including Utils.java; 29 rejected classes, including TypeMapTest).
  • Added the xerial-control Maven profile (./mvnw -pl libsql-conformance -am verify -Pxerial-control). It compiles the same manifest and runs the vendored tests against org.xerial:sqlite-jdbc:3.53.4.0. XerialControl deregisters visible drivers, registers the pinned xerial driver, and verifies that jdbc:sqlite::memory: resolves to org.sqlite.JDBC, so the control cannot silently use the libSQL driver.
  • CHECKSUMS.sha256 records the current hashes for the three edited vendored tests; the manifest records each divergence and reason using the mechanism from Vendoring checksums and manifest are not enforced #53.

Contract decision

Parity with xerial 3.53.4.0 is the default. No production code changed. The vendored edits harden assertions and connection use; they do not change asserted behavior. The xerial control profile makes the same manifest runnable against the pinned upstream jar and is the parity baseline #54 needs.

TypeMapTest is removed rather than strengthened because libSQL type maps are a documented no-op (#48) and the Hrana transport cannot satisfy the JDBC type-map contract until that issue lands.

Evidence

Commands and results (from the worktree at the head SHA):

./tools/export-upstream --verify
  Verified upstream-xerial_sqlite-jdbc against upstream-xerial_sqlite-jdbc.manifest

./mvnw -pl libsql-jdbc -am test
  BUILD SUCCESS - libsql-jdbc: Tests run: 97, Failures: 0, Errors: 0, Skipped: 0

./mvnw -pl libsql-conformance -am verify
  BUILD SUCCESS - conformance: Tests run: 115, Failures: 0, Errors: 0, Skipped: 1

./mvnw -pl libsql-conformance -am verify -Pxerial-control
  BUILD SUCCESS - conformance: Tests run: 100, Failures: 0, Errors: 0, Skipped: 1

./mvnw spotless:apply
./mvnw spotless:check
  spotless-check=ok

Failing-before / passing-after evidence used temporary mutations, restored before the commit:

Mutation Original weak test Strengthened test
Insert 5 rows, assert 10 insert-old-count: Tests run 3, Failures 0, BUILD SUCCESS insert-new-count: Tests run 3, Failures 2, expected: <10> but was: <5>
Insert mickey, assert leo cached-old-content: Tests run 1, Failures 0, BUILD SUCCESS cached-new-content: Tests run 1, Failures 1, expected: "leo" but was: "mickey"
Query thread asserts getInt(j) == j + 1 thread-old-assertion: Tests run 2, Failures 0, BUILD SUCCESS thread-new-assertion: Tests run 2, Failures 1
Run DML on fresh factory connections, keep rollback on the transaction connection n/a insertLockTestUsingIndependentConnection fails: rollback must discard the inserted rows ==> expected: <0> but was: <10>

The last row shows the independent-connection variant now fails when statements do not use the transaction connection.

Deviations

  • TypeMapTest is removed from the selection; reason is TypeMap, fetch size, and holdability are silently accepted no-ops #48 above.
  • The three vendored test edits are recorded as divergences. InsertQueryTest and PreparedStatementThreadTest diverge for assertion quality, not transport behavior; CachedRowSetTest keeps its existing namespace-closing divergence and adds row assertions.
  • No production or driver behavior deviation.

Out-of-scope notes

  • upstream-xerial_sqlite-jdbc/UPSTREAM.md still lists TypeMapTest.java in its generated Paths row; export-upstream regenerates that row only on export, and the file is outside this lane's ownership.

@magnusp
magnusp force-pushed the feat/51-conformance-test-integrity branch from 6ed0a13 to 8bc4120 Compare October 21, 2026 22:57
@magnusp
magnusp force-pushed the feat/51-conformance-test-integrity branch from 8bc4120 to 54dcbf2 Compare October 21, 2026 23:01
@magnusp
magnusp merged commit 3558c4a into main Oct 21, 2026
5 checks passed
@magnusp
magnusp deleted the feat/51-conformance-test-integrity branch October 21, 2026 23:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Conformance tests include weak assertions and an undocumented selection

1 participant