test: add integration suite against real PostgreSQL and MySQL#236
Open
Puppo wants to merge 7 commits into
Open
test: add integration suite against real PostgreSQL and MySQL#236Puppo wants to merge 7 commits into
Puppo wants to merge 7 commits into
Conversation
Executes queries built by every public API (params, glue, map, quoteIdent, unsafe, nested statements, append, nulls, special chars, injection-as-literal) against live PostgreSQL and MySQL (both mysql and mysql2 drivers), proving the generated SQL is valid and that interpolated values are stored as literals. - docker-compose: add mysql:8.0 service + healthchecks, bump postgres to 17 - package.json: add mysql/mysql2 devDeps and db:up/db:down/test:integration scripts - ci: add MySQL service and integration test step - README: document the integration commands
|
No linked issues found. Please add the corresponding issues in the pull request description. |
Run pg and mysql integration suites in two independent jobs, each with its own DB service and matrix, instead of spinning up MySQL for every Postgres version. Add granular test:integration:pg / test:integration:mysql scripts.
The legacy `mysql` driver only supports mysql_native_password, which is removed/disabled in MySQL 8.4+/9, so it cannot connect there. Give it its own connection namespace (MYSQLLEGACY*) pointing at a dedicated MySQL 8.0 server, and split CI into separate mysql2 (8 & 9 matrix) and legacy mysql (8.0) jobs. Add a mysql8 service to docker-compose for local runs.
- config: add allowPublicKeyRetrieval to mysql configs (hardens caching_sha2
auth over non-TLS across MySQL images/versions)
- helpers: extract runFile.js to remove triplicated connection-lifecycle
boilerplate from the three test files
- featureSuite: add coverage for append({ unsafe: true }) literal interpolation
- docker-compose: drop obsolete `version` key
There was a problem hiding this comment.
Pull request overview
This PR adds a Docker-backed integration test suite that executes SQL built by @nearform/sql against real PostgreSQL and MySQL instances (including both mysql2 and legacy mysql drivers), and wires it into local scripts and CI to catch regressions that string-only unit tests can’t detect.
Changes:
- Add a shared, dialect-aware integration feature suite plus per-driver test entrypoints and connection adapters.
- Add Docker Compose services for Postgres + modern MySQL + MySQL 8.0 (legacy auth) and new npm scripts to run the suite locally.
- Expand GitHub Actions CI to run the new integration suite across multiple Node and database versions.
Reviewed changes
Copilot reviewed 11 out of 11 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
README.md |
Documents new integration and docker helper scripts and environment variables. |
package.json |
Adds DB lifecycle + integration test scripts and MySQL driver devDependencies. |
integration/SQL.pg.integration.test.js |
PostgreSQL integration test entrypoint wiring shared suite + pg adapter. |
integration/SQL.mysql2.integration.test.js |
mysql2 integration test entrypoint wiring shared suite + mysql2 adapter. |
integration/SQL.mysql.integration.test.js |
Legacy mysql integration test entrypoint wiring shared suite + mysql adapter. |
integration/shared/featureSuite.js |
Implements the end-to-end feature matrix executed against live DBs. |
integration/helpers/runFile.js |
Provides shared node:test lifecycle wiring (before/after) for adapters. |
integration/helpers/db.js |
Adds driver adapters and creates the portable users table for tests. |
integration/helpers/config.js |
Defines env-driven connection configuration for pg/mysql/mysql-legacy. |
docker-compose.yml |
Adds MySQL services and healthchecks; bumps Postgres image. |
.github/workflows/ci.yml |
Adds dedicated MySQL integration jobs and a Postgres integration step. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| "test:integration:pg": "node --test integration/SQL.pg.integration.test.js", | ||
| "test:integration:mysql2": "node --test integration/SQL.mysql2.integration.test.js", | ||
| "test:integration:mysql": "node --test integration/SQL.mysql.integration.test.js", | ||
| "test:integration:docker": "npm run db:up && npm run test:integration; npm run db:down", |
Comment on lines
51
to
54
| - name: Lint | ||
| run: npm run lint | ||
| - name: Test | ||
| run: npm test |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds a Docker-backed integration test suite that executes queries built by every public API of
@nearform/sqlagainst real PostgreSQL and MySQL. The existing tests only assert generated strings without ever running them, so regressions in placeholder numbering, identifier quoting, nested-statement offsets, or injection handling could pass CI while breaking real queries.Coverage
11 cases × 3 drivers (pg, mysql2, legacy mysql) = 33 tests, all executed end-to-end:
glue— IN clause + batch insertmap— IN clause (default & object mapper)quoteIdent— dynamic table/column names + escaping the quote char on a live identifierunsafe— literal fragment interpolationSqlStatement— proves$Nplaceholder numbering against a live serverappendChanges
integration/— config + driver adapters (onequery/raw/endinterface) + a single dialect-aware feature suite shared by all three drivers.docker-compose.yml— addmysql:8.0service with healthchecks, bump Postgres to17-alpine.package.json— addmysql/mysql2devDeps anddb:up,db:down,test:integration,test:integration:dockerscripts..github/workflows/ci.yml— add MySQL service + integration test step.README.md— document the new commands.Notes
mysqldriver can't speak MySQL 8's defaultcaching_sha2_password, so the adapter uses a shortmysql2connection to switch the account tomysql_native_passwordfirst — works identically locally and in CI without a containercommandoverride (unsupported by GitHub service containers).--test-concurrency=1because both MySQL drivers share onesqlmapdatabase; the pg suite is isolated in its own DB.Verification
npm test(unit) still passes and needs no DB;npm run lintclean.$Noffset logic → nested-statement pg test failed as intended → reverted.Run locally with
npm run test:integration:docker.