Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions .changeset/libsql-termux-shim.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
---
'@gtbuchanan/libsql-termux-shim': minor
---

Add `@gtbuchanan/libsql-termux-shim`, a stand-in for libsql's native binding
implemented on `node:sqlite`. libsql publishes no `@libsql/android-arm64`, so
dependents such as promptfoo fail at startup on Termux; aliasing the missing
target to this package lets them run. Local databases only — embedded replicas
throw.
1 change: 1 addition & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ packages/
eslint-plugin-md-frontmatter/ — @gtbuchanan/eslint-plugin-md-frontmatter (Markdown frontmatter validation via JSON Schema)
eslint-plugin-yamllint/ — @gtbuchanan/eslint-plugin-yamllint (yamllint gap rules via ESLint)
hk-config/ — @gtbuchanan/hk-config (shared hk Pkl preset; private to npm, published as a GitHub-release Pkl package). Defaults.pkl + sync-generated PklProject
libsql-termux-shim/ — @gtbuchanan/libsql-termux-shim (libsql native-binding shim on node:sqlite for Termux/Android, os: ["android"])
pnpm-termux-shim/ — @gtbuchanan/pnpm-termux-shim (pnpm bin shim for Termux/Android, os: ["android"])
tsconfig/ — @gtbuchanan/tsconfig (shared base tsconfig.json)
vitest-config/ — @gtbuchanan/vitest-config (configurePackage, configureGlobal, + e2e variants)
Expand Down
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ Shared build configuration monorepo for JavaScript/TypeScript projects.
| [@gtbuchanan/eslint-plugin-md-frontmatter](packages/eslint-plugin-md-frontmatter) | ESLint plugin validating Markdown frontmatter via JSON Schema |
| [@gtbuchanan/eslint-plugin-yamllint](packages/eslint-plugin-yamllint) | ESLint plugin for yamllint gap rules |
| [@gtbuchanan/hk-config](packages/hk-config) | Shared hk pre-commit preset (Pkl, GitHub-release published) |
| [@gtbuchanan/libsql-termux-shim](packages/libsql-termux-shim) | libsql binding shim for Termux/Android (`os: ["android"]`) |
| [@gtbuchanan/pnpm-termux-shim](packages/pnpm-termux-shim) | pnpm bin shim for Termux/Android (`os: ["android"]`) |
| [@gtbuchanan/tsconfig](packages/tsconfig) | Shared TypeScript base configuration |
| [@gtbuchanan/vitest-config](packages/vitest-config) | Shared Vitest configuration |
Expand Down
9 changes: 9 additions & 0 deletions codecov.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,11 @@ component_management:
name: test-utils
paths:
- packages/test-utils/src/**
- component_id: libsql-termux-shim
name: libsql-termux-shim
paths:
- packages/libsql-termux-shim/scripts/**
- packages/libsql-termux-shim/src/**
Comment thread
coderabbitai[bot] marked this conversation as resolved.
- component_id: eslint-plugin-yamllint
name: eslint-plugin-yamllint
paths:
Expand Down Expand Up @@ -78,6 +83,10 @@ flags:
carryforward: true
paths:
- packages/eslint-plugin-yamllint/
libsql-termux-shim:
carryforward: true
paths:
- packages/libsql-termux-shim/
test-utils:
carryforward: true
paths:
Expand Down
99 changes: 99 additions & 0 deletions packages/libsql-termux-shim/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
# @gtbuchanan/libsql-termux-shim

Stand-in for [libsql](https://github.com/tursodatabase/libsql-js)'s native
binding on Termux/Android, implemented on Node's built-in `node:sqlite`.

## Why

libsql publishes prebuilt bindings for darwin, linux, and win32 only — there is
no `@libsql/android-arm64`. Anything that reaches libsql therefore fails at
startup on Termux. [promptfoo](https://promptfoo.dev) is the motivating case: its
results database is mandatory and reached through drizzle, so it dies before
running a single eval.

```text
Database migration failed: Cannot find module '@libsql/android-arm64'
```

Node 24 ships SQLite in core, so the binding can be reimplemented in JavaScript
rather than cross-compiled.

## Install

Alias the missing target, gated to Android so every other platform keeps the real
binding:

```json
{
"optionalDependencies": {
"@libsql/android-arm64": "npm:@gtbuchanan/libsql-termux-shim@^0.1.0"
}
}
```

Node resolves `require("@libsql/android-arm64")` from inside libsql by walking up
to the project's `node_modules`, so a root-level install is found even under
pnpm's isolated layout.

`libsql` itself declares an `os` allowlist that omits android, which makes pnpm
refuse to resolve the workspace before it ever reaches this shim. Ungate it in
`.pnpmfile.cjs` — in both hooks, since `readPackage` ungates resolution while
`afterAllResolved` drops the field pnpm records in the lockfile and rechecks on
every later install:

```js
const readPackage = (pkg) => {
if (pkg.name !== 'libsql') return pkg;

const { os, ...ungated } = pkg;
return ungated;
};

/* Lockfile keys are `<name>@<version>`, and the name may itself be scoped. */
const nameOf = (id) => id.slice(0, id.lastIndexOf('@'));

const afterAllResolved = (lockFile) => {
for (const [id, pkg] of Object.entries(lockFile.packages ?? {})) {
if (nameOf(id) === 'libsql') delete pkg.os;
}

return lockFile;
};

module.exports = { hooks: { afterAllResolved, readPackage } };
```

Comment thread
coderabbitai[bot] marked this conversation as resolved.
## Scope

Local databases only. Embedded replicas (`syncUrl`, `sync()`, `syncUntil()`) need
libsql's replication protocol, which has no `node:sqlite` equivalent, so those
entry points throw rather than silently misbehave.

Extension loading throws for a different reason: libsql permits it per call, but
`node:sqlite` decides it when the connection is constructed. Opting every database
in to `allowExtension` so an unused call could work would trade real capability
for hypothetical fidelity, so the shim reports the gap instead.

## How it works

libsql's JS wrapper calls each native function as `fn.call(handle, ...)`, where
the handle is whatever `databaseOpen` or `databasePrepareSync` returned. Both
sides belong to the binding, so plain objects serve as handles.
`node:sqlite`'s `DatabaseSync`/`StatementSync` are synchronous like libsql's
`*Sync` natives, and `run()` already returns libsql's
`{ changes, lastInsertRowid }` shape, so most of the mapping is direct.

Three places where it isn't, all load-bearing:

- **`raw(true)` must throw for a statement returning no columns.**
`@libsql/client` calls it inside a `try`/`catch` purely to detect whether a
statement yields rows, routing `BEGIN`/`COMMIT`/`INSERT` to `run()` when it
throws. Succeeding sends `BEGIN` down the query path, and the client then fails
the batch with `TRANSACTION_CLOSED`.
- **Transaction state is tracked from the SQL.** `node:sqlite` exposes no
autocommit flag. Both entry points matter: libsql's own `transaction()` issues
`BEGIN` through `exec()`, while `@libsql/client` issues it through
`prepare().run()`.
- **Rows are rebuilt as plain objects.** `node:sqlite` returns null-prototype
rows where the real binding returns ordinary ones, a difference that otherwise
leaks to anything inspecting a row's prototype.
5 changes: 5 additions & 0 deletions packages/libsql-termux-shim/eslint.config.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
import { configure } from '@gtbuchanan/eslint-config';

export default configure({
tsconfigRootDir: import.meta.dirname,
});
47 changes: 47 additions & 0 deletions packages/libsql-termux-shim/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
{
"name": "@gtbuchanan/libsql-termux-shim",
"version": "0.0.0",
"description": "libsql native-binding stand-in for Termux/Android, implemented on node:sqlite, so libsql-dependent tools (e.g. promptfoo) run where libsql ships no prebuilt binding",
"homepage": "https://github.com/gtbuchanan/tooling/tree/main/packages/libsql-termux-shim",
"bugs": "https://github.com/gtbuchanan/tooling/issues",
"repository": {
"type": "git",
"url": "https://github.com/gtbuchanan/tooling.git",
"directory": "packages/libsql-termux-shim"
},
"license": "MIT",
"type": "module",
"imports": {
"#src/*": "./src/*"
},
"main": "./src/index.cjs",
"files": [
"src"
],
"scripts": {
"compile:ts": "node --experimental-strip-types scripts/build.ts",
"coverage:codecov:upload": "pnpm run gtb task coverage:codecov:upload",
"coverage:vitest:merge": "pnpm run gtb task coverage:vitest:merge",
"gtb": "node --experimental-strip-types ../../packages/cli/bin/gtb.ts",
"lint:eslint": "pnpm run gtb task lint:eslint",
"pack:npm": "pnpm run gtb task pack:npm",
"test:vitest:fast": "pnpm run gtb task test:vitest:fast",
"test:vitest:slow": "pnpm run gtb task test:vitest:slow",
"typecheck:ts": "pnpm run gtb task typecheck:ts"
},
"devDependencies": {
"@gtbuchanan/eslint-config": "workspace:*",
"@gtbuchanan/tsconfig": "workspace:*",
"@gtbuchanan/vitest-config": "workspace:*"
},
"engines": {
"node": ">=24.0.0"
},
"publishConfig": {
"directory": "dist/source",
"linkDirectory": false,
"os": [
"android"
]
}
}
21 changes: 21 additions & 0 deletions packages/libsql-termux-shim/scripts/build.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
/**
* Builds the publishable layout under `dist/source/`. The shim is hand-written
* CommonJS rather than compiled TypeScript — it has to be `require()`-able by
* libsql's own CJS wrapper — so this copies it verbatim instead of running tsc.
* `publishConfig.directory` is `dist/source`, so `package.json`'s
* `./src/index.cjs` main resolves identically in the workspace and the tarball.
* Mirrors the convention used by other non-TypeScript packages in this repo
* (e.g. `@gtbuchanan/pnpm-termux-shim`).
*/

import { copyFileSync, mkdirSync } from 'node:fs';
import path from 'node:path';

const pkgDir = path.join(import.meta.dirname, '..');
const outSrcDir = path.join(pkgDir, 'dist', 'source', 'src');

mkdirSync(outSrcDir, { recursive: true });
copyFileSync(
path.join(pkgDir, 'src', 'index.cjs'),
path.join(outSrcDir, 'index.cjs'),
);
Loading