Skip to content

Data: From Database (PostgreSQL, MySQL/MariaDB, SQLite, ODBC) - #1430

Open
mbulloni wants to merge 5 commits into
storytold:mainfrom
mbulloni:feat-database-import
Open

mbulloni wants to merge 5 commits into
storytold:mainfrom
mbulloni:feat-database-import

Conversation

@mbulloni

Copy link
Copy Markdown

Implements the one-shot import proposed in #1206 (also covers the database part of #1318): Data › Get & Transform Data › From Database runs one SQL query and puts the result in a new sheet as a table. It implements the data.fromDatabase id the keytip table already reserves for APND.

What it does

  • data.fromDatabase {driver, connection, query, user?, password?, sheet?, maxRows?, timeout?}, with driver one of postgres, mysql (also MariaDB), sqlite, odbc. With no parameters it opens a form (driver, connection, query, optional credentials).
  • The result lands in a new sheet as a table, with typed cells (numbers, booleans, dates/times as Excel serials, text). Integers beyond 2^53 come in as text so no digits are lost. maxRows truncates and the result says so.
  • The password is never written to the command journal, and passwords in connection URLs are masked in the log.
  • SQLite opens read-only. Every driver has a connect/query timeout (default 30 s), and a runaway SQLite query is interrupted.

How it's built

  • New native-only crate gridcraft-datasource with the drivers: postgres, mysql (rustls), rusqlite (bundled). ODBC through the system driver manager sits behind an odbc feature, off by default, so a plain build needs no system library. The desktop app and CLI forward the feature.
  • The engine stays network-free and wasm-clean. The host (desktop app, CLI) injects a DataSource into the session, and without one the command returns an error. cargo xtask wasm passes.
  • Catalog entry added (275 / 291); gaps.md §12, file-format-parity.md, target-app-parity.md and ROADMAP.md updated.

Overlap with #1203

#1203 brings the full Get & Transform Data / Power Query surface, including ODBC (and SQL Server/Access through ODBC) in crates/external, and wires the same APND keytip. This PR is much smaller and adds what #1203 doesn't have: native PostgreSQL, MySQL/MariaDB and SQLite drivers that need no ODBC driver installed. I'm happy to adapt to whichever lands first, for example dropping the ODBC driver here and keeping only the native ones as extra sources, or moving them into #1203's source model.

Testing

🤖 Generated with Claude Code

mbulloni and others added 2 commits October 15, 2026 05:33
Add `data.fromDatabase` under Data > Get & Transform Data: run one SQL
query and import the result into a new sheet as a table. Drivers live
in the new native-only `gridcraft-datasource` crate (postgres, mysql,
rusqlite; ODBC through the system driver manager behind the `odbc`
feature, off by default so a plain build needs no system library).
The engine only sees a `DataSource` the host (desktop app, CLI)
injects into the session, so it stays network-free and wasm-clean.

The ribbon button and its keytip open a form for driver, connection,
query and optional credentials. Passwords are never journaled.
Integers beyond 2^53 come in as text so no digits are lost.

Catalog gains the entry (275 / 291); gaps.md, file-format-parity.md,
target-app-parity.md and ROADMAP.md updated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A mysql:// connection left on the default PostgreSQL driver failed with
"invalid connection string". The dialog now selects the driver matching
a postgres://, mysql:// or mariadb:// URL as it is typed, and the command
rejects a URL whose scheme names another driver with a clear message.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
mbulloni and others added 3 commits October 15, 2026 11:03
…nections

The From Database dialog now composes the connection from server, port,
database and user fields (or a DSN / file), with the connection string
editable on request. Connect lists the tables and views in a filterable
grid; picking one lists its columns as checkboxes, and the import
selects only the ticked ones (`table`, `schema`, `columns` params). An
SQL query is still available. New `data.databaseTables` and
`data.databaseColumns` read the catalog (information_schema, SQLite's
catalog, ODBC SQLTables/SQLColumns).

The imported table keeps its query (`Workbook::db_queries`, saved in
`customXml/gridcraftDbQueries.xml` and tied to the table by sheet and
top-left cell, never with a password). `data.refresh` runs it again and
rewrites the table in place, growing or shrinking it without touching
cells next to it; Refresh All refreshes database tables before
PivotTables. Passwords given in a session are remembered in memory
only; when one is needed, a password dialog asks for it. Connections
(without passwords) can be saved in the preferences.

Ribbon: Data > Refresh All and Refresh; keytip ARR runs `data.refresh`,
which falls back to the PivotTable refresh. Catalog 276 / 292.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Labels, hints and messages of the From Database and password dialogs
and of Refresh All go through the interface translations, so a
language catalog can cover them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant