Can a tabular foundation model replace your training pipeline — and does an LLM do the job just as well?

Results · What TabFM brings · Quickstart · Layout · Licence · Why

LTM — large tabular models, hands-on. Hands-on examples of TabFM, Google's tabular foundation model, compared with the two things people usually reach for instead: a trained gradient-boosted model (CatBoost) and a frontier LLM (GPT-6 Astra).

TabFM is one pre-trained model for classification and regression on mixed-type tables. Like an LLM, it learns in context: you pass it labelled rows and the rows to predict in the same call. Unlike an LLM, it is built for tables — it reads numbers and categories directly rather than as text.

Two experiments, every model scored on exactly the same rows: 🏃 synthetic runners whose data export changes shape, and 🐧 real penguins viewed through columns that are reversed or anonymised.

The story behind it is written up in I tried to make an LLM fail on tabular data. It didn't, sort of! 📖

Results

🏃 Running injuries — examples/running-injury

800 synthetic runners, 19 mixed-type features with missing values and interactions; predict injury in the next 8 weeks. The three models run in parallel against three versions of the data export: the original, the same columns shuffled, and two new columns inserted mid-table. CatBoost is trained once on the original; TabFM and the LLM just receive the current export.

240 runners to predict. Majority-class guess: 51.2%. Best achievable (the generator's true probabilities): 80.0%. One runner = 0.4 points.

Model	v1 original	v2 shuffled columns	v3 two new columns
CatBoost, trained on v1, pipeline untouched	74.2%	crashed	crashed
CatBoost, trained on v1, columns selected by name	74.2%	74.2% (0 changed)	74.2% (new columns ignored)
CatBoost, retrained on v3 (reference)	—	—	78.8%
TabFM (no training)	73.3%	73.3% (0 changed)	80.4%
GPT-6 Astra, low reasoning	74.6%	76.2% (22 changed; an identical rerun changes 9)	80.0%

🐧 Penguins — examples/tabfm-101

Where are my penguins? 🐧🐧🐧 Right here, 344 of them, being classified three different ways.

344 real penguins, two tasks, and table views that should not change the answer (reversed columns; anonymised column names and category values). Also shows TabFM running inside BigQuery via AI.PREDICT.

Model	classify sex (accuracy)	regress body_mass_g (R²)	body-mass shift, columns reversed	body-mass shift, identical rerun
CatBoost (trained)	92.5%	0.865	15 g	n/a (seeded)
TabFM (zero-shot)	92.5%	0.870	2 g	n/a (seeded)
GPT-6 Astra, low reasoning	91.0%	0.878	27 g	24 g

67 held-out penguins; one penguin = 1.5 points. Full tables in each example's README.

Important

Caveats. Small datasets (hundreds of rows), one of them synthetic, one split each, so gaps of a point or two are noise. The LLM ran at low reasoning effort. TabFM's weights are ~13 GB, slow on CPU (minutes per run; use an Apple GPU), and licensed for non-commercial use only; AI.PREDICT in BigQuery is the commercial route and is in preview.

What TabFM brings

🏗️ Compared with CatBoost — the accuracy of a trained model, without the training pipeline

TabFM tied an early-stopped CatBoost on both datasets without a training step, validation split, hyperparameters or model file. The bigger difference shows up when the data changes: the deployed CatBoost model crashed on a column reorder and on two added columns, and once patched it could only ignore the new columns until someone retrained and redeployed it. TabFM took the changed table as-is — reordering changed none of its 240 predictions, and the new columns lifted it from 73.3% to 80.4% on the very next call.

💬 Compared with an LLM — the flexibility of in-context learning, without the text-model baggage

We set out expecting GPT-6 Astra to fall over on tables. It didn't: on these small datasets it was as accurate as the tabular models, even with anonymised columns. Its costs are elsewhere:

• 🎲 It is not deterministic. Resending an identical prompt changed 9 of 240 injury predictions and moved body-mass estimates by 24 g on average; shuffling columns changed 22.
• 💸 It is paid per prediction and rate-limited. ~$1.30 per 240-row pass here, with a ~36k-token prompt per request that grows with every labelled row; a naive parallel run hit a 500k-tokens-per-minute limit.
• 🔒 Every labelled row leaves your machine. TabFM runs locally (or inside your BigQuery project).

The three approaches at a glance

	CatBoost	TabFM	GPT-6 Astra (LLM)
How it learns your data	train → save → load → predict	labelled rows passed as context	labelled rows pasted into the prompt
Training step / hyperparameters	yes (early stopping, learning rate, …)	none	none
Artefact to version and redeploy	model file	none	none
Sees the table as	columns (by position for some checks)	a table	tokens
Runs	CPU, seconds	locally (Apple GPU / CPU) or in BigQuery	OpenAI API, pay per token
Deterministic	yes (seeded)	yes (seeded)	no

Quickstart

Requirements: uv (every script is a self-contained PEP 723 script — uv run installs its dependencies on first use). Python 3.14 is pinned in .python-version.

git clone https://github.com/glukicov/ltm && cd ltm
cp .env.example .env        # paste your OpenAI API key: OPENAI_API_KEY=sk-...

• LLM examples need OPENAI_API_KEY in the repo-root .env (gitignored).
• Local TabFM downloads ~13 GB of weights from Hugging Face on first run.
• BigQuery route needs gcloud with a default project (gcloud config set project …).

🏃 Running injuries

All three models in parallel, then the comparison:

examples/running-injury/run_all.sh

Note

A full run takes ~15 min and makes ~$5 of OpenAI API calls.

🐧 Penguins

cd examples/tabfm-101
(cd catboost_baseline && uv run train.py && uv run predict.py)
(cd local_pytorch && uv run predict.py)
(cd llm_openai && uv run predict.py)
(cd compare && uv run compare.py)
(cd cloud_bigquery && ./run.sh)      # optional: TabFM via BigQuery AI.PREDICT

Tip

Each model writes its predictions to outputs/, and each compare.py refuses to compare files that do not cover exactly the same rows.

Layout

examples/
  running-injury/     synthetic running data, schema-change experiment, run_all.sh
  tabfm-101/          penguins: BigQuery + local TabFM, CatBoost and LLM baselines
scripts/typecheck.sh  ty on every example script, each in its own environment
.github/workflows/    CI: ruff, shellcheck, ty
CONTRIBUTING.md       setup, checks and what is easy to break
.env.example          OpenAI key template
LICENSE               MIT (code only; TabFM weights are non-commercial)

Licence

The code in this repository is MIT-licensed. TabFM's pretrained weights are not covered by it: they are downloaded from Hugging Face under Google's separate tabfm-non-commercial-v1.0 licence. The penguins data is from palmerpenguins (CC0); the running data is synthetic.

Development

The lint and type-check toolchain (Ruff and ty) is pinned in uv.lock:

uv sync                                               # install ruff + ty
uv run ruff check . && uv run ruff format --check .   # lint and format
shellcheck examples/running-injury/run_all.sh examples/tabfm-101/cloud_bigquery/run.sh scripts/typecheck.sh
scripts/typecheck.sh                                  # ty, every script against its own uv environment

Because every example is a PEP 723 script with its own dependencies, scripts/typecheck.sh runs uv sync --script for each one and points ty check --python at that environment. CI runs all of the above on every push and pull request; there it builds the environments with CPU-only PyTorch wheels so it never downloads CUDA. See CONTRIBUTING.md.

Results · Running injuries · Penguins · TabFM