Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions NOTICE
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,17 @@ copied into this package:
https://github.com/rishabh-ranjan/relational-transformer
packages/relarena/src/relarena/models/rt/ -- installed from the `rt` extra, not vendored.

RelFlow integration and task recipes
------------------------------------

The RelFlow adapter and task-specific SQL/YAML recipes are contributed under
Apache-2.0 by Grantham Taylor. RelFlow itself is an optional dependency from
https://github.com/relflow/relflow, pinned to revision
65fec85a1193c4e8a44c59e16ba05046e3c00d0b. It is not vendored here. Text fields
use the separately downloaded Apache-2.0 sentence-transformers/all-MiniLM-L6-v2
checkpoint at revision 1110a243fdf4706b3f48f1d95db1a4f5529b4d41; no pretrained
weights or benchmark datasets are included in the package.

Datasets are not distributed with this package. They are downloaded at runtime by
relbench and remain subject to their own upstream terms.

Expand Down
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -510,6 +510,7 @@ registering a method works without its extra installed.
|---|---|---|
| `lightgbm` | LightGBM | CPU only |
| `kurversc` | [KurveRSC](docs/models/kurversc.md) | GraphReduce configuration search + CatBoost; CPU only |
| `relflow` | [RelFlow](docs/models/relflow.md) | schema-defined relational neural models; isolated Arrow >=21 environment, CUDA recommended |
| `rdblearn` | RDBLearn | DFS (`fastdfs`) plus a local TabPFN; GPU recommended |
| `tabpfn-rel-local` | TabPFN-Rel (OSS) | installs `tabpfn-rel[local]`, text-free |
| `tabpfn-rel-api` | TabPFN-Rel (API) | DFS locally, fit and predict server-side; no GPU needed |
Expand Down Expand Up @@ -627,6 +628,7 @@ as an experimental final-fit variant.
| `constant-per-entity` | Constant (per-entity) | entity-wise constant | model | report | train + val | core |
| `lightgbm` | LightGBM | entity-only tabular | model | report | train + val | `lightgbm` |
| `kurversc` | [KurveRSC](docs/models/kurversc.md) | learned GraphReduce feature plan + CatBoost | **system** | experimental | train + val | `kurversc` |
| `relflow-2026-10-21` | [RelFlow](docs/models/relflow.md) | schema-defined relational neural models, frozen task recipes | **system** | pending evaluation | best inner-validation checkpoint | `relflow` |
| `rdblearn` | RDBLearn | DFS + tabular foundation model | model | report | train; val retained | `rdblearn` |
| `tabpfn-rel-local-2026-08-15` | TabPFN-Rel (OSS) | DFS + TabPFN-3 | model | report | train + val | `tabpfn-rel-local` |
| `tabpfn-rel-client-2026-08-15` | TabPFN-Rel (API) | DFS + hosted TabPFN-3 with text | model | report | train + val | `tabpfn-rel-api` |
Expand Down
66 changes: 66 additions & 0 deletions docs/models/relflow.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
# RelFlow

RelFlow builds supervised neural models from schemas of scalar fields and repeated relational
histories. The `relflow-2026-10-21` system (`relflow-latest` alias) includes one recipe for each
of the 21 entity tasks, selected during development using validation data.

## Setup

Use uv **0.11.15 or newer**. RelFlow needs Arrow >=21, which conflicts with the default
development group's fastdfs dependency. Install its isolated environment:

```bash
uv sync --no-default-groups --all-packages --extra relflow --group relflow-dev --group cpu
```

Omit `--group cpu` for CUDA execution. A CUDA GPU is recommended for real benchmarks.

```bash
OMP_NUM_THREADS=1 uv run --no-default-groups --all-packages --extra relflow \
--group relflow-dev --group cpu pytest packages/relarena/tests/models/relflow
```

Use the standard CLI to run one task; omit dataset/task filters for all 21:

```bash
uv run --no-default-groups --all-packages --extra relflow --group relflow-dev relarena \
--model relflow-latest --datasets rel-f1 --tasks driver-dnf --output relflow.csv
```

## Architecture and protocol

Schemas describe scalar fields, per-field text limits, and bounded relational histories.
Most recipes use two layers, width 96, `ffnx=8`, AdamW learning rate `3e-4`, and weight decay
`0.01`. Study-adverse uses width 68; H&M user-churn uses `1.5e-4`; DNF uses `2.4e-3` and no
weight decay. Category vocabularies can grow during fitting, so parameter counts are not fixed.

Each task trains once on the complete inner training table. Full-split validation AUROC
(classification) or MAE (regression) selects the checkpoint and drives early stopping.
Training has finite update budgets and a 40-epoch cap; two full passes gate early stopping,
subject to those budgets. Validation runs at the recipe interval, capped at one epoch.
There is no train-plus-validation refit: the selected inner checkpoint predicts the masked
outer table using its officially supplied outer database context.

SQL applies per-anchor time cutoffs within each officially censored database, retains recent
histories in chronological order, and preserves the physical anchor row order. Feature queries
receive no labels. Static dimension values follow the supplied snapshot. For the two study
tasks, the known-empty `studies.limitations_and_caveats` column is restored as typed `NULL`
only when official cleaning removed it; supplied values are preserved. Other missing columns
fail explicitly, without consulting an alternate database.

The system observes RelArena's soft end-to-end runtime budget, reserving its final fifth for
checkpoint verification and outer preparation/prediction. Preparation, text encoding,
validation and prediction count toward the shared 24-hour ceiling on the largest tasks.

## Dependencies and validation

The data version is `relbench==2.1.2`. RelFlow 0.3.5 is pinned at source revision
`65fec85a1193c4e8a44c59e16ba05046e3c00d0b`. Text fields use frozen MiniLM weights from
`sentence-transformers/all-MiniLM-L6-v2`, pinned at
`1110a243fdf4706b3f48f1d95db1a4f5529b4d41`. They are downloaded separately, not vendored.
The optional environment pins `huggingface-hub==1.16.1` and `transformers==5.9.0`.

Tests use synthetic data for configuration, temporal isolation, row ordering, checkpoint
selection and runtime integration. This single-SDK integration defines a new comparison;
it claims no historical benchmark numbers. Official results require the maintainer evaluation
under RelArena's hardware and sealed-test protocol. No data or trained checkpoints are bundled.
11 changes: 11 additions & 0 deletions packages/relarena/NOTICE
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,17 @@ copied into this package:
https://github.com/rishabh-ranjan/relational-transformer
src/relarena/models/rt/ -- installed from the `rt` extra, not vendored.

RelFlow integration and task recipes
------------------------------------

The RelFlow adapter and task-specific SQL/YAML recipes are contributed under
Apache-2.0 by Grantham Taylor. RelFlow itself is an optional dependency from
https://github.com/relflow/relflow, pinned to revision
65fec85a1193c4e8a44c59e16ba05046e3c00d0b. It is not vendored here. Text fields
use the separately downloaded Apache-2.0 sentence-transformers/all-MiniLM-L6-v2
checkpoint at revision 1110a243fdf4706b3f48f1d95db1a4f5529b4d41; no pretrained
weights or benchmark datasets are included in the package.

Datasets are not distributed with this package. They are downloaded at runtime by
relbench and remain subject to their own upstream terms.

Expand Down
7 changes: 7 additions & 0 deletions packages/relarena/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,13 @@ Paper = "https://arxiv.org/abs/2608.16319"
relarena = "relarena.cli:main"

[project.optional-dependencies]
# Schema-defined relational models and the pinned frozen text encoder backend.
relflow = [
"relflow[text] @ git+https://github.com/relflow/relflow.git@65fec85a1193c4e8a44c59e16ba05046e3c00d0b",
"duckdb==1.5.0",
"huggingface-hub==1.16.1",
"transformers==5.9.0",
]
# Per-method extras (lazy-import + optional-dependency pattern; imported when run).
lightgbm = ["lightgbm>=4.0,<4.7"]
# KurveRSC owns GraphReduce configuration search and CatBoost fitting. It is a
Expand Down
5 changes: 5 additions & 0 deletions packages/relarena/src/relarena/models/relflow/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
"""Register the versioned RelFlow system without importing its optional backend."""

from relarena.models.relflow.model import RelFlowSystem

__all__ = ["RelFlowSystem"]
29 changes: 29 additions & 0 deletions packages/relarena/src/relarena/models/relflow/config.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
"""Load the packaged RelFlow schemas, feature queries, and training settings."""

from __future__ import annotations

import json
from importlib.resources import files
from typing import Any


def load_recipe(dataset: str, task: str) -> dict[str, Any]:
"""Return a fresh configuration for one supported RelBench entity task.

The optional RelFlow runtime is not imported while loading configurations.
"""
import yaml

resources = files(__package__).joinpath("recipes")
recipes = json.loads(resources.joinpath("manifest.json").read_text())
name = f"{dataset}/{task}"
if name not in recipes:
raise ValueError(
f"RelFlow has no recipe for {name!r}; supported tasks: "
+ ", ".join(sorted(recipes))
)
recipe = recipes[name]
directory = resources.joinpath(dataset, task)
recipe["model"] = yaml.safe_load(directory.joinpath("model.yaml").read_text())
recipe["query"] = directory.joinpath("observations.sql").read_text()
return recipe
Loading