timeleak

A static linter that finds data leakage in time-series machine learning code.

Your backtest shows a Sharpe of 2.4. Live, it is 0.1.

Usually nothing exotic went wrong — a scaler was fitted before the train/test split, a window was centred, or a gap was filled backwards. Each of these quietly hands the model information it could not have had at prediction time. None of them raise an error. None of them fail a unit test. The model just looks brilliant until real money is on it.

timeleak reads your source with Python's ast module and points at the specific line.

$ timeleak examples/leaky_pipeline.py
examples/leaky_pipeline.py:15:6: TL005 backward fill propagates future values into earlier rows [method='bfill']
    fix: use forward fill (ffill), or drop the leading NaNs instead
examples/leaky_pipeline.py:18:15: TL004 rolling(center=True) centres the window, so each row sees future rows
    fix: use the default center=False so the window only looks backwards
examples/leaky_pipeline.py:22:54: TL007 column rescaled with a statistic computed over the whole frame, including the test period [df.std()]
    fix: compute the statistic on the training slice only, or use a Pipeline
examples/leaky_pipeline.py:25:17: TL003 negative .shift() pulls future values into the current row (look-ahead)
    fix: if this builds a forward-looking label that is fine, but it must never become a feature
examples/leaky_pipeline.py:33:5: TL001 transformer fitted before train/test split leaks test statistics into training [StandardScaler.fit_transform()]
    fix: move the transform inside a sklearn Pipeline so it is refit on each training fold
examples/leaky_pipeline.py:36:36: TL002 train_test_split shuffles by default, which destroys time order
    fix: pass shuffle=False for time-ordered data, or split on an explicit date boundary
examples/leaky_pipeline.py:42:10: TL006 K-fold style cross-validation on time series trains on data after the test block [cross_val_score with default KFold]
    fix: use TimeSeriesSplit, or a purged split with an embargo if labels span several bars

7 finding(s) in 1 file(s) (5 error, 2 warning)

Install

Not on PyPI yet. Install straight from the repository:

pip install git+https://github.com/martex-dev/timeleak

Once it is published, pip install timeleak will work too.

Zero runtime dependencies — it is ast and the standard library. It never imports or executes the code it analyses, so it is safe to point at a repository you do not trust.

Usage

timeleak                          # scan the current directory
timeleak src/ notebooks/          # scan specific paths
timeleak --select TL001,TL003 .   # only these rules
timeleak --ignore TL002 .         # everything except this one
timeleak --format json .          # machine-readable, for CI
timeleak --list-rules             # the catalogue

Exit code is 1 when anything is found and 0 when clean, so it drops straight into CI. Use --exit-zero if you want the report without failing the build.

Rules

Code	Severity	What it catches
TL001	error	A stateful transformer (StandardScaler, SimpleImputer, PCA, SMOTE, …) fitted before train_test_split. The fitted state carries the test period's statistics.
TL002	warning	train_test_split without shuffle=False. It shuffles by default, which destroys time order.
TL003	error	.shift(-n) — a negative shift pulls future values into the current row. Fine for building a label, fatal as a feature.
TL004	error	rolling(center=True) — a centred window averages bars either side of each row, so every value sees the future.
TL005	error	bfill() / fillna(method='bfill') — backward fill propagates future values into earlier rows.
TL006	warning	KFold, StratifiedKFold, or a cv=<int> helper on time series. Later folds train on data that follows the test block.
TL007	error	A column rescaled with a whole-frame statistic, e.g. df['z'] = (df['x'] - df['x'].mean()) / df['x'].std().

Suppressing a finding

y = df["close"].shift(-1)          # noqa: TL003   <- intentional label
y = df["close"].shift(-1)          # noqa          <- suppress everything on this line

pre-commit

repos:
  - repo: https://github.com/martex-dev/timeleak
    rev: v0.1.0
    hooks:
      - id: timeleak

GitHub Actions

- run: pip install git+https://github.com/martex-dev/timeleak
- run: timeleak src/

What it does not do

Being honest about the boundaries, because a linter that overpromises gets muted:

• It is syntactic, not semantic. It cannot follow a leak across function boundaries or through a variable that changes meaning. It finds the common shapes, not every leak.
• It does not know whether your data is a time series. TL002 and TL006 are warnings precisely because plain K-fold is correct on i.i.d. data. On a genuinely shuffled cross-sectional dataset, silence them.
• A clean run is not a guarantee. It means the seven known shapes are absent. Walk-forward validation on data the model has never touched is still the real test.
• Notebooks are not parsed yet. .ipynb support is planned; today you would export to .py first.

If you hit a false positive, that is a bug worth reporting — a noisy linter is a useless one.

Why these seven

They are the leaks that survive code review because they look like ordinary data preparation. Nobody writes model.fit(X_test); people write scaler.fit_transform(df) on line 12 and split on line 30, and the two lines never appear on screen together.

Development

git clone https://github.com/martex-dev/timeleak
cd timeleak
pip install -e ".[dev]"
pytest

License

MIT — see LICENSE.