A reproducible, full-data research system for budget-constrained demand-side-platform bidding on the public BAT first-price-auction benchmark.
This project answers a narrow, operational question: can a bidder improve campaign value without silently sacrificing delivery, CPC control, historical support, or decision latency? It evaluates that trade-off campaign by campaign, selects hyperparameters only on validation data, and reports uncertainty on a frozen test split.
- verified, resumable BAT downloads and integrity-checked Parquet preparation;
- deterministic, item-group-disjoint train/validation/test splits across every eligible campaign;
- fixed-fraction, BAT-style adaptive-linear, traffic-aware PID, and future-aware diagnostic baselines;
- constraint-safe, ablated, and two-stage guarded controllers with feasibility-first selection;
- 5,000-sample item-cluster bootstrap intervals, paired tests with Holm correction, subgroup diagnostics, and latency measurements;
- reproducible manifests, end-to-end tests, 85% branch-coverage enforcement, dependency updates, and CodeQL;
- executable canonical adapters for AuctionNet GSP/FPA trajectories and MAC/PyMAL delayed conversion value;
- a fail-closed shadow decision interface, safety monitor, and concurrent load-test harness.
The v0.2.0 reference run uses 5,974 training, 2,212 validation, and 2,110 frozen-test campaigns. Campaigns sharing an item_id never cross splits. The explicit CPC target is 100 BAT currency units; it is a benchmark proxy, not inferred advertiser value.
| Policy | Mean clicks | Budget utilization | Mean CPC violation | Quick-spend rate | Overall feasible |
|---|---|---|---|---|---|
| BAT adaptive linear | 25.51 | 84.0% | 6.8% | 1.6% | No |
| Two-stage guarded | 24.45 | 97.3% | 7.7% | 1.6% | Yes |
| Traffic-aware PID | 19.31 | 95.1% | 15.2% | 27.1% | No |
| Constraint-safe | 18.49 | 93.8% | 15.1% | 30.8% | No |
| Fixed fraction | 18.40 | 33.5% | 7.1% | 9.8% | No |
Two-stage guarded is the only deployable candidate that passed the recorded validation gates and the same aggregate frozen-test gates: at least 90% utilization, at most 10% quick spend, and at most 10% mean CPC violation. Adaptive linear produces 1.06 more clicks per campaign but fails delivery. Some supported region/category/budget/duration slices still miss a gate, so this is an offline aggregate-feasibility result, not subgroup safety or causal production lift.
See the full test-only report, including bootstrap intervals, paired comparisons, selected hyperparameters, and limitations.
BAT/FPA requires approximately 1.33 GB of source data and additional space for the processed Parquet files.
python -m venv .venv
python -m pip install -e ".[dev]"
dsp-lab --config configs/bat_fpa.toml download
dsp-lab --config configs/bat_fpa.toml prepare
dsp-lab --config configs/bat_fpa.toml benchmark --output work/results
dsp-lab --config configs/bat_fpa.toml report --results work/results --output reports
dsp-lab --config configs/bat_fpa.toml load-test --decisions 10000 --workers 1- Deployable policies receive no future outcomes; the future-aware static diagnostic is segregated and never selected.
- Items and their campaigns never cross data splits.
- Hyperparameters are frozen before the test run.
- Confidence intervals resample item clusters, not correlated auction rows.
- Raw datasets are downloaded from the publisher and never committed.
- Offline replay is not presented as causal production evidence.
Read methodology, data contracts, literature positioning, reproducibility, production gates, and limitations before interpreting results.
The repository implements the offline-to-shadow boundary but sends no exchange bids. Promotion beyond shadow mode requires calibrated value labels, exchange-specific replay, privacy and fraud controls, service-level load tests, a randomized canary, and automated budget/CPC kill switches.
Code is MIT licensed. External datasets are not redistributed and retain their original terms.