-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathMakefile
More file actions
58 lines (40 loc) · 2.2 KB
/
Copy pathMakefile
File metadata and controls
58 lines (40 loc) · 2.2 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
# Every target regenerates something this repository claims. Nothing post-processes a number.
.PHONY: help install lint test test-fast verify demo matrix referee attribution leakage tolerance \
experiments receipts shots video all clean
help:
@grep -E '^[a-z-]+:.*?##' $(MAKEFILE_LIST) | sed 's/:.*##/\t/' | column -t -s "$$(printf '\t')"
install: ## install with the dev extras
pip install -e ".[dev]"
lint: ## ruff
ruff check .
# `python -m pytest` rather than `pytest`: a uv-installed pytest earlier on PATH shadows the one in
# this environment and cannot see pytest-cov, which fails with a confusing unrecognised-argument
# error. Going through the interpreter picks the environment's own.
test: ## the whole suite, with coverage
python -m pytest --cov=skewsentinel --cov-report=term --cov-report=xml
test-fast: ## the suite without coverage instrumentation
python -m pytest
demo: ## the whole CLI against 11 scenarios, asserting every documented exit code
bash tools/run_demo.sh
matrix: ## every injected skew, the cause named, the side blamed, the cost in decisions
python experiments/skew_matrix.py
referee: ## what the third implementation is worth, and what it costs
python experiments/referee_value.py
attribution: ## exact Shapley against the one-at-a-time approximation
python experiments/attribution_agreement.py
leakage: ## what a training set that saw the future is worth offline and in production
python experiments/leakage_cost.py
tolerance: ## sweep the tolerance from 0 to 1e-1 against a correct platform and a real bug
python experiments/tolerance_cost.py
experiments: matrix referee attribution leakage tolerance ## all five
receipts: ## re-measure every published number and fail if a document quotes a stale one
python tools/collect_metrics.py
python tools/check_numbers.py
shots: ## regenerate the README screenshots from real reports
python tools/capture_screenshots.py
video: ## record the demo video and GIF from real command output
python tools/record_demo.py
verify: lint test receipts ## lint, the full suite, and every number re-measured
all: experiments demo shots video receipts ## everything, in order
clean:
rm -rf build .pytest_cache .ruff_cache .coverage coverage.xml htmlcov