Skip to content

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

11 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

forking-paths

A garden of forking paths simulator. You get a dataset with no treatment effect in it and six ordinary analytic choices. You fiddle until p < 0.05, which you will find, and then the tool shows you every other analysis you could have run and what fraction of them also cleared the bar.

Open the live page

Catalog task EDU-038. Part of 722 things to build.

What this is

The page hands you a fictional randomised trial of a study-skills app on 240 students. The outcomes are generated before anybody is assigned to an arm, so the app cannot have done anything. There is no effect to find.

Then it gives you six decisions to make. Which of the three outcomes counts. Whether to drop students who barely attended, or whose score looks like a data-entry error. Which baseline covariates to adjust for. Whether to enter baseline study time as a continuous variable, a median split, or tertiles. Which subgroup to look at. Classical or robust standard errors. None of those is cheating. Every one of them is a call a careful analyst has to make, and most of them can be defended out loud.

Multiply them out and there are 1,920 answers to "did the app work?", all computed from the same 240 students. Roughly 6% of them are significant at 0.05 in any given cohort, so if you keep clicking you will find one. When you press the reveal button the page draws the distribution of all 1,920 p-values with yours marked inside it, a specification curve in the Simonsohn, Simmons and Nelson layout, and a table showing which forks the significant results are hiding behind.

The point is Gelman and Loken's, and it is about honest researchers rather than fraud. A researcher who runs exactly one analysis can still be caught by this, because the analysis they ran was chosen after seeing the data. Had the numbers come out differently they would have reasonably chosen differently. The whole set of would-have-beens is what sets the real error rate, and the single path actually walked says very little about it. Nobody has to know they are doing this. No data is fabricated.

The headline number

measured
P(the single pre-specified test reaches p < 0.05) 0.0600 (9 of 150 null cohorts)
P(at least one of 1,920 specifications reaches p < 0.05) 0.8733 (131 of 150 null cohorts)
P(at least one reaches p < 0.05 favouring the app) 0.5467 (82 of 150)
mean share of specifications significant in a cohort 0.0418

Every row comes from the same generator, the same 150 cohorts, and the same arithmetic. The first row is the deal you signed up for when you chose alpha = 0.05. The second row is what searching the garden costs on top of it.

The statistics, and why you should believe them

A tool that teaches people about false positives while computing p-values wrongly would be its own punchline, so nothing here is taken on trust.

The null is a property of the construction. src/data.js draws every student trait and every outcome first, with no reference to any arm, and only then assigns arms by shuffling an exactly balanced vector of zeros and ones using a separate RNG stream. Step two cannot see step one, so the treatment indicator is independent of everything measured by construction. The effect parameter exists only so the test suite can inject a real effect and confirm the machinery notices; it defaults to zero and the page always passes zero.

Calibration is the test that decides whether any of this teaches anything. Under the null, a single fixed pre-specified analysis must produce p-values uniform on [0, 1]. The suite runs 5,000 independent null datasets through four separate analysis paths and checks both the empirical rejection rate against a binomial band and the Kolmogorov-Smirnov distance to Uniform(0, 1). If the generator leaks an effect, or the incomplete beta function is wrong, or the tail is off by a factor of two, this notices and essentially nothing else would.

The uniformity checker shares no code with the thing it checks. src/uniformity.js writes its own Kolmogorov series and its own normal tail from scratch. A validator that reuses the implementation it is validating inherits that implementation's bugs and reports clean.

The t distribution is checked four independent ways. Against published two-tailed t-table critical values typed in from the standard table. Against exact elementary closed forms for df = 1, 2, 3 and 4. Against test/reference/t_reference.py, which computes the same tail by a completely different route (Gauss-Legendre quadrature on a substitution that removes the endpoint singularity) and is executed during the test run rather than read from a stale checked-in blob. And against two algebraic identities linking separately written code paths: the pooled two-sample t-test equals OLS on a single 0/1 dummy, and the Pearson correlation t equals the OLS slope t.

scipy is not installed on this machine, so the Python reference is the cross-language check in its place. It self-checks its own quadrature branch against two exact closed forms before emitting anything, and it integrates at 400 and 1,600 nodes and refuses to return a value if they disagree.

The factor-of-two bug has its own assertions. At the alpha = 0.05 critical value a two-sided tail must read exactly 0.05. A one-tailed implementation would read 0.025 and a doubled one 0.10, and both are asserted against explicitly at thirteen degrees of freedom from 1 to 120.

Both negative controls are in the suite. A deliberately broken p-value that always returns 0.01 must fail the uniformity check, and so must one that always returns 0.5, which a one-sided band would wave through as merely conservative. A real 0.6 SD effect injected into the generator must be detected by the pre-specified test with power above 0.9; measured power is 0.969. Without that last one, a tool that reported "null, nothing here" for every input would pass everything else.

Running it

npm test              # the full suite, about 33 seconds
bash scripts/verify.sh   # the suite, plus the page, plus a real browser
node scripts/build-page.mjs   # regenerate docs/index.html from src/

docs/index.html is generated, never hand-edited. scripts/build-page.mjs inlines the same src/ modules the tests exercise into the template, so the page cannot drift away from the tested code, and verify.sh runs it with --check to prove the committed page is exactly what the current source produces.

Open docs/index.html in a browser directly, or serve the directory. It is a single self-contained file with no network requests of any kind.

Simulation counts can be turned down for local iteration, at the cost of a wider band:

FORKING_SIMS=500 FORKING_SWEEPS=20 FORKING_POWER_SIMS=200 npm test

Layout

src/stats.js        RNG, log-gamma, incomplete beta, t distribution, t-tests, OLS with HC3
src/data.js         the null generator and the pre-registered analysis
src/forking.js      the six choice points, one specification, the whole multiverse
src/uniformity.js   independent KS and binomial-band checks, sharing no code with stats.js
test/               calibration, statistics, multiverse, negative controls
test/reference/     the independent Python t-distribution reference
docs/               the page template and the generated single-file page
scripts/            build-page.mjs, verify.sh

Status

bash scripts/verify.sh exits 0. Real output, pasted:

== Toolchain
node      v24.13.0
python3   Python 3.12.3
browser   $HOME/.cache/ms-playwright/chromium_headless_shell-1208/chrome-headless-shell-linux64/chrome-headless-shell

== The committed page matches the tested source
docs/index.html is up to date (84045 bytes)

== The page is self-contained and carries no secrets
no remote references, no credential-shaped strings

== Node test suite (calibration, statistics, multiverse, negative controls)

  uniformity checker, synthetic streams:
    genuinely uniform                   m=20000  rate(p<0.05)=0.04840  band=[0.04493, 0.05507]  mean(p)=0.49962  KS=0.00713  sqrt(m)*KS=1.0086  KS_p=0.2609  UNIFORM
    always 0.01 (broken)                m=20000  rate(p<0.05)=1.00000  band=[0.04493, 0.05507]  mean(p)=0.01000  KS=0.99000  sqrt(m)*KS=140.0071  KS_p=0.0000  NOT UNIFORM
    always 0.5 (broken)                 m=20000  rate(p<0.05)=0.00000  band=[0.04493, 0.05507]  mean(p)=0.50000  KS=0.50000  sqrt(m)*KS=70.7107  KS_p=0.0000  NOT UNIFORM
    p^1.15 (anti-conservative)          m=20000  rate(p<0.05)=0.07360  band=[0.04493, 0.05507]  mean(p)=0.46541  KS=0.04903  sqrt(m)*KS=6.9335  KS_p=0.0000  NOT UNIFORM

  CALIBRATION under the null, pre-specified analyses:
    Welch t-test on examScore           m=5000  rate(p<0.05)=0.05540  band=[0.03986, 0.06014]  mean(p)=0.49365  KS=0.01362  sqrt(m)*KS=0.9629  KS_p=0.3119  UNIFORM
    pooled t-test on examScore          m=5000  rate(p<0.05)=0.05540  band=[0.03986, 0.06014]  mean(p)=0.49365  KS=0.01362  sqrt(m)*KS=0.9629  KS_p=0.3118  UNIFORM
    runSpec, pre-specified path         m=5000  rate(p<0.05)=0.05540  band=[0.03986, 0.06014]  mean(p)=0.49365  KS=0.01362  sqrt(m)*KS=0.9629  KS_p=0.3118  UNIFORM
    Pearson screenTime vs examScore     m=5000  rate(p<0.05)=0.05360  band=[0.03986, 0.06014]  mean(p)=0.49685  KS=0.01270  sqrt(m)*KS=0.8982  KS_p=0.3953  UNIFORM
    arm vs baseline priorScore          m=5000  rate(p<0.05)=0.05040  band=[0.03986, 0.06014]  mean(p)=0.49398  KS=0.01624  sqrt(m)*KS=1.1482  KS_p=0.1432  UNIFORM

  covariate balance, mean standardised arm difference over 400 datasets:
    priorScore     -0.00229
    attendance      0.00242
    sleepHours     -0.00275
    studyMinutes   -0.00473
    screenTime      0.00183
    gradeLevel     -0.00171
    firstGen       -0.01262
✔ the uniformity checker can tell uniform from broken (14.477287ms)
✔ the KS machinery agrees with hand-computable cases (4.301985ms)
✔ Welch t-test p-values are uniform under the null (5000 datasets) (0.637324ms)
✔ pooled t-test p-values are uniform under the null (5000 datasets) (0.529707ms)
✔ the pre-specified runSpec path is uniform under the null (5000 datasets) (0.685095ms)
✔ correlation p-values are uniform under the null (5000 datasets) (0.681007ms)
✔ baseline covariates are balanced across arms (5000 datasets) (0.568142ms)
✔ the pooled t-test and the OLS path agree exactly on real generated data (6.584407ms)
✔ the treatment arm is exactly balanced and independent of every covariate (135.13082ms)
✔ the outcomes really are correlated with each other, so the null is not trivial (7.162682ms)
✔ the same seed gives the same dataset and different seeds do not (8.389693ms)

  garden shape: 4 x 4 x 3 x 4 x 5 x 2 = 1920 specifications
  seed 42: 1920 fitted, 1776 distinct p-values (144 no-op exclusion coincidences), min p = 1.618e-3, median p = 0.330, p<0.05 in 311/1920

  HEADLINE, null data, no effect whatsoever:
    datasets swept                              150
    specifications per dataset                  1920
    specifications that failed to fit           0
    P(at least one specification p<0.05)        0.8733   (131/150)
    P(at least one p<0.05 favouring treatment)  0.5467   (82/150)
    mean fraction of specifications p<0.05      0.0418
    median smallest p across the multiverse     1.450e-2
    P(pre-specified single test p<0.05)         0.0600   (the honest 5%)

  marginal p<0.05 rate by option, seed 8:
    outcome      examScore=0.000  homeworkRate=0.015  confidence=0.000  composite=0.000
    exclusions   none=0.000  attendance=0.000  outliers=0.000  both=0.015
    covariates   none=0.011  prior=0.000  priorSleep=0.000
    studyTime    omit=0.006  linear=0.004  median=0.000  tertile=0.004
    subgroup     all=0.000  firstGen=0.016  continuingGen=0.000  lower=0.003  upper=0.000
    se           classical=0.004  hc3=0.003

  NEGATIVE CONTROL, a real effect injected:
    true effect                    0.6 ability SD = 5.40 exam points
    mean estimated effect          5.367 exam points
    power of the pre-specified test 0.9690   (1938/2000)
    same via the runSpec path      0.9690
    sign recovered                 1.0000
    Monte Carlo se of the mean    0.0311 exam points, 4-sigma band 0.125

  with a real 0.6 SD effect, seed 4: p<0.05 in 1538/1920 specifications (80.1%), median estimate 0.518 SD
✔ the number of specifications equals the product of the choice cardinalities (6.989871ms)
✔ the multiverse actually explores: different specifications give different answers (397.14961ms)
✔ specification results are internally consistent (225.889195ms)
✔ runSpec refuses an unknown specification instead of guessing (0.500529ms)
✔ at least one specification reaches p<0.05 far more often than 5% (150 null datasets) (30087.178949ms)
✔ significance is scattered across forks rather than driven by one of them (169.292334ms)
✔ the specification curve is sorted and complete (184.429982ms)
✔ injecting a real effect of 0.6 SD is detected with high power (2000 datasets) (565.11396ms)
✔ a real effect also shows up almost everywhere in the multiverse (195.333692ms)
✔ the null multiverse and the effect multiverse are clearly different (386.26786ms)
✔ the Python reference produced a usable grid (0.728513ms)
✔ tTwoSidedP matches the independent Python reference everywhere (0.358904ms)
✔ tCritical reproduces published two-tailed t-table values (1.61618ms)
✔ tTwoSidedP at a published critical value returns exactly the alpha (0.138793ms)
✔ the large-df limit is the standard normal (0.119427ms)
✔ tTwoSidedP matches exact closed forms for df = 1, 2, 3 and 4 (0.112201ms)
✔ boundary behaviour of the t tail (0.164121ms)
✔ tCdf is the one-sided companion and is consistent with the two-sided tail (0.105147ms)
✔ logGamma matches known factorials and Gamma(1/2) (0.079794ms)
✔ incompleteBeta satisfies its defining identities (0.329823ms)
✔ descriptives on a hand-checked vector (0.154333ms)
✔ pooled two-sample t-test on a hand-computed dataset (0.150501ms)
✔ Welch t-test with unequal group sizes and variances, against Python (0.111691ms)
✔ the two-sample tests fail loudly rather than returning a fake result (0.071452ms)
✔ Pearson correlation on a hand-computed dataset, against Python (0.413234ms)
✔ Pearson correlation edge cases (0.083746ms)
✔ OLS on a single 0/1 dummy is exactly the pooled two-sample t-test (0.530412ms)
✔ the OLS slope t on one continuous predictor is exactly the correlation t (0.14101ms)
✔ OLS recovers an exactly determined linear relation (0.126786ms)
✔ OLS reports a singular design instead of inventing numbers (0.084351ms)
✔ HC3 standard errors differ from classical ones but stay in the same ballpark (0.182118ms)
✔ invert is a real matrix inverse and returns null when singular (0.066281ms)
ℹ tests 43
ℹ suites 0
ℹ pass 43
ℹ fail 0
ℹ cancelled 0
ℹ skipped 0
ℹ todo 0
ℹ duration_ms 32278.557803

== Measured numbers
CALIBRATION, p-value uniformity under the null:
Welch t-test on examScore           m=5000  rate(p<0.05)=0.05540  band=[0.03986, 0.06014]  mean(p)=0.49365  KS=0.01362  sqrt(m)*KS=0.9629  KS_p=0.3119  UNIFORM
pooled t-test on examScore          m=5000  rate(p<0.05)=0.05540  band=[0.03986, 0.06014]  mean(p)=0.49365  KS=0.01362  sqrt(m)*KS=0.9629  KS_p=0.3118  UNIFORM
runSpec, pre-specified path         m=5000  rate(p<0.05)=0.05540  band=[0.03986, 0.06014]  mean(p)=0.49365  KS=0.01362  sqrt(m)*KS=0.9629  KS_p=0.3118  UNIFORM
Pearson screenTime vs examScore     m=5000  rate(p<0.05)=0.05360  band=[0.03986, 0.06014]  mean(p)=0.49685  KS=0.01270  sqrt(m)*KS=0.8982  KS_p=0.3953  UNIFORM
arm vs baseline priorScore          m=5000  rate(p<0.05)=0.05040  band=[0.03986, 0.06014]  mean(p)=0.49398  KS=0.01624  sqrt(m)*KS=1.1482  KS_p=0.1432  UNIFORM

HEADLINE, at least one specification significant on null data:
P(at least one specification p<0.05)        0.8733   (131/150)
P(at least one p<0.05 favouring treatment)  0.5467   (82/150)
mean fraction of specifications p<0.05      0.0418
P(pre-specified single test p<0.05)         0.0600   (the honest 5%)

NEGATIVE CONTROL, a real effect must be detected:
power of the pre-specified test 0.9690   (1938/2000)

uniformity verdicts: 6 uniform, 3 deliberately broken streams caught

== Real browser, docs/index.html at 390px and 1280px
node says:    prespecP=0.6553990324 specs=1920 failed=0 sig=119 minP=0.0050276413 medianP=0.4407064268
browser 390,844: SELFTEST:OK viewport=375 controlsFlipped=6 seed=1 prespecP=0.6553990324 specs=1920 failed=0 sig=119 minP=0.0050276413 medianP=0.4407064268 curveMarks=1920 curveRows=22 paths=26 histBars=41 tableRows=22 labelTexts=32
               console messages: 0
browser 1280,900: SELFTEST:OK viewport=1265 controlsFlipped=6 seed=1 prespecP=0.6553990324 specs=1920 failed=0 sig=119 minP=0.0050276413 medianP=0.4407064268 curveMarks=1920 curveRows=22 paths=26 histBars=41 tableRows=22 labelTexts=32
               console messages: 0

== PASS
The page, the statistics, and the claims all check out.

The verify was attacked twice to confirm it can fail.

attack exit code
the null generator leaks a real 0.22 SD effect into every treated student 1, calibration caught it: rate 0.284 outside [0.039859538965116036, 0.06014046103488397], 5 tests failed
tTwoSidedP always returns 0.5 1, 20 tests failed, starting with rate 0 outside [0.039859538965116036, 0.06014046103488397]

In both cases the page was rebuilt before running verify, so the failure came from the statistics rather than from the staleness check noticing an edited source file.

Browser results, Chrome via Playwright against docs/index.html served over HTTP on a port bound to 0 and confirmed by comparing the served bytes to the file on disk:

  • zero console messages at 390px and at 1280px, in light and in dark, including pageerror and requestfailed
  • no horizontal body scroll at either width, and no element in main extending past the viewport except inside the two containers that scroll on purpose
  • the choice controls move the p-value: 0.6554 on the pre-specified path, 0.3985 after switching the sample to grades 9 and 10
  • data-theme overrides the OS setting in both directions: a light stamp under OS dark gives --plane: #f9f9f7, a dark stamp under OS light gives --plane: #0d0d0d

Finding that took a real browser. The page had an unbalanced parenthesis in the histogram code, so the entire inline script failed to parse and the page rendered as static HTML with no numbers in it. Every node test passed the whole time.

Limitations

  • The multiverse is 1,920 specifications, which is far short of all of them. Sample size, the alpha level, one-sided versus two-sided, transformations of the outcome, interaction terms, and measurement decisions made before the data reached the analyst are all fixed here. A real garden is larger. 87.3% is a lower bound on the problem.
  • Cluster and multiplicity corrections are not offered. There is no Bonferroni, no false discovery rate control, and no bootstrap-based joint test of the specification curve as Simonsohn and colleagues describe. The page shows the distribution; it does not offer to fix it.
  • The data is Gaussian and well behaved. No missingness, no attrition, no measurement error, no floor or ceiling effects, no non-normal outcomes. That makes the p-values exactly uniform under the null and it makes the demonstration cleaner than reality. Real data would give the garden more branches than this.
  • The headline is measured on 150 cohorts. The binomial standard error on 0.8733 is about 0.027, so read it as "roughly 87%" rather than as three significant figures. Those 150 cohorts are seeds 1 through 150, so the figure is byte-reproducible across runs but it is still one Monte Carlo sample. The suite asserts only that it exceeds 0.5 and is more than ten times the nominal 0.05.
  • The composite outcome is standardised within whatever sample survives the exclusions, which is one specific defensible convention among several. A different convention would give slightly different numbers for the four composite-outcome branches.
  • prefers-reduced-motion is respected but there is little motion to reduce. The page has no animation beyond a tooltip fade.
  • The specification curve is a 1,920-column plot. It scrolls inside its own container. On a phone the plot window is about 200px wide, which is enough to see the red band at the end but not enough to read the middle without scrolling.

Sources

  • Gelman, A. and Loken, E. (2013). The garden of forking paths: Why multiple comparisons can be a problem, even when there is no "fishing expedition" or "p-hacking" and the research hypothesis was posited ahead of time. Department of Statistics, Columbia University.
  • Simmons, J. P., Nelson, L. D. and Simonsohn, U. (2011). False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science 22(11), 1359-1366.
  • Steegen, S., Tuerlinckx, F., Gelman, A. and Vanpaemel, W. (2016). Increasing transparency through a multiverse analysis. Perspectives on Psychological Science 11(5), 702-712.
  • Simonsohn, U., Simmons, J. P. and Nelson, L. D. (2020). Specification curve analysis. Nature Human Behaviour 4, 1208-1214.

License

MIT. All data on the page is synthetic. No real students, no real school, no real app.

About

Garden of forking paths simulator. Hunt for p<0.05 in a null dataset, then see the whole multiverse of 1920 specifications

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages