sample-ratio-mismatch

The SRM Checker Chrome extension is no longer being maintained and has been retired. The website and related documentation will remain available for reference, but the extension itself will no longer be updated, supported, or guaranteed to work.

Command-line guardrail for online experiments. Checks traffic allocation for sample ratio mismatch, runs the metric test that suits each metric's shape, corrects for multiple testing, re-estimates the effect under propensity score matching, and reports one of three verdicts. Validated against the 13.98M-row Criteo Uplift dataset.

End-to-end A/B testing & product monetization pipeline with power analysis, SRM auditing, hypothesis testing (Z-Test/Bootstrap), and CRO revenue impact analysis.

Checkout-redesign A/B test: power analysis, Sample Ratio Mismatch check, bias-corrected two-proportion z-test, bootstrap AOV cross-check, and Bonferroni-adjusted segment analysis on synthetic checkout session data.

Audit completed A/B experiment readouts for SRM, peeking risk, practical significance, guardrail movement, and pre-period imbalance. Zero dependencies. Structured PASS/WARN/FAIL reports in JSON, Markdown, and HTML.

Prompt versioning and A/B testing — plus the measurement showing the two arms are not independent, because they share a cache, a rate limit and a logging pipeline.

Precise Sample Ratio Mismatch calculator.