Potential features: property-based testing, golden scenarios, integration tests -- which matters most to you? #4
shridipavansaber
started this conversation in
Ideas
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
A few bigger features are on the long-term radar but not yet scheduled. Curious which direction is most useful before we commit time to any of them.
Property-based testing (Hypothesis / fast-check)
Instead of fixed examples, automatically generate hundreds of random inputs and find edge cases you would never think to write manually. High value for pure functions with wide input domains (parsers, calculators, validators). Opt-in via config. Requires Hypothesis (Python) or fast-check (TypeScript).
Golden scenario promotion
Scenarios that have caught real bugs get automatically promoted to a persistent "golden set" that always runs -- even if the file changes significantly. Think of it as an evolving regression suite built from your actual bug history, not a static test file.
Multi-run flakiness detection
Run every test suite 3 times and flag any test that fails fewer than 3 times as flaky. Surfaces tests that are hiding intermittent problems. Opt-in because it triples test execution time.
Integration test generation for multi-file changes
When Codex edits several related files in one turn, generate a test that exercises the interaction between them, not just each file in isolation. Harder to get right but closer to how real bugs surface.
Which of these would change how you use tailtest? Or is there something else entirely missing?
All reactions