Add experiments framework for compiler perf work - #13
Merged
Conversation
Instrument the typechecker with traceMarker calls at three levels: - Per-declaration: wraps each case in typeCheckAll (TypeChecker.hs) Format: "tc ModuleName kind:name start/end" - Per-phase: wraps infer vs constraint-solve in typesOf (Types.hs) Format: "tc-phase ModuleName bindName infer|solve start/end" - Per-constraint: wraps each entails call in the solver (Entailment.hs) Format: "tc-entails ClassName start/end" (recursive, shows full call stack) The markers use traceMarker (pure, zero cost when not profiling). GHC 9.6 enables eventlog by default — no build flags needed. Includes analysis scripts: - debug/eventlog.js: text report with declaration + phase breakdown - debug/eventlog-speedscope.js: Chrome trace format for flamegraph - debug/README.md: profiling workflow docs Initial profiling of pr-admin identified Data.Record.HasField as 81.7% of entailment time (35s/43s). See experiments/experiment_queue.md. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Entailment start markers now include abbreviated type arguments: "tc-entails Data.Record.HasField 'name' Int Record start" - After solving, emit instance marker: "tc-entails-instance Data.Record.HasField hasFieldRecord" - Move tracing into recursive solve/go so sub-constraints nest properly - Converter: attach type args as Chrome trace `args` (visible on click), emit instance resolution as instant events - Add --min-ms flag to converter to skip short entailment events (--min-ms 1 reduces profile from 138MB to 28MB) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The previous approach used fmap (\r -> traceMarker endTag r) which fires the end marker when the result Expr is demanded, not when the solve action completes. With lazy StateT/WriterT this causes end markers to fire out of order, making timing data and nesting in profile.json completely unreliable. Fix: emit let !_ = traceMarker endTag () before each return in the three solver branches (Solved, Unsolved, Deferred). The bang pattern forces the marker in the monadic sequence after the solving work completes. Also adds debug/analyze-hasfield.js for detailed HasField entailment analysis from eventlog data. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The entailment solver's fundep enforcement unifies inferred types with constraint types, but for wide row types (e.g. 667-field Translations record), both sides are often structurally identical. Adding an eqType guard before unifyTypes avoids O(n) row alignment on identical types. Results on pr-admin (1758 modules): full: 74.1s -> 62.6s (-15.5%) nochange: 1.24s -> 1.29s (within noise) prelude: 5.25s -> 5.44s (within noise) leaf: 2.03s -> 2.00s (within noise) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Rename debug/eventlog-speedscope.js -> debug/eventlog-chrome-trace.js
to reflect that we primarily use chrome://tracing, not speedscope
- Add `exp profile <id> --phase before|after` command that:
- Runs an app-only eventlog build of pr-admin (clears Restaumatic.*
only, keeps deps cached for smaller traces and faster profiling)
- Converts to chrome trace JSON via eventlog-chrome-trace.js
- Outputs to experiments/<id>/<id>-profile-{before,after}.json
- Add experiments/.gitignore for profile JSONs (100-150MB)
- Update CLAUDE.md, SCHEMA.md, scripts/README.md, debug/README.md
with the new workflow and renamed script
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
experiments/directory with structured framework for tracking compiler performance optimization experimentsrun-profile.sh), lifecycle driver (expscript), baseline management, and per-experiment schema (EXPERIMENT.md, TASK.md, HANDOFF.md, results.md)CLAUDE.mdwith guidance on performance work, measurement scenarios, and noise disciplineDetails
The framework provides:
LESSONS.md) to prevent re-attempting known dead endsTest plan
purs compile $(spago sources)withset -fcorrectly bypasses shell glob expansionexp runagainst an existing experiment branch to verify end-to-end flow🤖 Generated with Claude Code