Hi. I build ast-guard, a zero-dependency deterministic detector for
structural reward hacking via pure AST analysis. No ML, no LLM,
<10ms/scan, Python stdlib only.
I evaluated on MALT standalone (81k samples) and built a 36-pair
structural ground-truth benchmark (100% F1, no LLM in labeling).
Your DistilBERT classifier and my AST checks are orthogonal signals.
You catch semantic patterns in CoT traces; I catch structural patterns
in code (hardcoded if-chains, eval obfuscation, extensional enumeration).
Combining both as an ensemble could push detection above either alone,
with AST providing an interpretable second opinion.
Would you be open to running a quick comparison on the same MALT subset?
Repo: https://github.com/Nick-is-building/ast-guard
Hi. I build ast-guard, a zero-dependency deterministic detector for
structural reward hacking via pure AST analysis. No ML, no LLM,
<10ms/scan, Python stdlib only.
I evaluated on MALT standalone (81k samples) and built a 36-pair
structural ground-truth benchmark (100% F1, no LLM in labeling).
Your DistilBERT classifier and my AST checks are orthogonal signals.
You catch semantic patterns in CoT traces; I catch structural patterns
in code (hardcoded if-chains, eval obfuscation, extensional enumeration).
Combining both as an ensemble could push detection above either alone,
with AST providing an interpretable second opinion.
Would you be open to running a quick comparison on the same MALT subset?
Repo: https://github.com/Nick-is-building/ast-guard