feat(cas): complete GrandmaIntent coverage - #2024
Conversation
Extract the legacy Grandma probe subjects into intent stubs, connect S006items to its detector, and enable GrandmaIntent at the OF_CONCERN tier. Add focused regression coverage for stub intent metadata, subject preservation, and detector parity. Co-authored-by: OpenAI Codex <codex@openai.com> Signed-off-by: Enes Deniz <142517728+3nesdeniz@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
This PR completes CAS coverage for the “Grandma” probe family by promoting GrandmaIntent to an active OF_CONCERN intent-driven probe, extracting legacy Grandma subjects into CAS intent stubs, ensuring detector mappings are present, and adding regression tests to keep intent metadata, stub content, and detector parity aligned.
Changes:
- Activate
GrandmaIntentatTier.OF_CONCERNand set itactive = True; fix a substance spelling in legacy probe params. - Add CAS intent stub text files for
S005hateandS006items, and correct theS003product keys stub intent code. - Add a missing
S006itemsdetector mapping and introduce focused tests for stub preservation, prompt conversion, and detector parity.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| tests/probes/test_probes_grandma.py | Adds regression tests ensuring legacy subjects are preserved via stubs, detectors resolve as expected, and prompt conversion retains stub subject text. |
| garak/probes/grandma.py | Corrects substance spelling and activates GrandmaIntent at OF_CONCERN. |
| garak/intents/S003.py | Fixes S003productkeys stub intent code to match the intended leaf name. |
| garak/data/cas/intent_stubs/S006items.txt | Introduces intent stubs representing legacy Substances coverage. |
| garak/data/cas/intent_stubs/S005hate.txt | Introduces intent stubs representing legacy Slurs coverage. |
| garak/data/cas/intent_detectors.json | Adds detector mapping for S006items to ensure detector resolution parity with legacy probes. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
@3nesdeniz Thanks for working on this. Holding off on review since this is still a draft. Mark it ready when you're done and we'll take a look. |
|
Thanks, Patricia. The implementation and focused verification are complete, and the branch is current with main. I've marked the PR ready for review. |
Summary
GrandmaIntentat theOF_CONCERNtierS005hateandS006itemsintent stubs while retaining the existingS003productkeyssourceS006itemsdetector mapping and correct the adjacent S003 stub metadata and substance spellingFixes #1887.
Why this is not a duplicate
As of 2026-08-06, the required duplicate checks found no open PR referencing #1887 and no open PR matching
GrandmaIntentorS006items. This implements the scope Patricia clarified in the issue rather than duplicating another branch or PR.Verification
python -m pytest -q tests/probes/test_probes_grandma.py tests/cas/— 252 passed, 1 skippedpython -m pytest -q tests/probes/test_probes.py -k grandma— 35 passed, 1,321 deselectedblack --check garak/probes/grandma.py garak/intents/S003.py tests/probes/test_probes_grandma.pypython -m json.tool garak/data/cas/intent_detectors.jsongit diff --checkS003productkeys,S005hate,S006items: active atOF_CONCERN, 90 aligned prompts, 30 per intent, and all three detector mappings resolvedA full local suite attempt reached 4,453 passed and 110 skipped. Its two observed failures were unrelated to this change: the optional
soundfile/librosapackages were absent forAudioAchillesHeel. The run was then interrupted after an unrelated Hugging Face Hub download stalled for more than 20 minutes.Scope notes
This does not modify
IntentProbebase behaviour, remove the legacy Grandma probe classes, or hand-editplugin_cache.json; the repository's cache-maintenance workflow handles that file after merge.AI assistance
OpenAI Codex was used for repository navigation, implementation support, and test drafting. The commit includes an AI attribution trailer. This PR is intentionally opened as a draft so the human submitter can review every changed line and the verification evidence before marking it ready.