From Rome's 2026-06-04 paper feedback (F2).
§5.5 reports QFIRE blocking 100% on the strict hipaa_phi/default chains vs single-shot adaptive families (gemma2-generated, already caveated as a human-adversary lower bound). The calibrated chain already bends under the Stage-3 in-the-loop loop (0.37/0.13).
Concerns / improvements
- A 100% result invites "weak attacker" skepticism — consider a stronger or human-crafted adaptive attacker, and/or a transfer/external adaptive set.
- The FPR cost of the strict chain at 100% recall isn't reported next to the 100% — surface it so the recall/over-refusal trade is visible.
Tracking doc: osb/Projects/Research/Prompt-Firewall/paper-feedback-2026-06-04.md.
From Rome's 2026-06-04 paper feedback (F2).
§5.5 reports QFIRE blocking 100% on the strict
hipaa_phi/defaultchains vs single-shot adaptive families (gemma2-generated, already caveated as a human-adversary lower bound). The calibrated chain already bends under the Stage-3 in-the-loop loop (0.37/0.13).Concerns / improvements
Tracking doc:
osb/Projects/Research/Prompt-Firewall/paper-feedback-2026-06-04.md.