[luca-belli]

Hotfix for 2.0.x. release/2.1 has the same code and gets this with the next main sync.

Problem

vera pipeline (and vera score / vera pool) on HFO printed a KeyError traceback during scoring:

personas file …/data/HFO/personas_HFO.tsv has no 'Short Current Suicide Risk Level' column

The run itself finished, but HFO never got a risk-level breakdown (scores_by_risk.json and its chart). The breakdown grouped conversations by a hard-coded SI column, with SI's level order.

Fix: the target's data says how to break it down

• Manifest section. Each target's manifest.json gets an optional risk_levels section, with the persona column and its levels in order:
  • SI: {"column": "Short Current Suicide Risk Level", "order": ["None", "Low", "High", "Immediate"]} (unchanged behavior)
  • HFO: {"column": "Active Life Threat", "order": ["N/A: No harm", "Absent", "Present"]}
• Reading it. load_risk_levels(personas_path) reads it from the manifest beside the personas file, which must list that file. Scoring and the charts use its order.
• No section. Without risk_levels, vera score and vera pool print one line and skip the breakdown, with no traceback.
• No target knowledge in code. RISK_LEVEL_ORDER and RISK_LEVEL_COLUMN are removed.
• Why Active Life Threat for HFO: its three values match the HFO rubric's Danger Tier Classification (No Danger / Imminent Danger Absent / Imminent Danger Present). Every Present persona is also Imminent on Harm imminence, but only half the imminent ones are Present, so it's the tier, not just the timing. It's also the column HFO's manifest already copies into results.csv.

Behavior change: a custom personas file passed to --personas with no target manifest beside it now skips the breakdown, even if it has SI's column. It's in the CHANGELOG entry.

docs/targets.md documents the new manifest field; docs/scoring.md is updated. The roadmap's longer-term plan (take the risk level from a rubric question) is unchanged; this is the interim fix.

Checks

• ruff and pyright clean on score/ and its tests

• SI levels and order unchanged, so SI scores and charts are identical; run records don't checksum manifests, so old config.json files stay valid

• New and updated unit tests:

  • reading risk_levels (present, missing, or a manifest that lists other files);
  • HFO grouping and order;
  • each target's personas file uses exactly its manifest's levels.

  They're written but not run locally (CI runs them).

Worth confirming with the HFO rubric authors that Active Life Threat is the breakdown they want.

🤖 Generated with Claude Code

[luca-belli]

that feels something that the TSV should know about not the code

[luca-belli]

Agreed, moved in 2f62d4c. Each target's manifest.json now has an optional risk_levels section with the persona column and its levels in order:

"risk_levels": {"column": "Active Life Threat", "order": ["N/A: No harm", "Absent", "Present"]}

Scoring reads it from the manifest beside the personas file (it must list that file). Without it, there is no breakdown, just a one-line note. The code no longer knows any target's column or levels: RISK_LEVELS, RISK_LEVEL_ORDER and RISK_LEVEL_COLUMN are gone. One consequence: a custom personas file passed to --personas with no manifest beside it now skips the breakdown, even if it has SI's column. That's in the CHANGELOG entry.

[jgieringer]

Thanks for the fix!