diff --git a/docs/_builtin-scorers.md b/docs/_builtin-scorers.md
index 2d06b0b35f..cebb6f1972 100644
--- a/docs/_builtin-scorers.md
+++ b/docs/_builtin-scorers.md
@@ -40,6 +40,18 @@ Inspect includes both text matching scorers as well as model graded scorers. Bel
 
 :::
 
+## When the output doesn't match
+
+These scorers use the `CORRECT` and `INCORRECT` constants, which the default metrics convert to `1.0` and `0.0` (see [Custom Scorers](custom-scorers.qmd#score) for the `Value` types and `value_to_float()`). They differ in how they treat output that does not match the target:
+
+`includes()`, `match()`, and `exact()`
+:   Score a non-matching output `INCORRECT`, so the sample stays in the denominator as a `0.0`.
+
+`pattern()` (and `answer()`, which builds on it)
+:   Score `INCORRECT` when the pattern matches but the captured value is wrong, and `NOANSWER` when the pattern does not match at all. `NOANSWER` also converts to `0.0` by default, so it does not change accuracy; it is a distinct label for "no answer was found in the expected form" rather than "the model answered incorrectly".
+
+Whether a failure to produce output in the expected format should count as a wrong answer (`INCORRECT`) or as no answer (`NOANSWER`) is a judgement call, since failing to follow the requested format is itself an instruction-following failure. See [Scoring Policy](scoring-policy.qmd#verdicts-on-the-model) for the distinction and how it interacts with metrics.
+
 ## Metrics
 
 Each scorer provides one or more built-in metrics. Most report `accuracy` and `stderr`; `exact()` and `f1()` report `mean` and `stderr`; and the perplexity scorers report `perplexity_per_token` and `perplexity_per_seq`. You can override these by passing your own `metrics` to the `Task`:
diff --git a/docs/_quarto.yml b/docs/_quarto.yml
index 9f4b889b7b..b5e98fa35f 100644
--- a/docs/_quarto.yml
+++ b/docs/_quarto.yml
@@ -81,6 +81,8 @@ inspect-docs:
       contents:
         - href: standard-scorers.qmd
         - href: custom-scorers.qmd
+        - text: Scoring Policy
+          href: scoring-policy.qmd
         - text: Model Grading
           href: model-graded.qmd
         - text: Scoring Metrics
diff --git a/docs/custom-scorers.qmd b/docs/custom-scorers.qmd
index f9abc8c13c..ff76cd44e5 100644
--- a/docs/custom-scorers.qmd
+++ b/docs/custom-scorers.qmd
@@ -131,6 +131,8 @@ return Score.unscored(
 
 Unscored samples are skipped by aggregate metrics and epoch reducers and are counted toward `EvalScore.unscored_samples` rather than included as zeros. This works for scalar, dict-valued, and list-valued scorers.
 
+For guidance on *when* to return `Score.unscored()` versus a score or an error, see [Scoring Policy](scoring-policy.qmd).
+
 ## Score Value
 
 `Value` is union over the main scalar types as well as a `list` or `dict` of the same types:
@@ -149,7 +151,13 @@ Next, we'll take a look at the source code for a couple of the built in scorers
 
 ## Models in Scorers
 
-You'll often want to use models in the implementation of scorers. Use the `get_model()` function to get either the currently evaluated model or another model interface. For example:
+You'll often want to use models in the implementation of scorers. The best way of doing this is to specify the grader model using [model roles](models.qmd#model-roles), allowing them to be chosen at runtime. By default, model-graded scorers look for the `"grader"` model role:
+
+```python
+grader_model = get_model(role="grader")
+```
+
+You can also use the `get_model()` function to get either the currently evaluated model or another model interface. For example:
 
 ``` python
 # use the model being evaluated for grading
@@ -246,15 +254,21 @@ def model_graded_qa(
                 explanation=result.completion,
             )
         else:
-            return Score(
-                value=INCORRECT,
+            # The grader failed to produce a parseable verdict. That is a
+            # failure of the measurement instrument, not a wrong answer from the
+            # model, so record it as unscored (excluded from metrics) rather than
+            # charging the model INCORRECT. See Scoring Policy for the rationale.
+            return Score.unscored(
                 explanation="Grade not found in model output: "
                 + f"{result.completion}",
+                metadata={"reason": "grader_parse_failure"},
             )
 
     return score
 ```
 
+The grade-parse-failure branch returns `Score.unscored()` rather than `INCORRECT`: a grader that cannot render a verdict is an instrument failure, not a verdict on the model. See [Scoring Policy](scoring-policy.qmd#model-graded-scorers) for when to score, raise, or return `Score.unscored()`.
+
 Note that the call to `model_grader.generate()` is done with `await`. This is critical to ensure that the scorer participates correctly in the scheduling of generation work.
 
 Note also we use the `input_text` property of the `TaskState` to access a string version of the original user input to substitute it into the grading template. Using the `input_text` has two benefits: (1) It is guaranteed to cover the original input from the dataset (rather than a transformed prompt in `messages`); and (2) It normalises the input to a string (as it could have been a message list).
diff --git a/docs/handling-errors.qmd b/docs/handling-errors.qmd
index 94eb61ca89..202cd0f61c 100644
--- a/docs/handling-errors.qmd
+++ b/docs/handling-errors.qmd
@@ -92,6 +92,8 @@ Consequently, when enabling `retry_on_error` you should do some post-hoc analysi
 
 Some evaluations are designed so that an error during the agent run is itself a meaningful (often failing) outcome — for example, a tool-using agent that crashes after producing partial state, or a benchmark where "the model errored" should count as a scoreable result rather than as missing data.
 
+This page covers the run-level options for tolerating errors. For the complementary authoring decision, namely what a scorer should return for a given failure (a `Score`, a `raise`, or `Score.unscored()`), see [Scoring Policy](scoring-policy.qmd).
+
 The `score_on_error` option causes errored samples to be scored anyway (using whatever `TaskState` was reached before the error), and prevents `fail_on_error` from crashing the eval mid-run:
 
 ``` bash
diff --git a/docs/model-graded.qmd b/docs/model-graded.qmd
index 9c81a2c10a..e385b65833 100644
--- a/docs/model-graded.qmd
+++ b/docs/model-graded.qmd
@@ -169,6 +169,8 @@ Grade extraction binds to the last grade. The default `grade_pattern` (`(?is).*(
 
 Structural delimiters are neutralized. The default templates wrap content in `[BEGIN DATA]` / `[END DATA]` markers. Before formatting the prompt, Inspect rewrites any `[BEGIN DATA]` / `[END DATA]` markers that appear in the model-controlled `answer`, the `question`, the `criterion`, and any `metadata` values (for example to `[END-DATA]`) so a model cannot inject a fake delimiter and smuggle grading instructions into the prompt. The `instructions` you provide are author-controlled and left untouched.
 
+When the grader's verdict still cannot be parsed after these precautions, that is a failure of the measurement instrument rather than a verdict on the model. See [Scoring Policy](scoring-policy.qmd#model-graded-scorers) for how to handle it (bounded grader-only retry, then `Score.unscored()`), and why it should not be scored `INCORRECT`.
+
 ## Reproducible Grading
 
 By default the grader model runs with the provider's default generation settings. In practice this often means a non-zero temperature (e.g. 1.0) and no fixed seed, so grades for borderline answers can change from run to run and across epochs.
diff --git a/docs/scoring-policy.qmd b/docs/scoring-policy.qmd
new file mode 100644
index 0000000000..cfbf7cb2a0
--- /dev/null
+++ b/docs/scoring-policy.qmd
@@ -0,0 +1,169 @@
+---
+title: Scoring Policy
+llms-description: How to route a sample-ending failure in a scorer to the right outcome (return a Score, raise, or Score.unscored()) so that metrics reflect the model under test rather than the run machinery.
+---
+
+## Overview
+
+A scorer's job is to turn what a model did on a sample into a verdict about the model's capability on the task, which Inspect then aggregates into the metrics that summarise the evaluation. In the clean case this is simple: the model under test (MUT) produced an answer, the scorer judges it correct or incorrect, and that verdict counts.
+
+The complication is that not everything which ends a sample is a statement about the model. The model can genuinely fail, with a wrong answer or output that doesn't follow the requested format. But the *run machinery* can also break (an API error, a dead container, a bug in your own code), and where a grader model or other measurement instrument is involved, the *instrument itself* can fail to produce a reading. These are different in kind, and only the first is a verdict on the model.
+
+Distinguishing them matters because each affects the metric denominator differently, so conflating them distorts the headline number (see the [worked example](#the-three-outcomes) below). The organising question throughout is: **whose behaviour does the outcome reflect, the model under test or the run machinery?** A grader model counts as machinery here, because it is part of the measurement instrument, not the subject of measurement.
+
+## The three outcomes
+
+A scorer has three ways to record what happened on a sample, and each routes to different downstream handling:
+
+- **Return a `Score`** is a verdict on the model. It counts in the denominator of the resultant metrics.
+- **Raise an error** signals a machinery malfunction. The sample is marked errored: excluded from metrics, retried if [`retry_on_error`](handling-errors.qmd#sample-retries) is set, counted against the [`fail_on_error`](handling-errors.qmd#failure-threshold) threshold, and scored only if [`score_on_error`](handling-errors.qmd#scoring-errored-samples) is set.
+- **Return `Score.unscored()`** records that the scorer could not render a verdict even though nothing malfunctioned. The sample is excluded from the metrics but counted separately.
+
+The choice has real consequences for the reported numbers. Using `accuracy` as an example, an eval run with one correct answer, one incorrect answer, one sample that errored, and one sample that triggered a content filter on the grader model will report an accuracy of **0.5**:
+
+| Sample | Scorer outcome     | Value | Counts in denominator? |
+|--------|--------------------|-------|------------------------|
+| 1      | `Score(CORRECT)`   | `1.0` | yes                    |
+| 2      | `Score(INCORRECT)` | `0.0` | yes                    |
+| 3      | `raise`            | n/a   | no (errored)           |
+| 4      | `Score.unscored()` | `NaN` | no (unscored)          |
+
+: {tbl-colwidths=[15,35,15,35]}
+
+Only samples 1 and 2 are scored, so `accuracy = 1 / 2 = 0.5`, with `scored_samples = 2` and `unscored_samples = 1`; the errored sample is recorded separately via `sample.error`. Contrast the two ways of getting this wrong:
+
+- Had samples 3 and 4 been scored `0.0`, accuracy would read `0.25`, penalising the model for an infrastructure error and a grader failure it had no control over.
+- Had they simply been dropped with no accounting, the distinction would be invisible, and a run that errored on nine of ten samples could report `100%` over the one that happened to succeed.
+
+The [`score_on_error`](handling-errors.qmd#scoring-errored-samples) run option changes this picture for errors raised during *solving*: rather than dropping the sample, it runs the scorer on whatever partial state was reached and counts the result. If sample 3 had errored mid-solve and its partial state scored `INCORRECT`, it would enter the denominator and accuracy would read `1 / 3`. It does **not**, however, rescue a `raise` from the scorer itself (the scorer is the thing that failed), so it is not a substitute for returning a `Score` when you want a sample counted.
+
+## Choosing an outcome
+
+If you are writing a scorer, the practical question is which of the three outcomes to return for a given failure. A quick test is to ask **what the correct remedial action would be**:
+
+| Correct remedy | Outcome |
+|---|---|
+| Retry the sample and it will probably work | `raise` (transient) |
+| Fix your code, environment, or dataset and re-run | `raise` (defect; must be loud even though a retry won't fix it) |
+| Nothing, because due diligence is done and the instrument can't say | `Score.unscored()` |
+| The model did this to itself | return a `Score` |
+
+: {tbl-colwidths=[60,40]}
+
+The sections below work through each outcome, and how to tell which applies when the cause is ambiguous.
+
+### Verdicts on the model
+
+A failure that is a genuine statement about the model's behaviour is a score, and it stays in the denominator. This includes output that doesn't parse: failing to follow the requested output format is an instruction-following failure, which is a legitimate capability miss.
+
+``` python
+return Score(
+    value=INCORRECT,
+    explanation=f"Expected JSON, could not parse: {state.output.completion}",
+    metadata={
+        "reason": "invalid_response_format",
+    },
+)
+```
+
+Inspect provides two labels for model-fault outcomes. Both map to `0.0` by default and both stay in the denominator, so the choice is about what the label *means* and what downstream tooling can partition on, not about the number:
+
+- **`INCORRECT`**: the model produced an answer that is wrong or unusable (including a format or instruction-following failure).
+- **`NOANSWER`**: the model produced *nothing to grade*, such as an empty completion or a detected refusal.
+
+::: {.callout-note appearance="simple"}
+If you intend to split on `NOANSWER` in a custom metric (for example to report an answer rate), be aware that the default `mean` epoch reducer collapses `NOANSWER` to `0.0` *before* metrics run, erasing the label. Use a value-preserving epoch reducer (see [Reducing Epochs](metrics.qmd#reducing-epochs)) if you need the distinction to survive.
+:::
+
+Never let a model-fault outcome raise; raising removes the sample from the denominator and inflates accuracy.
+
+### Malfunctions
+
+A failure that tells you nothing about the model, because the run machinery went wrong, should `raise`. There are two flavours, and raising is correct for both:
+
+- **Transient**: an API error, a network blip, a container that died. A retry may succeed, and `retry_on_error` will attempt it.
+- **A defect**: a bug in your solver or scorer, a broken environment, a missing or unrunnable ground-truth artifact. A retry won't help, but raising surfaces the problem loudly against `fail_on_error` instead of silently biasing the result.
+
+Raising routes the sample to Inspect's error-handling machinery rather than baking a run-time policy decision into the scorer at authoring time. See [Handling Errors](handling-errors.qmd) for `retry_on_error`, `fail_on_error`, and `score_on_error`.
+
+Give the raise a message that lets a human reading the errored sample attribute it after the fact: what failed, why it isn't a verdict on the model, and a bounded tail of the relevant output or traceback.
+
+::: {.callout-note appearance="simple"}
+Deterministic machinery should not get the unscored escape hatch. Sandbox execution, file I/O, your own runner, JSON your code wrote, and the grader *API call* itself either work or are broken; hence a failure is always transient or a defect, so it should `raise`. `Score.unscored()` is reserved for instruments that are inherently *stochastic* (see below). Unscoring a deterministic failure hides a bug.
+:::
+
+### Unscoreable samples
+
+Some failures tell you nothing about the model, yet nothing malfunctioned either: the run worked, but the measurement instrument could not render a verdict on this sample after reasonable due diligence. The canonical case is a probabilistic instrument such as a grader model, which has an inherent, irreducible per-sample failure rate. Sometimes it emits a parseable verdict; sometimes it refuses or produces an unparsable one, and no retry or code fix will change that for this sample.
+
+For these, return `Score.unscored()`. This produces a `Score` whose `value` is `NaN`, which Inspect excludes from metric computation and counts as `unscored_samples` on the `EvalScore` (see [Unscored Samples](custom-scorers.qmd#unscored-samples) for the API details):
+
+``` python
+return Score.unscored(
+    answer=state.output.completion,
+    explanation="Grader did not return a parseable verdict after 3 retries.",
+    metadata={
+        "reason": "grader_parse_failure",
+        "grader_payload": grader_output.message,
+    },
+)
+```
+
+Always report the unscored rate. A non-trivial rate can invalidate the run regardless of how the individual samples are handled, because it is a fact about your measurement instrument, not about the model.
+
+### Attribution: whose output failed?
+
+The three outcomes above turn on a single question: *whose output* failed to parse or comply.
+
+- **The model's output** failing to parse is a verdict on the model → `INCORRECT` (detect this before invoking a grader where possible).
+- **The instrument's output** failing (a grader model's verdict, or a harness the scorer invokes) is never a verdict on the model → apply the malfunction / unscoreable decision above.
+
+Beware misidentifying the author of a payload. A JSON file written by *your own runner* is instrument output, even though the pipeline started from model output; a JSON blob the *model itself* emitted is model output. The author determines the bucket, not where the data happens to sit.
+
+Where a reliable signal exists, classify from it rather than collapsing every failure into one bucket. For example, bracketing test output with sentinel markers lets a scorer tell "the tests ran and failed" (a verdict → score) apart from "the test process never started" (infrastructure → raise).
+
+## Model-graded scorers
+
+When a model grades the model under test, the grader is the *measurement instrument*, and a failure to parse the **grader's** verdict is an instrument failure, not a verdict on the model. Charging it to the model as `0.0` measures the grader's competence and biases results toward whichever grader you chose. Handle grader-verdict failures in this order:
+
+1. **Constrain the grader's output structurally** rather than regex-parsing free text; prefer a tool call with a schema-validated payload. This removes most format-failure ambiguity at the source.
+2. **Retry the grader only**, bounded, with the model's output held fixed. Note that `retry_on_error` re-runs the *whole sample* including model generation, which re-rolls the answer and can inflate scores, so grader-only retry is a separate loop inside the scorer.
+3. **Use a capable, pinned grader.** A grader too weak to follow its own output format is an eval-validity problem to flag, not something to charge to the model. Pin its generation settings so grades are reproducible (see [Reproducible Grading](model-graded.qmd#reproducible-grading)).
+4. **Only then** return `Score.unscored()` for the residual, and report the grader-failure rate as a first-class number.
+
+The `raise` vs `unscored()` line still applies within grading: the grader *API call* raising or timing out is a malfunction → `raise` (the transport failed, not the verdict), whereas a well-formed-but-unparsable *verdict* after bounded retry is an instrument miss → `unscored()`.
+
+## Recording the reason
+
+Whatever the outcome, set `explanation` to a human-readable message, and record a machine-readable reason in `Score.metadata` so failures are filterable in the log viewer and in analysis dataframes:
+
+``` python
+return Score(
+    value=INCORRECT,
+    answer=state.output.completion,
+    explanation="Model response was not valid JSON.",
+    metadata={"reason": "invalid_response_format"},
+)
+```
+
+A consistent vocabulary makes failures groupable across evals. For example, use `invalid_response_format` and `refusal` for model-fault outcomes, and `grader_parse_failure`, `grader_refusal`, or `grader_no_tool_call` for instrument failures. Keeping the actor in the value (a `grader_` prefix) lets a single query separate instrument failures from model failures.
+
+## Recommended run settings
+
+The scorer decides *what a sample's outcome means*; the run configuration decides *how much failure a run tolerates*. Keep the two separate: return the right `Score` / `raise` / `unscored()` from the scorer, and set the tolerance at the run level.
+
+- Set `fail_on_error` to a proportion that reflects how many machinery failures would invalidate your run. Tolerating a small fraction such as `0.1` is a reasonable default for real runs; the default (`True`) aborts the run on the first error, which is useful during development but brittle for long runs. Setting `fail_on_error` to `False` can produce *highly misleading results* if a significant number of samples fail.
+- Enable `retry_on_error` to recover from transient malfunctions.
+
+``` bash
+inspect eval task.py --fail-on-error=0.1 --retry-on-error=3
+```
+
+Because errored and unscored samples both leave the denominator, a run can report a clean, high accuracy computed over only a fraction of its samples. When reporting or reviewing results, check the coverage (how many samples were actually scored) alongside the headline metric. The counts are recorded on each `EvalScore` as `scored_samples` and `unscored_samples`; errored samples are visible per-sample via `sample.error`.
+
+## See also
+
+- [Custom Scorers](custom-scorers.qmd): writing scorers and the `Score`, `Value`, and `Target` types.
+- [Model Grading](model-graded.qmd): built-in model-graded scorers and how to customise them.
+- [Scoring Metrics](metrics.qmd): metrics, epoch reducers, and custom metrics.
+- [Handling Errors](handling-errors.qmd): `retry_on_error`, `fail_on_error`, and `score_on_error` at the run level.
diff --git a/docs/scoring.qmd b/docs/scoring.qmd
index dbff57da68..37eb28be52 100644
--- a/docs/scoring.qmd
+++ b/docs/scoring.qmd
@@ -9,6 +9,7 @@ Scoring turns the raw `output` a model produces for each sample into a `Score`,
 |---|---|
 | [Standard Scorers](standard-scorers.qmd) | The built-in scorers (text matching, multiple choice, math, model grading, perplexity) and how to choose among them. |
 | [Custom Scorers](custom-scorers.qmd) | Write your own scorers using the `Score`, `Value`, and `Target` types, including scorers that call models or inspect a sandbox. |
+| [Scoring Policy](scoring-policy.qmd) | Route a sample-ending failure to the right outcome (score it, `raise`, or `Score.unscored()`) so metrics reflect the model, not the run machinery. |
 | [Model Grading](model-graded.qmd) | Use another model to grade open-ended answers; customise templates, instructions, grader models, and chat history. |
 | [Scoring Metrics](metrics.qmd) | Built-in metrics, grouping, clustered standard errors, custom metrics, and reducing epochs. |
 | [Multiple Scorers](multiple-scorers.qmd) | Use several scorers together, emit multiple scores from one scorer, and reduce multiple scores into one. |