Skip to content

feat(slop): slop routing policy as a settings section - #398

Merged
sudoTomas merged 1 commit into
mainfrom
feat/slop-routing-settings
Nov 6, 2026
Merged

sudoTomas merged 1 commit into
mainfrom
feat/slop-routing-settings

Conversation

@sudoTomas

@sudoTomas sudoTomas commented Nov 6, 2026 •

Copy link
Copy Markdown
Owner

Summary

What embry0 does with a code job or a finding judged slop moves into a slop_policy section of the settings document: whether and where a code job judged slop is routed, and from which score; the kinds a verdict may name; what the classifiers and the reviewer of a cleanup are told; the heading a slop finding's ticket opens with; and the two Slop-O-Meter levels embry0 acts on (triage's lean and the dungeon). Every default is the old constant, so the default triage prompt is the old one to the byte. This is the second and last pull request for #169; the first (#384) moved the meter's scoring.

Motivation — Intent + Problem

Slop routing kept all of its policy in code (#169, part of the settings-in-the-database rule, #159):

  • Every code job judged slop ran slop-cleanup, and the built-in template is seeded again at every start, so deleting it, the only way to stop the routing, did not stick.
  • The seven kinds were fixed and any other kind was dropped; the classifiers' and the reviewer's words, the ticket heading, the dungeon threshold and the lean level were constants, and the lean level was never read.
  • An organization could not route its cleanups elsewhere, turn the routing off, or describe slop in its own words.

How it works

The section (embry0/settings/sections/slop_policy.py, reference in docs/configuration.md#slop_policy, prose in the new docs/slop-routing.md, linked from docs/slop-meter.md):

Key Default Layers
enabled true organization, repository
route_template slop-cleanup organization, repository
min_score_to_route 0 (every slop verdict) organization, repository
kinds the seven kinds organization
classifier_guide the old CLASSIFIER_GUIDE organization
review_guide the old SLOP_REVIEW_GUIDE organization
ticket_heading Slop, not a bug organization
triage_lean_min_level 4 (Sloppy Joe) organization
dungeon_min_level 2 (Spaghetti Bolognese) organization
  • Validation (422 naming every refused key, through PUT/PATCH /config/settings, embry0 settings apply and set, MCP settings_set*, the Policy tab and an undo): at least one kind, at most 20, each a_word once; guides plain text (lines and tabs) of at most 4,000 and 2,000 characters, not blank; a one-line heading of at most 80 characters without # marks; levels 1 to 5 or null; a score 0 to 100; a template name's shape. History and undo are the document's; the Policy tab lists sections from the API, so the section appears there with no frontend change.
  • Which keys a repository may set. The three that route its own jobs: a person can already name the pipeline of any job of a repository, so they grant nothing new. The guides, the kinds, the heading and the levels are the organization's own definition of slop, its review bar and how its tickets read. The guides and the heading go into agents' system prompts, and a repository's layer can be written by a key with a repository allowlist, so it holds none of them.
  • From the layer order (documented in docs/slop-routing.md#layers): a repository's layer that sets enabled: true routes that repository's jobs judged slop under an organization's enabled: false, and a repository's layer that sets route_template holds that template against a delete or a rename until that layer's key is changed or taken out.

Who reads it, for which repository.

  • A job reads the section for its repository when it starts and keeps it (job_settings.JOB_SECTIONS, as done_when and salvage): triage's prompt, plan_route's route and the reviewer's guide come from it. A job that started before this release reads the defaults, which are what it started under.
  • The watcher reads its stack's repository's section for each analysis (the guide, the kinds) and each draft it files (the heading), through a reader app.py builds per stack (_watcher_slop_policy, pinned in test_watcher_wiring.py).
  • The world reads each repository's dungeon_min_level when it draws the dungeons.
  • A verdict stored earlier is read back with every kind it kept (parse_work_nature(kinds=None)), so one organization's own kind is never dropped by a reader of another.

Triage's prompt. The paragraph after the guide is built from the policy: the heading it weighs, what a verdict does to the route (nothing with enabled: false; from a score with min_score_to_route), the route (the Slopmaster's cleanup for the built-in slop-cleanup, whose graph is seeded at every start; the cleanup pipeline by name for any other, whoever codes it), and the lean, from triage_lean_min_level by the meter's own level names (slop_meter.level_names of the job's repository, read when triage writes its prompt as its reading is). Wrapped at 78 columns, the defaults give the old paragraph byte for byte. The prompt's slots are now filled in one pass, so text an operator wrote (a guide, a heading, a level name) is never read for another slot's marker.

Routing (plan_route). A code job judged slop runs route_template when the policy is on and its slop_score reaches min_score_to_route; otherwise it takes its normal route (triage's pick, else its kind's default), logged as plan_route_slop_not_routed. A template a person named still wins. A job pinned to the full pipeline (a watcher-origin case, #306) keeps review and QA: it takes the slop route only when that route has a review step and a QA step, as slop-cleanup does, else its kind's default; a job that is not pinned may take a lighter route such as quick-fix. The slop route is now its own tier: a template that is gone, invalid, does not take a code job, or lacks the steps a pinned job keeps, is logged (plan_route_template_not_found with slop_route, plan_route_template_invalid, plan_route_slop_template_unfit) and falls to the kind default. Unlike a person's invalid template it never fails the job (before, an invalid slop-cleanup with no code default raised TemplateInvalidError).

The route template and deleting or renaming it.

  • A template that takes a code job (row_behaviour.code_job_problem, from refactor(agents): behaviour from agent and template rows #385's row columns): its jobs are recorded as code jobs (recorded_kind: not the default of another kind, no other job_kind, not a route of specialists only), and its route has a coding step (developer or slopmaster). A write that changes the template a document's readers route to must leave one. That covers a write naming another template and one that takes the key, or the whole section, out of the last document that set it, so its readers fall back to what they inherit or the default: the field's validator checks a value a document of the chain sets, and a new Section.effective_check (run by document.check with the readers' models before and after the write, document.effective_models) checks the fallback to the default, which no field validator sees. The same check runs for PUT, PATCH, their dry runs, an undo and settings apply. Only a change is checked, so a write of another key never trips over a template its readers already route to.
  • Deleting, renaming or unfitting one a document routes to answers 409, naming each document: the deployment's (its default while it sets none, which names slop-cleanup) and each layer that sets the key. The operator points those documents elsewhere first, which is recorded and undoable.
  • No race. A settings write that may change slop_policy reads the templates FOR SHARE in its own transaction (a new write_reads step of the store, write_context.deployment_reads, given the sections the write may change; a write of any other section, its dry run included, reads none and locks nothing), and a template's change locks its row FOR UPDATE before it reads the documents (services/slop_routing.py, a guard PipelineTemplatesRepository.update and delete run in their transaction). Whichever comes second waits and sees what the first committed; two tests hold each order open and prove it.

Smaller pieces. docs/examples/settings.yaml writes a blank line inside a text block as the bare # (the review guide has one). SettingsDocumentsRepository.all takes the caller's connection.

Decisions

  1. Template deletion and rename: refused (409), not carried, plus a runtime fallback. I chose refusal over carrying a rename into the documents, although a rename carries an agent's quest_template (refactor(agents): behaviour from agent and template rows #385): that one is a column in the same transaction, while a settings document is the operator's text, possibly kept in a file that the next embry0 settings apply writes back, so an automatic rewrite would be undone or refused by it. A fallback alone would let a deleted template silently stop the cleanups, which is the failure the issue names. The fallback stays behind the refusal for a template edited in the database by hand.
  2. enabled does not release a template. A document that names a template holds it even with routing off: turning the policy on again writes no route_template, so it would never be checked again.
  3. The boot seed of slop-cleanup stays as it is. refactor(agents): behaviour from agent and template rows #385 made what a built-in row says about its jobs the operator's (written on a first insert only), while its graph and description stay embry0's and are written at every start. Keeping deleted built-ins deleted would need a record of deletions for every built-in, which changes them all, beyond this issue. Routing no longer depends on whether the template exists: an operator turns it off with enabled or routes elsewhere with route_template, and a slop-cleanup seeded again is then unused.
  4. The dungeon threshold lives in slop_policy, as dungeon_min_level. It is where embry0 acts on a meter reading, beside triage_lean_min_level, not how a score is computed. In slop_meter it would have had to stay out of config_version or rescore every repository when it moved.
  5. No xp key. The issue predates feat(progression): read game content from a shipped pack #207: a merged cleanup's experience (20,000, plus 300 per net line removed up to 1,000) is already the progression pack's outcomes.cleaned (feat(progression): game content from a pack, original by default #176), an operator's document with versions and restore. A second home would be one more place for one value.
  6. Kinds are ids; the guide is free text. The guide's default names the kinds in prose wrapped by hand, which no rendering from structured kinds reproduces byte for byte. I did not add a check that the guide names each kind: it compares keys across layers, so a deployment adding a kind would make every organization's own guide read as inherited.

Moved environment variables

None: no variable ever fed slop routing; every key was a constant in the code. Nothing is imported at the first start, nothing is retired, and .env.example is unchanged. There is no .env key to remove after the deploy.

Pattern provenance

N/A — originated in this PR. The section, layers and job snapshot follow embry0's own done_when, salvage and slop_meter sections (#182, #384); the template checks read #385's row columns.

Verification

Automated

  • Full unit suite with Postgres, -n 4 --fail-on-warnings, DOCKER_HOST=unix:///nonexistent.sock: 19009 passed, 21 skipped, 0 failed, on 705b142e (ba34013b adds two test cases only; the touched files and the slop tests: 96 passed) (the 21 skips: the Docker-backed infra tests and the usual six). CI on ba34013b: 19025 passed, 6 skipped, every job green.
  • ruff format --check . and ruff check . (whole tree) clean; mypy embry0: no issues in 658 source files
  • Settings reference and example regenerated; docs tests (links, tables, reference) pass; name denylist clean; gitleaks on the changed files clean
  • No frontend change: the Policy tab lists sections from the API, a template delete's 409 is shown by its existing toast, and no dungeon or lean level is in the frontend

Defaults reproduce origin/main, pinned from c80ba42e, not from the new code

  • test_the_defaults_are_what_the_code_held_before_the_section: the kinds, the template, the two guides (copied as text), the heading, the dungeon level 2 and the lean level 4.
  • test_the_default_prompt_is_the_one_triage_was_given_before_the_policy: the sha256 of the whole triage system prompt as c80ba42e builds it, with tiers and with one default model, and the old paragraph as text.
  • test_the_default_ticket_and_review_are_what_they_were: a slop ticket's opening and the reviewer's section, as c80ba42e wrote them.
  • test_a_job_that_read_no_policy_routes_as_before (passes on main by design) and test_the_policy_s_defaults_route_as_the_code_did.

The acceptance criterion, on a real database (tests/unit/api/test_slop_policy_settings_live.py, real lifespan, store and seeded templates): PATCH /config/settings/slop_policy?owner=acme {"enabled": false}; a job of acme/widgets judged slop, with the settings it reads when it starts, gets full-magnum-opus and the developer, while globex/api's still gets slop-cleanup and the Slopmaster. Undoing the revision routes acme's again. The same file proves the 422s through the API, the 409s on delete, rename and unfit, and both lock orders.

Fail-first. The seven new test files, copied into a clean origin/main tree: 51 failed, 3 passed, each failure with its own message:

  • the section tests (24): "the settings document has no slop_policy section: slop routing is code", and for the template checks "a settings write is not checked against the pipeline templates";
  • the routing tests (7): "... a job judged slop is routed by code", and "an invalid slop route with no code default failed the job";
  • the triage tests (6): "... triage's words about slop are code";
  • the watcher tests (3): "... the watcher's words about slop are code", "each watcher must read its own stack's slop_policy, not None", "app.py has no reader for a stack's slop_policy";
  • the readers tests (3): "... slop routing is code", "the level a meter draws a dungeon from is fixed in the code";
  • the live tests (7): "acme's layer cannot turn slop routing off" (unknown_section), "the 422 names each refused key", "the Policy tab lists the sections the API answers", "the template the default policy routes to was deleted", "nothing guards a template a settings document routes slop jobs to" (both lock orders), "triage is told no level names but the defaults";
  • the world test (1): "acme draws from Sloppy Joe".

The three that pass there pin behaviour that must not change: test_a_job_that_read_no_policy_routes_as_before, test_a_watcher_without_a_reader_reads_as_before, and test_a_reader_that_fails_reads_the_defaults (main never reads a reader; the mutation below shows it fails when a failing reader is not caught).

Mutations (first round), each against the tree then, each caught by its test with its message:

Mutation Caught by
plan_route ignores enabled "slop_policy.enabled: false, so a job judged slop runs its kind's default"
DELETE runs no guard "the template the default policy routes to was deleted"
the default is no holder the same
a rename is not guarded "a template a document routes to was renamed"
a write reads the templates without FOR SHARE "the delete waits for the settings write that read its template"
a delete locks no row FOR UPDATE "the settings write waits for the delete of the template it names"
a write's template is not checked the section test's "the deployment's write of ['route_template'] was taken"
an unchanged stored template is checked again test_only_a_template_the_write_changes_is_checked (422 on the unrelated write)
an unfit slop route runs anyway "a research template never runs a code job"
a repository may set classifier_guide "a repository's layer, which a repository-scoped key writes, puts no text into a system prompt"
the slots are filled one replace after another test_an_operator_s_text_is_never_read_for_a_slot
triage is given the default level names "triage leans by the meter's own names"
the watcher parses with the default kinds "the verdict keeps the stack's kinds, and only those"
the watcher files under the fixed heading "the ticket opens with the stack's heading"
a failing policy reader is not caught "a policy that cannot be read never stops a tick"
the reviewer hears the default guide "the reviewer is told the guide its job read"
the world draws every repository at level 2 "acme draws from Sloppy Joe"

The first run of these found one gap: triage given the default level names passed every test, since no test gave the node other names. test_triage_is_built_from_the_policy_its_job_read now gives it a repository's own names, and test_triage_leans_by_the_level_names_its_repository_reads reads them from a real layer.

Review round (two defects, five test gaps, three wording and scope fixes). Each new test run against the previous head e4356ac5:

Fix Test On e4356ac5
A job pinned to the full pipeline keeps review and QA when judged slop test_a_job_pinned_to_the_full_pipeline_keeps_review_and_qa_when_judged_slop "a job pinned to the full pipeline judged slop ran quick-fix: ['developer', 'reviewer']"
Taking route_template out checks the default it falls back to test_taking_route_template_out_checks_the_template_its_readers_then_route_to (key out, section null, section left out) "... left the deployment routing to the default slop-cleanup, which is gone" (3)
The same for a layer test_a_layer_that_takes_route_template_out_is_checked_for_what_it_inherits "the organization's layer took route_template out and now routes to slop-cleanup, which is gone"
The same through the API, tidy-up stays held test_taking_route_template_out_never_leaves_a_template_that_is_gone "'slop_policy:\n enabled: true\n' left the deployment routing to the default slop-cleanup, which is gone"
Only a write that may change slop_policy locks the templates test_only_a_write_that_may_change_slop_policy_waits_on_a_template "a write of another section, and its dry run, take no lock on the templates"
Triage names the route it is sent test_triage_is_told_the_route_its_policy_names "triage is told every slop route is the Slopmaster's cleanup"
A write that leaves the route as it was is taken while its default is missing test_a_write_that_leaves_the_route_as_it_was_is_taken_while_its_template_is_missing passes by design; mutation below

The reviewer's two tests, on this head: both pass (on e4356ac5 both fail, with "a pinned watcher job judged slop ran quick-fix: ['developer', 'reviewer']" and "the deployment's document now routes to its default slop-cleanup, which was deleted"). The default triage prompt's digest test passes unchanged.

The five test gaps, and two more from the re-review (N03, N08), each a test that passes on e4356ac5 and its mutation, now caught:

Mutation Test Caught with
M06: the write's template SELECT drops job_kind test_a_template_whose_row_records_another_kind_is_refused "a template whose row records research jobs was taken as a slop route"
M09: proposals read a stored verdict with the default kinds test_a_stored_verdict_is_read_back_with_every_kind_it_kept "an organization's own kind is read back as stored"
M10: a dry run reads no templates test_a_dry_run_checks_the_template_as_a_write_does "a dry run passed a template the write would refuse: 200"
M12: the guard ignores repository layers test_a_repository_layer_holds_the_template_it_routes_to "a template a repository's layer routes to was deleted"
M19: an undo reads no templates test_an_undo_that_would_route_to_a_template_that_is_gone_is_refused "an undo put back a route to a deleted template: 200"
N03: the review step is no longer required of a pinned job's route test_a_job_pinned_to_the_full_pipeline_keeps_review_and_qa_when_judged_slop (a route with developer and QA, no review) "a job pinned to the full pipeline judged slop ran checked-fix, which has no review step"
N08: the field validator compares against the stored value alone, not the default its readers route to test_naming_the_default_it_already_routes_to_is_taken_while_that_template_is_missing "writing out the default its readers already route to was refused"

And the fixes' own mutations, each caught: the pinned check removed ("... ran quick-fix: ['developer', 'reviewer']"); effective_check not wired ("... left the deployment routing to the default slop-cleanup, which is gone"); the hook judging an unchanged default ("{'slop_policy': {'min_score_to_route': 40}} was refused for a template it does not change"); every write reading the templates FOR SHARE ("a write of another section, and its dry run, take no lock on the templates"); every route called the Slopmaster's ("triage is told every slop route is the Slopmaster's cleanup").

Deploy notes

  • No migration, no new environment variable, no image rebuild: nothing under embry0/sandbox/, safety/, agents/ or execution/ changed.
  • The first start reads the section's defaults, which route and prompt as before; nothing is imported.
  • A job running across the deploy keeps what it read: one that started before reads the defaults.
  • Behaviour changes: DELETE or a rename of slop-cleanup (or of a template a document routes to) now answers 409 instead of deleting it; an invalid slop route with no code default now takes the legacy plan instead of failing the job; and a settings write that would leave its readers on a template that is gone answers 422.
  • embry0/services/slop.py and embry0/workflows/row_behaviour.py are in the progression rollup's FOLD_MODULES, so the rollup is rebuilt once at its next read (one read of the history).

Out of scope / follow-ups

  • Keeping a deleted built-in template deleted across restarts (decision 3): a change to every built-in, not to slop routing.
  • The Slopmaster's own system prompt is its agent row's, already editable as an agent definition.

Design + Plan

Anti-goals respected

  • No new npm/Python dependency
  • No silent default flip: every default is the constant it replaces, proven above; the two behaviour changes are listed under Deploy notes
  • No secrets in diff
  • No emoji in code

Fixes #169

@sudoTomas
sudoTomas force-pushed the feat/slop-routing-settings branch from e4356ac to 705b142 Compare November 6, 2026 18:55
What embry0 did with a code job or a finding judged slop was code.
Every code job judged slop ran slop-cleanup, seeded again at each start
so deleting it did not stick. The kinds a verdict could name, what the
classifiers and the reviewer were told, the heading of a slop finding's
ticket and the meter levels triage leaned at and dungeons appeared from
were constants, and the lean level was never read.

Add a slop_policy section: enabled, route_template, min_score_to_route,
kinds, classifier_guide, review_guide, ticket_heading,
triage_lean_min_level and dungeon_min_level. Each default is the old
constant, so the default triage prompt is the old one to the byte. An
organization may set every key, a repository only the three that route
its own jobs: the guides go into system prompts, which a key with a
repository allowlist must not write. A job reads the section when it
starts, the watcher its stack's on each analysis and filing, the world
each repository's dungeon level when it draws.

Triage's paragraph is built from the policy: the heading it weighs, what
a verdict does to the route, the route by name unless it is the built-in
cleanup, and the lean, named with the meter's own level names. Its slots
are filled in one pass, so an operator's text is never read for a
marker. plan_route sends a code job judged slop to the policy's template
when routing is on and the score reaches the threshold. A job pinned to
the full pipeline keeps review and QA: it takes the slop route only when
that route has both. A template that is gone, invalid or unfit is logged
and the job takes its kind's default, and never fails.

A write that changes the template a document's readers route to, by
naming another or by taking the key or the section out so they fall back
to what they inherit or the default, must leave one that records code
jobs and has a coding step; a section's new effective_check sees the
default fallback, which a field validator never does. Dry runs and undo
are checked the same way. Only a write that may change slop_policy reads
the templates, FOR SHARE in its transaction. Deleting, renaming or
unfitting a template a document routes to answers 409 naming the
documents; the change locks its row FOR UPDATE before it reads them, so
neither side races the other. The built-in templates are still seeded at
every start: routing no longer depends on whether the template exists.

No variable fed these keys, so nothing is imported. A cleanup's
experience stays the progression pack's outcomes.cleaned (#176).
METER_DUNGEON_LEVEL moves into the policy beside the lean level: both
are where embry0 acts on a meter reading, not how a score is computed.

Fixes #169
@sudoTomas
sudoTomas force-pushed the feat/slop-routing-settings branch from 705b142 to ba34013 Compare November 6, 2026 19:25
@sudoTomas
sudoTomas merged commit a0f766c into main Nov 6, 2026
13 checks passed
@sudoTomas
sudoTomas deleted the feat/slop-routing-settings branch November 6, 2026 19:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(slop): Slop-O-Meter scoring and slop routing as settings

1 participant