Skip to content

feat(slop): the Slop-O-Meter's scoring as a settings section - #384

Merged
sudoTomas merged 4 commits into
mainfrom
feat/slop-meter-settings
Nov 6, 2026
Merged

sudoTomas merged 4 commits into
mainfrom
feat/slop-meter-settings

Conversation

@sudoTomas

@sudoTomas sudoTomas commented Nov 5, 2026 •

Copy link
Copy Markdown
Owner

Summary

The Slop-O-Meter's scoring policy moves into a slop_meter section of the settings document: file weights, vendored and test directories, the pace and cap constants, the small-repository rule, the outlier thresholds, the judge's guidance, the five levels' bounds and names, the owner's bots and AI agents, and how often and how hard it reads. Nothing written scores bit for bit as before. Each score row records the version of the settings it was computed with, and a settings change rescores only the repositories it touches, within five minutes. This is the first of two pull requests for #169 (the meter); slop routing (slop_policy) comes in the second, which finishes the issue.

Motivation — Intent + Problem

The meter kept all of its policy in code (#169, part of the settings-in-the-database rule, #159):

  • An organization could not say what counts as code, how fast its team attends to it, or where its level bands fall.
  • Changing any constant silently rescored every repository, and nothing recorded which rules a score was computed with.
  • Its reading pace was three environment variables, and an owner's own bots (fix(slop): count embry0's own commits as a bot in the Slop-O-Meter #184) sat on the organization row, a fourth mechanism.

How it works

The section (embry0/settings/sections/slop_meter.py, reference in docs/configuration.md#slop_meter, prose in docs/slop-meter.md):

Group Keys
When it reads enabled, interval_hours (deployment only); api_calls_per_tick, rate_reserve
What a line weighs weights (glob → weight, merged glob by glob), excluded_dirs, test_dirs, test_files, test_marks, test_weight
Size dampening bootstrap_lines, complexity_weight
Attention commit_hours, subcommit_hours, coauthor_shares, ai_coauthor_share, event_hours, lines_per_hour, weekly_hours_cap
Core team core_min_commits, core_full_commits, small_repository_in_full (embry0's departure from the published algorithm, now data)
Outliers outlier_min_lines, outliers_per_year, judge_prompt
Levels level_bounds (four), level_names (five)
Identities bots, ai_agents (each layer adds to the list it inherits)
  • Defaults are the old constants. The section is the only home of the defaults; weights.py, score.py and judge.py read them from it. The judge's answer form stays in code and is said after the guidance, so the default system prompt is the old one character for character.

  • Layers. An organization may set every key but enabled and interval_hours; a repository may set what counts as code in its layout (weights, excluded_dirs, test_dirs, test_files, test_marks). weights merges glob by glob. bots and ai_agents add to what each layer inherits, through a new Section.added_keys for list fields (union, each item once without regard to case, no layer takes one out).

  • Counts across layers. A write counts a merged or added key with what it inherits (the defaults, the deployment's, the organization's): one over the key's bound is a 422 naming the key and the count. weights over its 79 defaults counts too.

    • When a document below grows later, a layer read past the bound keeps the most specific entries, with a warning (settings_layer_entries_left_out): its own, then its organization's, then the deployment's, then the default's, each in its order. Two repositories of one organization therefore agree on the organization's globs, and the read never falls back to the inherited list alone.
    • A layer left over the bound that way can still be written. Only a write that makes the merged total larger than readers read before is refused, so it may change an unrelated key or take entries out. The same holds for the broad-glob bound.
    • I chose that over refusing the wider write. Refusing would tie every deployment write to every organization's layer, so an admin could not add a deployment-wide bot because of one organization's list. The store's preview would also have to resolve each layer for a length rule alone. excluded_dirs and the test_* lists replace what they inherit, so a repository can take back a directory the default excludes.
  • Globs. A glob without / matches a file's name in any directory, one with / its path from the root; *, ? and ** as usual; matching ignores case, so globs are written in lowercase (a leaf module, embry0/services/slop_meter/globs.py, checks, compiles and indexes them).

  • Weigh paths by specificity. Where several globs match, the most specific wins (globs.specificity):

    1. a glob with a / over any without one, so gen/**: 0 weighs .py, .proto and .graphql under gen/ alike;
    2. between two with a /: more segments free of wildcards, then fewer ** segments, then fewer other segments with a wildcard, then more segments, then more literal characters, so src/main.py beats src/**/main.py and a/b/*.ts beats a/**/b/*.ts;
    3. between two without: more literal characters (*.min.js over *.js);
    4. then alphabetical order.

    The generated files (*.min.js, *.d.ts, …) are weights entries at 0, which is how the defaults stay exact.

  • Weighing cost. GlobIndex files each glob once (globs.bucket) and checks a path only against the globs that could match it. A path is weighed once a run (FileRules keeps the weights), and the history, the outliers to judge and the scores are each built in asyncio.to_thread.

    A glob… is checked against
    naming one file that name
    of one name with an extension (*.py, test_?*.py) names with that extension
    of one name with three literal characters in a row (*fixture*) names holding them (a trigram index)
    with a literal first directory (src/**) paths under it
    with a literal segment (**/vendored/**, */gen/*.py) paths holding that segment
    whose last segment has an extension (*/*.py) names with that extension
    with three literal characters in a row (**/*_gen*) paths holding them

    Anything else is broad (*/*, *x*) and checked against every path. A write may hold at most 16 broad globs in weights and 16 in test_files, counted with what it inherits. That is checked on writes only, so a document below that grows later costs time, never the key. A write that leaves no more broad globs than readers read before is taken.

    On 100,000 paths with 900 extra globs: about 117,000 checks and 0.3 s for literal-head, wildcard-head-with-segment and wildcard-head-with-extension globs, and 0.6 s for name globs with inner text. Before this round these took 90 million checks and 7 to 11 s.

  • Validation. 422 names every refused key (a glob the matcher cannot read, a weight outside 0 to 10, a duplicate directory, bounds not rising, five names not distinct, a ramp that does not rise, …), through PUT/PATCH /config/settings, embry0 settings apply, MCP settings_set* and the Policy tab. History and undo are the document's. The Policy tab lists sections from the API, so the section appears there with no frontend change.

The loop (embry0/services/slop_meter/loop.py):

  • A tick reads the section from the same snapshot as its enrollment and judge model, and scores each repository with the section as that repository reads it.
  • config_version is a 16-hex digest of every key but enabled, interval_hours, api_calls_per_tick, rate_reserve and judge_prompt (keys sorted, values as dumped to JSON). It is stored on each score row (migration 144) and shown in GET /repos/{repo}/-/slop-meter as reading.config_version.
  • A settings change rescores only the repositories it touches. The schedule looks at the settings every five minutes, as the leads loop does, instead of sleeping the interval.
    • A full run (fetch, history, forge events, judgments, scores) happens when the interval has passed, or at once when the meter was just turned on.
    • Between full runs, each look compares every enrolled repository's stored score version (SlopMeterRepository.versions()) with the version it reads now. Only the ones that differ are rescored (rescore_changed): from their rows and mirrors as already read, with no fetch, no forge call and no judgment. The full run keeps its time.
    • A repository still reading its history is left to its catch-up. A version whose rescore failed is not retried every look (an in-memory record per version). An outlier the change makes new is judged at the next full run.
    • With enabled: false a run only removes what left the scope, once an interval (before, SLOP_METER_INTERVAL_SECONDS=0 pruned once at start and stopped).
  • Forge budget. A credential's budget is the deployment's api_calls_per_tick and rate_reserve, one ApiBudget per credential (fix(slop): keep the Slop-O-Meter's forge API budget per credential #185). Each organization reads through a ShareBudget of it: at most min(its calls, the deployment's), stopping at max(its reserve, the deployment's). An organization's values narrow only its own share and never lower another organization's on the same credential. Only the credential's own budget running out keeps the meter on its catch-up cadence, as before.
    • A repository stopped only by its own organization's share keeps its previous score, or stays unscored, and waits for the next run at the interval. A settings change does not rescore it from that short read either.
    • Its events are stored and its listings' cursors move, but each run starts reading at the first listing again. A share too small for one whole read (1 to 3 calls in the tests' shape) therefore never reaches the reviews, and a score from it would look complete; 4 or more is scored.
    • fix(slop): the meter never scores repositories when many share one credential #396 (filed separately, reproduced on main: many repositories on one credential) is related. Keeping the reader's place between runs, as fix(slop): the meter never scores repositories when many share one credential #396 needs, would also let a small share finish over several runs.
  • Bots and AI agents. bots are matched whole (as fix(slop): count embry0's own commits as a bot in the Slop-O-Meter #184's). ai_agents are an address up to and including its @ (aider@, any domain) or a whole address; any other prefix is refused, so dev cannot make every devon@ a bot. Both are applied when scoring, so history read before they were listed is corrected without reading it again.
  • Until the import succeeds (a first start whose environment import failed), the meter reads SLOP_METER_INTERVAL_SECONDS, _API_CALLS_PER_TICK and _RATE_RESERVE as the release before did (meter_reader(store, config), as the budget's readers do, feat(budget): unattended-job caps and continue policy as settings #171). A meter that was off stays off. The budget policy retries the import while the orchestrator runs (the import is one transaction for every section).

slop_meter_bots moves into the organization's layer as slop_meter.bots (a MovedSetting, as #170 and #171 moved theirs):

  • Copied once at the first start, and written by PATCH/POST /orgs, which still take it until 0.2.0 with a deprecated note, through one cleaning (MovedSetting.write).
  • The cleaning strips, drops blanks and keeps each name once without regard to case. An entry the section cannot hold (a tab, a line break, a zero-width character, too long) is left out and logged by position (slop_meter_bots_entries_dropped) instead of refusing the list. So a value main accepted, such as ["release-bot", "Release-Bot"], is never refused.
  • POST /orgs checks each moved value against its section before registering anything, so it never registers an owner and then answers 422.
  • The row is read through BudgetView.moved_values until the copy succeeds.

Smaller pieces.

  • The settings document's dump_yaml writes a text of several lines as a literal block (|-), so the example settings file and the env import write the judge's prompt as it reads. No other section has a multi-line value, and the round trip is tested for awkward texts. Progression packs keep their own writer (progression_pack_store.dump_yaml), so their stored texts and versions are unaffected; a test pins it.
  • LEAN_LEVEL (slop_meter/context.py) was never read, so it is deleted rather than wired here. Triage's lean is routing policy: the routing pull request adds slop_policy.triage_lean_min_level.

Choices I made that a reviewer may want to revisit:

  1. Five levels, fixed in number. The bounds and names are data, the count is not. The repository page's tones, the dungeon threshold (METER_DUNGEON_LEVEL = 2) and the routing PR's triage_lean_min_level all read level numbers. Allowing another count later is an additive schema change.
  2. Flat keys, not pace: and caps: objects. A layer replaces a key whole (spec 2026-10-29, L2), so an organization can change lines_per_hour without restating six other numbers. weights is the one mapping, merged entry by entry.
  3. test_marks beside test_files. The old rule for a.test.ts allowed exactly one extension after the mark, which no glob can say: *.test.* would also count a.test.helper.ts. A list of words kept the defaults exact.
  4. judge_prompt is not in config_version. Judgments are kept per commit and never asked again, so new guidance rescores nothing until another commit is judged.
  5. interval_hours is at least a quarter hour. An imported SLOP_METER_INTERVAL_SECONDS below 900 becomes 0.25.
  6. bots and ai_agents add up across layers; excluded_dirs and the test_* lists replace. A deployment-wide CI bot should always count, and the first-start copy writes every organization's old list into its layer, so replace semantics would have cut the deployment's list off from those organizations. A directory list has to be able to take an entry back (a repository whose own code lives in deps/), so those replace, as their descriptions say.
  7. The loop compares the stored score version, not an in-memory record. So a score stored with another version, or with the empty version of a row scored before migration 144, is rescored, and a change made while the orchestrator was down or written by another process is seen at the next look. A repository with no stored score at all is skipped on purpose: it has never been scored, so the full run that reads its history scores it. An in-memory record per version only stops a failing rescore from retrying every look. Full runs still score every caught-up repository, as before, so a restart's first run behaves as it always did. config_version's docstring no longer claims stability across releases: a release that adds a scored key changes every digest, and its first run records the new ones.

Fate of each SLOP_METER_* variable

Variable Fate
SLOP_METER_INTERVAL_SECONDS Import source of slop_meter.interval_hours (seconds / 3600, at least 0.25), and 0 of slop_meter.enabled: false. Read once at this release's first start, then ignored; remove after the deploy is verified
SLOP_METER_API_CALLS_PER_TICK Import source of slop_meter.api_calls_per_tick. Read once, then ignored; remove after deploy
SLOP_METER_RATE_RESERVE Import source of slop_meter.rate_reserve. Read once, then ignored; remove after deploy
SLOP_METER_REPOS, SLOP_METER_EXCLUDE Unchanged: already only the import source of background.slop_meter (#165)
SLOP_METER_CATCHUP_SECONDS Stays a variable: the process's pacing while a history is still being read
SLOP_METER_DATA_PATH Stays: a container path
SLOP_METER_GIT_TIMEOUT_SECONDS Stays: a process limit
SLOP_METER_COMMITS_PER_TICK, SLOP_METER_EVENT_PAGES, SLOP_METER_REVIEWS_PER_TICK, SLOP_METER_LOGINS_PER_TICK, SLOP_METER_JUDGMENTS_PER_TICK Stay: per-tick bounds of one repository's work; what that work may cost is api_calls_per_tick, rate_reserve and the budget section

The three imported variables are gone from .env.example. organizations.settings.slop_meter_bots is copied into each organization's layer and then read by nothing; PATCH /orgs takes it until 0.2.0.

Pattern provenance

The scoring is still the clean-room reimplementation of Slop-O-Meter's published slop score algorithm (docs/slop-score-algorithm.md), credited in docs/slop-meter.md; no code was read or copied. The section, layering, env import and moved-setting mechanics follow embry0's own context_audit (#170) and budget (#171) sections. Everything else originated in this PR.

Verification

Automated

  • Full unit suite with Postgres, -n 4 --fail-on-warnings, DOCKER_HOST=unix:///nonexistent.sock, on the branch rebased onto main after refactor(agents): behaviour from agent and template rows #385 (f63f5802): 18941 passed, 21 skipped, 0 failed. CI on it: 18956 passed, 6 skipped, every job green. Before refactor(agents): behaviour from agent and template rows #385 landed, the only failures were the 35 checks that the migration list has no gap, from the missing 143.
  • ruff format --check . and ruff check . (whole tree) clean; mypy embry0: no issues in 654 files
  • cd frontend && npm run lint (0 errors, the 5 existing warnings), npx tsc -b clean, npm run test: 326 files, 2666 tests passed. The only frontend change is the reading.config_version field on the API type
  • Settings reference and example regenerated (--check both clean); name denylist clean; gitleaks on the changed files clean

Nothing written scores exactly as before

  • test_the_fixture_scores_exactly_as_the_code_before_the_section_did: a 240-commit fixture (core and drive-by authors, a bot, squashes, human and AI co-authors, judged outliers, ignore lists, two areas, forge events) gives the pinned score, level, name and summed cost, capacity and slop per scope. The numbers were computed on 65cf0eaf with the same builder and match to the last bit.
  • test_the_default_document_scores_the_fixture_identically_to_the_old_constants: the same with an empty stored document, and with a document that writes every key at its default, through the snapshot read a tick makes.
  • test_every_path_weighs_what_the_rules_before_the_section_weighed_it: 86,363 paths, tricky names included (a.test.helper.ts, test_.py, x.d.ts, Button.stories.module.css, case variants, vendored and test directories), weigh exactly what the pre-change rules (copied verbatim into the test) gave them.
  • The existing scoring tests (tests/unit/services/slop_meter/test_score.py) pass unchanged.

The acceptance criterion, on a real database (tests/unit/api/test_slop_meter_settings_live.py, real lifespan, store and settings documents): the meter scores acme/widgets and the row records the version the repository reads. PATCH /config/settings/slop_meter?owner=acme {"weights": {"*.sql": 0.1}} makes the next look rescore at once; the row's config_version changes to the new effective version, the score changes, and the API shows it. Undoing the revision rescores it back to the first score and version.

Fail-first. The five new test files, copied into a clean origin/main tree: 37 failed, 3 errors, 2 passed. Every failure carries its own message:

  • the section tests: "the settings document has no slop_meter section: the meter's weights and pace are code";
  • the scoring tests: "... the meter scores with constants in code";
  • the loop tests: "... the meter reads no settings";
  • the live tests: "the meter has no schedule that looks at the settings: a change waits for a restart";
  • migration 144: "migration 144 must add slop_meter_scores.config_version" and "slop_meter_scores has no config_version column after migrating".

The two that pass there are the golden fixture and the path corpus, which pin behaviour that must not change. They fail when a default moves: "*.sql": 0.81 fails both, with "276 of 86363 paths weigh otherwise" and "a repository read with no settings scores as it did before #169".

Third review round (each new or changed test, run against the tree before it, 77e0cd23; 6 failed there, each with its own message):

Fix Test Fails before with
A share too small for a whole read writes no score test_a_share_too_small_for_a_whole_read_scores_nothing (1, 2 and 3 calls) "1 calls a run never read acme's reviews: no score is written"
The previous score stays, and is not rescored test_a_repository_whose_share_cuts_its_events_short_keeps_its_previous_score "the run its share cut short wrote no score: the previous one stays"; mutation (rescore a held one): "nor does a settings change rescore it from that short read"
A share large enough is scored test_a_share_large_enough_for_a_whole_read_scores passes before by design; mutation (hold every narrower share): "four calls a run read every listing: acme is scored"
The meter still waits for the interval test_an_organization_that_reads_less_never_holds_the_meter_on_catch_up (0, 3, reserve 100,000) passes before (fixed last round); kept
The most specific layers first test_a_read_past_the_bound_keeps_the_most_specific_layers_entries_first "acme/widgets keeps all of acme's globs"
Only a write that grows an over-bound total is refused test_a_layer_over_the_bound_because_a_document_below_grew_can_still_write "an unrelated key"

The reviewer's probes on the pushed tree:

  • narrow_stages: shares of 1 to 3 are unscored, 4 is scored.
  • narrow_org: the organization at 10 calls is unscored, with no backlog.
  • narrow_score: no longer prints, since the narrowed repository has no score to read.
  • trim_layers: both repositories keep acme's 200 globs and weigh org199/x.py 0.0.
  • broad_cap: the unrelated write is accepted.

Second review round (each new or changed test, run against the tree before it, 1a15a379; 13 failed there, each with its own message):

Fix Test Fails before with
A narrower share no longer holds the meter on catch-up test_an_organization_that_reads_less_never_holds_the_meter_on_catch_up (api_calls_per_tick 0 and 3, rate_reserve 100,000) "one full run in six hours, not one every catch-up: 72"
The credential's own budget still does test_the_credential_s_own_budget_still_holds_the_meter_on_catch_up passes before by design; mutation (every stop treated as a share's): "a backlog: not scored until its events are read"
Counts with what a write inherits test_a_list_added_to_is_counted_with_what_it_inherits_when_written, test_globs_are_counted_with_the_defaults_and_what_they_inherit_when_written the 210-bot and 1069-glob writes accepted; mutation (case-sensitive union): "a bot spelled otherwise is the same bot … 200, not 250"
A layer keeps its own entries test_a_layer_keeps_its_own_entries_when_the_document_below_grows_past_the_cap "nothing of acme's layer is refused: (LayerRefusal(… 'bots' …"
An exact path over a wildcard path test_an_exact_path_wins_over_one_that_reaches_it_through_wildcards "src/main.py over src/**/main.py"
Wildcard-led and inner-text globs are indexed test_a_path_is_checked_against_the_few_globs_that_could_match_it (five glob sets) "18043429 glob checks for 20000 paths" (the literal-head set passed before too)
Broad globs bounded on writes test_globs_checked_against_every_path_are_bounded_when_written no bound before; mutation (no check): "DID NOT RAISE SectionRefused"
The history built off the event loop test_the_judge_hears_the_repository_s_guidance_and_the_weighing_runs_off_the_event_loop "building the history, finding the outliers to judge and scoring each … in a thread"

The reviewer's probes, rerun on the pushed tree:

  • probe_loop2 and narrow_share: one full run in six hours, globex fetched once (acme was scored then; since the third round a share that cuts its read short holds its score).
  • cap_probe: the 210-bot write refused, naming 210; a grown deployment leaves acme its 60 bots.
  • spec_probe: 0 mismatches in 663,000, and the three cases resolved.
  • perf_wild: 0.3 to 0.6 s.
  • three_orgs, scope_probe, version_cost: as before.
  • many_repos (fix(slop): the meter never scores repositories when many share one credential #396): the same as main.

First review round (each new or changed test, run against the tree before it, 398abab4; 19 failed there, each with its own message):

Fix Test Fails before with
A settings change rescores only what it touches test_a_changed_weight_rescores_only_the_repository_it_touches_with_no_fetch_and_no_forge_call the old full run on change; as a mutation (a full run on change): "with no fetch"
A failing rescore is tried once per version test_a_repository_whose_rescore_fails_is_not_tried_again_every_look mutation (no record): "one try per version, not one a look: 3 tries"
The same, live test_a_weight_changed_in_the_document_rescores_on_the_next_run_and_records_the_version mutation (a full run on change): "from what was read: nothing is fetched"
Weigh paths by specificity test_a_folder_glob_weighs_every_file_under_it_whatever_its_extension, test_the_most_specific_glob_wins "gen/api.proto is under gen/"
Weighing cost test_a_path_is_checked_against_the_few_globs_that_could_match_it mutation (no directory index): "18043429 glob checks for 20000 paths"
Weighing off the event loop test_the_judge_hears_the_repository_s_guidance_and_the_weighing_runs_off_the_event_loop "finding the outliers to judge weighs every path, so it runs in a thread …: ['compute']"
An organization narrows only its own share test_an_organization_s_budget_narrows_only_its_own_share_of_a_credential "acme's 0 stops acme's reads alone; globex, on the same credential, reads the deployment's 20"
bots / ai_agents add up test_bots_and_ai_agents_add_to_what_each_layer_inherits, test_an_added_key_holds_what_every_document_below_lists_and_its_own, test_only_a_list_a_layer_may_set_can_be_added_to "a section names the list keys a layer adds to"; mutation (no added_keys): "no layer takes out a bot the layer below lists"
AI agent entries test_an_ai_agent_is_an_address_or_the_start_of_one_up_to_its_at "['dev'] would name people's addresses too"
Moved bots cleaned on the routes test_an_organization_s_bots_move_into_its_layer "a list the route took before is taken, each bot once" (422 before)
Checked before registering test_a_moved_setting_the_section_refuses_is_refused_before_the_owner_is_registered mutation (no check): the owner registered, then 422
An unholdable entry at the copy test_the_first_start_copies_each_organization_s_bots_into_its_layer "… leaving out the entry the section cannot hold rather than refusing the list: {}"
Old variables until imported test_until_the_import_succeeds_the_meter_reads_the_variables_it_read_before meter_reader() took no config; mutation (no fallback): "not imported yet: the variable's 0 still turns it off"
Packs keep their writer test_a_text_of_several_lines_is_a_literal_block_in_the_settings_document_only passes before by design (it pins packs unchanged); mutation (packs on the block writer): "a pack's YAML is unchanged, so no pack gets a new version"

One mutation passed: building a share from the organization's raw values instead of min/max with the deployment's. It is equivalent, since every share also draws on the credential's own budget, which holds the deployment's values. The min/max only make each share's own numbers honest in the logs.

First round's mutations, each caught: no early rescore, version not stored, section bots ignored, deployment values instead of the repository's, no turn-on run, small-repository rule always on, default judge guidance, a repository may set the pace, slop_meter_bots not moved.

Deploy notes

  • Migration 144 adds slop_meter_scores.config_version (NOT NULL DEFAULT ''). It follows refactor(agents): behaviour from agent and template rows #385's 143 on main; the branch is rebased onto it, and tests/unit/storage/test_migration_144.py now holds the "is the last entry" assertion, handed over from test_migration_143.py. The number lives in embry0/storage/migrations/runner.py, tests/unit/storage/test_migration_144.py (its name and VERSION), the hand-over docstring in tests/unit/storage/test_migration_143.py, embry0/storage/repositories/slop_meter.py (two docstrings), docs/architecture.md and docs/slop-meter.md.
  • First start: it imports SLOP_METER_INTERVAL_SECONDS, _API_CALLS_PER_TICK and _RATE_RESERVE when they are set and differ from the defaults, and copies each organization's slop_meter_bots into its layer. Every existing score row has config_version ''; the meter's first run, which scores every caught-up repository, does so with the same numbers and records a version.
  • After the deploy is verified, remove SLOP_METER_INTERVAL_SECONDS, SLOP_METER_API_CALLS_PER_TICK and SLOP_METER_RATE_RESERVE from .env if set; startup logs settings_env_ignored while they are.
  • Behaviour change: with the meter off, the loop keeps running and only removes what left the scope once an interval. Before, SLOP_METER_INTERVAL_SECONDS=0 pruned once at start and stopped. Turning it on needs no restart.
  • No image rebuild: nothing under embry0/sandbox/, safety/, agents/, execution/ changed. No new route; reading.config_version is an additive field.

Out of scope / follow-ups

Design + Plan

Anti-goals respected

  • No new npm/Python dependency
  • No silent default flip: every default is the constant it replaces, proven above
  • No secrets in diff
  • No emoji in code

Refs #169

@sudoTomas
sudoTomas force-pushed the feat/slop-meter-settings branch from f58357e to 1a15a37 Compare November 6, 2026 14:30
The meter kept all of its policy in code: the file weights and which
directories are vendored or tests, the pace and cap constants, the
small-repository rule, the level bounds and names and the judge's
prompt. An organization could not say what counts as code, how fast
its team works or where its levels fall, and changing a constant
silently rescored every repository. Its reading pace was three
variables, and an owner's own bots sat on its organization row.

Add a `slop_meter` section to the settings document. Every constant is
a key whose default is the value the code held, so nothing written
scores bit for bit as before: a fixture history reproduces the pinned
pre-change numbers, and 86,000 paths weigh what the old rules gave
them. An organization's layer may set every key but `enabled` and
`interval_hours`; a repository's what counts as code in its layout.
`weights` maps a glob to a weight, merged glob by glob through every
layer, the longest glob winning.

Each score row records `config_version` (migration 144), a digest of
the keys its repository's score is computed from. The loop now looks
at the section every five minutes: a change that gives a repository it
read another version rescores at once, `enabled` and `interval_hours`
apply within the look, and turning the meter on reads at the next one.
The forge budget is read per organization; organizations sharing one
credential get the fewest calls and the largest reserve.

SLOP_METER_INTERVAL_SECONDS (0 is `enabled: false`),
SLOP_METER_API_CALLS_PER_TICK and SLOP_METER_RATE_RESERVE are imported
once and then ignored. The organization setting `slop_meter_bots` moves
into its layer as `bots`, copied at the first start and still taken by
PATCH /orgs until 0.2.0. The process limits stay variables. The
unread LEAN_LEVEL constant is deleted: triage's lean belongs to the
routing section a later pull request adds. Multi-line texts in the
settings YAML are now written as literal blocks, so the judge's prompt
reads as written.

Refs #169
The meter kept all of its policy in code: the file weights and which
directories are vendored or tests, the pace and cap constants, the
small-repository rule, the level bounds and names and the judge's
prompt. An organization could not say what counts as code, how fast
its team works or where its levels fall, and changing a constant
silently rescored every repository. Its reading pace was three
variables, and an owner's own bots sat on its organization row.

Add a `slop_meter` section to the settings document. Every constant is
a key whose default is the value the code held, so nothing written
scores bit for bit as before: a fixture history reproduces the pinned
pre-change numbers, and 86,000 paths weigh what the old rules gave
them. An organization's layer may set every key but `enabled` and
`interval_hours`; a repository's what counts as code in its layout.
`weights` maps a glob to a weight, merged glob by glob through every
layer. Weigh paths by specificity: a folder glob beats any extension
glob, then more literal segments, then more literal characters. Globs
are indexed by extension and first directory and each path is weighed
once a run, in a thread.

Each score row records `config_version` (migration 144), a digest of
the keys its repository's score is computed from. The loop looks at
the section every five minutes: a settings change rescores only the
repositories it touches, from the rows and mirrors already read, with
no fetch, forge call or judgment, and the full run keeps its interval.
`enabled` and `interval_hours` apply within the look. A credential's
forge budget is the deployment's; an organization's values only narrow
its own share. `bots` and `ai_agents` add to what each layer inherits
(a new `added_keys` for list fields), and an AI agent is an address or
the part of one up to its `@`.

SLOP_METER_INTERVAL_SECONDS (0 is `enabled: false`),
SLOP_METER_API_CALLS_PER_TICK and SLOP_METER_RATE_RESERVE are imported
once and read as before until the import succeeds. The organization
setting `slop_meter_bots` moves into its layer as `bots`, cleaned
(each bot once, unholdable entries left out) at the first start and on
PATCH or POST /orgs, which still take it until 0.2.0. The process
limits stay variables. The unread LEAN_LEVEL constant is deleted:
triage's lean belongs to the routing section a later pull request
adds. Multi-line texts in the settings YAML are written as literal
blocks; progression packs keep their own writer.

Refs #169
The meter kept all of its policy in code: the file weights and which
directories are vendored or tests, the pace and cap constants, the
small-repository rule, the level bounds and names and the judge's
prompt. An organization could not say what counts as code, how fast
its team works or where its levels fall, and changing a constant
silently rescored every repository. Its reading pace was three
variables, and an owner's own bots sat on its organization row.

Add a `slop_meter` section to the settings document. Every constant is
a key whose default is the value the code held, so nothing written
scores bit for bit as before: a fixture history reproduces the pinned
pre-change numbers, and 86,000 paths weigh what the old rules gave
them. An organization's layer may set every key but `enabled` and
`interval_hours`; a repository's what counts as code in its layout.
`weights` maps a glob to a weight, merged glob by glob through every
layer. Weigh paths by specificity: a folder glob beats any extension
glob, then more literal segments, fewer `**` and wildcard segments,
more segments, more literal characters. Globs are indexed by name,
extension, literal directory or segment and three-character runs; at
most 16 per list may be checked against every path. Each path is
weighed once a run, and the history, outliers and scores are built in
a thread.

Each score row records `config_version` (migration 144), a digest of
the keys its repository's score is computed from. The loop looks at
the section every five minutes: a settings change rescores only the
repositories it touches, from the rows and mirrors already read, with
no fetch, forge call or judgment, and the full run keeps its interval.
`enabled` and `interval_hours` apply within the look. A credential's
forge budget is the deployment's; an organization's values only narrow
its own share, and a repository stopped by that share is scored with
what it read instead of keeping the meter on catch-up. `bots` and
`ai_agents` add to what each layer inherits (a new `added_keys` for
list fields, each item once without regard to case), and an AI agent
is an address or the part of one up to its `@`. A write counts merged
and added keys with what they inherit; a read past the cap keeps the
layer's own entries and the inherited ones that fit.

SLOP_METER_INTERVAL_SECONDS (0 is `enabled: false`),
SLOP_METER_API_CALLS_PER_TICK and SLOP_METER_RATE_RESERVE are imported
once and read as before until the import succeeds. The organization
setting `slop_meter_bots` moves into its layer as `bots`, cleaned
(each bot once, unholdable entries left out) at the first start and on
PATCH or POST /orgs, which still take it until 0.2.0. The process
limits stay variables. The unread LEAN_LEVEL constant is deleted:
triage's lean belongs to the routing section a later pull request
adds. Multi-line texts in the settings YAML are written as literal
blocks; progression packs keep their own writer.

Refs #169
The meter kept all of its policy in code: the file weights and which
directories are vendored or tests, the pace and cap constants, the
small-repository rule, the level bounds and names and the judge's
prompt. An organization could not say what counts as code, how fast
its team works or where its levels fall, and changing a constant
silently rescored every repository. Its reading pace was three
variables, and an owner's own bots sat on its organization row.

Add a `slop_meter` section to the settings document. Every constant is
a key whose default is the value the code held, so nothing written
scores bit for bit as before: a fixture history reproduces the pinned
pre-change numbers, and 86,000 paths weigh what the old rules gave
them. An organization's layer may set every key but `enabled` and
`interval_hours`; a repository's what counts as code in its layout.
`weights` maps a glob to a weight, merged glob by glob through every
layer. Weigh paths by specificity: a folder glob beats any extension
glob, then more literal segments, fewer `**` and wildcard segments,
more segments, more literal characters. Globs are indexed by name,
extension, literal directory or segment and three-character runs; at
most 16 per list may be checked against every path. Each path is
weighed once a run, and the history, outliers and scores are built in
a thread.

Each score row records `config_version` (migration 144), a digest of
the keys its repository's score is computed from. The loop looks at
the section every five minutes: a settings change rescores only the
repositories it touches, from the rows and mirrors already read, with
no fetch, forge call or judgment, and the full run keeps its interval.
`enabled` and `interval_hours` apply within the look. A credential's
forge budget is the deployment's; an organization's values only narrow
its own share. A repository that share stops before its events are all
read keeps its previous score, since a run starts its reading over, and
waits for the next run without keeping the meter on catch-up. `bots` and
`ai_agents` add to what each layer inherits (a new `added_keys` for
list fields, each item once without regard to case), and an AI agent
is an address or the part of one up to its `@`. A write counts merged
and added keys with what they inherit and is refused only when it makes
an over-bound total larger; a read past the bound keeps the most
specific layers' entries first.

SLOP_METER_INTERVAL_SECONDS (0 is `enabled: false`),
SLOP_METER_API_CALLS_PER_TICK and SLOP_METER_RATE_RESERVE are imported
once and read as before until the import succeeds. The organization
setting `slop_meter_bots` moves into its layer as `bots`, cleaned
(each bot once, unholdable entries left out) at the first start and on
PATCH or POST /orgs, which still take it until 0.2.0. The process
limits stay variables. The unread LEAN_LEVEL constant is deleted:
triage's lean belongs to the routing section a later pull request
adds. Multi-line texts in the settings YAML are written as literal
blocks; progression packs keep their own writer.

Refs #169
@sudoTomas
sudoTomas force-pushed the feat/slop-meter-settings branch from 0cfcb0a to f63f580 Compare November 6, 2026 16:42
@sudoTomas
sudoTomas merged commit c80ba42 into main Nov 6, 2026
13 checks passed
@sudoTomas
sudoTomas deleted the feat/slop-meter-settings branch November 6, 2026 17:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant