feat(agents): system_instructions_mode — let an agent config EXTEND the corpus persona instead of replacing it

[JSv4]

Stacked on #2261 (base is perf/vector-store-debug-counts, so the diff here is just this change).

The asymmetry

In unified_agent_conversation.py, a selected agent's system_instructions replace the context-derived prompt, while its available_tools merge:

if self.agent_config.system_instructions:
    agent_kwargs["system_prompt"] = self.agent_config.system_instructions
config_tools = list(self.agent_config.available_tools or [])

That forces a bad trade on any corpus-scoped agent: attaching a tool costs you the corpus persona. You get the persona that knows the corpus, or the agent that can call search_across_corpora — not both.

The only workaround is to paste the entire persona into system_instructions, which duplicates it into a second row that drifts from Corpus.corpus_agent_instructions silently.

Measured cost of the workaround

On a 4,679-section authority deployment (21 corpora, 18-member group), scoring answers on whether their quotations exist verbatim in the installed corpus:

	persona only	orchestrator only	persona + reach
main-set delta	8/15	6/15	10/15
expansion delta	2/7	5/7	6/7

The combination is worth roughly +2 and +4 over either alone, and it is the only configuration that reaches case law and Federal Register preambles. There is no way to express it today except duplication.

The change

Adds AgentConfiguration.system_instructions_mode:

• REPLACE — the historical behaviour, and the default, so no existing configuration changes.
• EXTEND — the instructions are appended to the context-derived prompt.

EXTEND rides a new extra_system_context kwarg, which the factory drains through AgentConfig.resolve_system_prompt after persona resolution. That is the mechanism added for #2247 for exactly this "append without consuming the persona signal" case — assigning to system_prompt here instead would consume the "caller supplied no prompt" signal and discard the persona, which is #2247.

Wired into both the document and corpus factories, not only the corpus one I needed.

Verification

test/verify_extend_mode.py asserts the composition on the built agent, not on the intention — #2247 was silent, so intention is not evidence:

config us-export-control-reach-extend: mode=EXTEND instructions=4,212 chars
corpus persona: 14,753 chars
built system prompt: 20,284 chars
  contains corpus persona:      True
  contains config instructions: True
  config tools: ['search_across_corpora']
OK — persona preserved AND config instructions appended

With EXTEND the config stores 4,212 chars (only what it adds) instead of 18,721 (persona + additions), and the persona stays single-sourced on the corpus.

Caveat

The repo suite was not run here: under local.yml, pytest dies at django.setup() on a duplicate storages app label before any of this is imported, and standing up test.yml risked evicting a database another session was using mid-measurement. Migration 0018 is additive with a default, so it is backward-compatible, but CI should exercise it.

🤖 Generated with Claude Code

[claude]

Review

Nice fix for a real asymmetry (instructions REPLACE / tools MERGE), and the mechanism itself is sound: add_computed_context / resolve_system_prompt correctly defers the persona-vs-caller-prompt decision, extra_system_context is popped in both create_document_agent and create_corpus_agent before reaching AgentConfig/get_default_config, and the migration is additive with a safe REPLACE default so existing configs are unaffected. The #2247 mechanism reuse is the right call — appending directly to system_prompt would indeed have destroyed the caller-supplied-nothing signal.

That said, I think the PR is incomplete relative to its own stated goal — a few things worth addressing before merge:

1. Two other call sites still hard-REPLACE, unaware of the new mode

• opencontractserver/llms/tools/delegation_tools.py:314-315 — the sub-agent delegation/mention tool does common_kwargs["system_prompt"] = agent.system_instructions unconditionally. An EXTEND-configured agent invoked through delegation will still lose its corpus persona.
• opencontractserver/tasks/agent_tasks.py:192 and :205 — the Celery task that answers an @mention on a corpus thread does agent_api.for_corpus(system_prompt=agent_config.system_instructions, ...), also unconditionally REPLACE.

Given corpus-thread @mentions and delegation are arguably the most common way these AgentConfigurations get invoked (more so than the interactive WebSocket session this PR wires up), EXTEND will silently behave like REPLACE through those paths — reproducing the exact persona-loss bug this PR is fixing, just via a different entry point. Worth auditing all agent_config.system_instructions consumers and either routing them through the same extra_system_context mechanism or consciously documenting why they are excluded.

2. The new mode isn't reachable through the app's own API

AgentConfigurationService.create_agent / update_agent have no system_instructions_mode parameter, and AgentConfigurationType (config/graphql/agent_types.py) doesn't expose the field either. The existing frontend admin UIs (CorpusAgentManagement.tsx, GlobalAgentManagement.tsx) that manage these configs have no way to set EXTEND — today it's only reachable via Django admin or a raw shell edit. If this is intentional for a first cut, worth calling out explicitly (e.g. a follow-up issue), since otherwise the feature is effectively invisible to users.

3. No automated test coverage

The PR description mentions a manual verification script (test/verify_extend_mode.py) asserting composition on the built agent, but it isn't part of the diff, and no test in opencontractserver/tests/websocket/test_unified_agent_consumer.py / test_unified_agent_consumer_delegation.py references system_instructions_mode or extra_system_context. A unit test on AgentConfig.resolve_system_prompt/add_computed_context ordering plus a consumer-level test for the EXTEND branch in _initialize_agent would lock in the behavior and guard against the ordering assumptions the docstrings call out (drain after persona resolution).

Minor

• system_instructions_mode max_length=8 fits both choices (REPLACE/EXTEND) exactly — fine today, but leaves zero headroom if a third mode is ever added; not a blocker.
• The extensive inline comments explaining the REPLACE/EXTEND rationale and the Corpus/document agent instructions are silently discarded: temporal grounding consumes the system_prompt is None signal #2247 connection are genuinely helpful for future readers — good call keeping that context close to the code given how non-obvious the ordering constraint is.

Also noting the PR's own caveat: the backend suite wasn't run against this change (blocked by an unrelated django.setup() issue under local.yml), so CI should be relied on to confirm the migration and factory changes are clean.

[JSv4]

Verified both branches, not just the new one — REPLACE is the default every existing configuration relies on, so it is the more important of the two to prove unchanged:

config us-export-control-reach-extend: mode=EXTEND instructions=4,212 chars
corpus persona: 14,753 chars

REPLACE: prompt  4,868 chars  persona=False config=True    <- unchanged
EXTEND : prompt 20,284 chars  persona=True  config=True
  config tools: ['search_across_corpora']

REPLACE excludes the persona and applies the config text (the historical behaviour); EXTEND carries both with the persona first; the merged tool survives either way. Harness is test/verify_extend_mode.py in the ITAR workspace — it asserts on the built agent's config.system_prompt, because #2247 was silent and 'the code looks right' is not evidence on this seam.

[JSv4]

Added tests to opencontractserver/tests/test_agent_system_prompt_assembly.py — the #2247 regression file, which is the same seam this change touches.

ExtraSystemContextFactoryTestCase is the regression coverage. It drives the real factory (with PydanticAIAgent patched at the seam the rest of that file uses, so no LLM is built) using extra_system_context= — the kwarg the consumer actually passes — and asserts on the assembled config.system_prompt. Verified both cases fail without the change, by reverting agent_factory.py to its pre-feature state:

# agent_factory.py reverted (0 references to extra_system_context)
exit_WITHOUT_fix=1
FAILED test_corpus_factory_appends_extra_system_context
FAILED test_document_factory_appends_extra_system_context

# with the change
exit=0

Covers the document factory as well as the corpus one, since an agent config can be selected for a document too — and that test caught a real gap in my own fixture rather than passing by luck.

test_omitting_it_changes_nothing is the control: absent the kwarg, the prompt is unchanged.

ExtendModeAssemblyTestCase is contract coverage, not regression coverage, and is labelled that way in the file: it exercises add_computed_context ordering, which predates EXTEND, so those tests would pass without this PR. They are there to pin the ordering guarantee EXTEND depends on (persona first, appended blocks after) so a future change to resolve_system_prompt can't silently invert it.

Combined run with the diagnostics tests from #2261: 23 passed, exit 0. Broader run across five related files: 81 passed.

[claude]

Review

Clean, well-scoped change with real production evidence behind it (measured retrieval deltas, and a mechanism — resolve_system_prompt/add_computed_context — that already existed from #2247, reused rather than reinvented). The default-REPLACE migration is safe and the new tests correctly assert on the assembled prompt rather than intent, consistent with why #2247 slipped through in the first place.

A few things worth addressing before/soon after merge:

1. EXTEND mode is wired into only one of several call sites that read system_instructions

_initialize_agent in config/websocket/consumers/unified_agent_conversation.py is the only place taught to check system_instructions_mode. Two other production paths read AgentConfiguration.system_instructions and hard-code REPLACE-style behavior, silently ignoring the new field:

• opencontractserver/llms/tools/delegation_tools.py:314-315 — the sub-agent delegation tool (if agent.system_instructions: common_kwargs["system_prompt"] = agent.system_instructions). This is arguably the most relevant call site given the PR's own motivating scenario — a conductor delegating to a corpus-scoped sub-agent configured with tools like search_across_corpora. An AgentConfiguration set to EXTEND will still lose its persona when invoked through delegation.
• opencontractserver/tasks/agent_tasks.py:189-207 — the mention-triggered Celery agent task always does system_prompt=agent_config.system_instructions, unconditionally.

As shipped, whether EXTEND actually extends the persona depends on which surface triggered the agent (direct WS chat vs. delegation vs. corpus-action/@mention task) — the same AgentConfiguration behaves differently depending on invocation path. If that's intentionally out of scope for this stacked PR, it'd help to say so explicitly in the PR description / a follow-up issue, since the current framing ("Wired into both the document and corpus factories, not only the corpus one I needed") reads as if the mode is fully plumbed through.

2. No way to actually set system_instructions_mode outside the ORM/shell

• config/graphql/agent_mutations.py (CreateAgentConfigurationMutation / UpdateAgentConfigurationMutation) doesn't accept it as an input field.
• config/graphql/agent_types.py::AgentConfigurationType doesn't expose it for reads.
• opencontractserver/agents/admin.py::AgentConfigurationAdmin uses an explicit fieldsets tuple that omits it, so it won't even render in the Django admin form.

Right now the only way to flip a config to EXTEND is direct DB/shell access (as the PR's own verification script does). Worth a fast follow-up, otherwise this ships as effectively unreachable by normal users/admins.

3. Minor: magic-string comparison instead of the model's own constant convention

AgentConfiguration already has a documented pattern for this exact situation — SCOPE_GLOBAL/SCOPE_CORPUS class constants exist specifically "so callers don't have to spell the raw strings" (opencontractserver/agents/models.py:65-68). The new system_instructions_mode field doesn't follow it: there's no AgentConfiguration.MODE_EXTEND, and unified_agent_conversation.py:681 compares against the literal string "EXTEND" directly. Small thing, but it's inconsistent with an explicit, recently-stated convention in the same file, and a typo in the literal wouldn't be caught anywhere near as easily as an AttributeError on a missing constant.

4. Test coverage gap on the actual glue code

The new tests (ExtendModeAssemblyTestCase, ExtraSystemContextFactoryTestCase) thoroughly cover AgentConfig.resolve_system_prompt/add_computed_context and the factory's extra_system_context kwarg — but nothing exercises unified_agent_conversation.py::_initialize_agent's actual dispatch (if self.agent_config.system_instructions_mode == "EXTEND": ... else: ...), which is the only production code this PR ships that reads the new field. opencontractserver/tests/test_unified_agent_consumer_delegation.py already exists as a precedent for testing this consumer and would be a natural home for a case that sets system_instructions_mode="EXTEND" on a real AgentConfiguration and asserts agent_kwargs ends up with extra_system_context rather than system_prompt.

5. Nit: comment/code placement

In both create_document_agent and create_corpus_agent, the new if extra_system_context: config.add_computed_context(...) block was inserted directly under the # Public corpus/document ⇒ strip approval-gated tools section header, which describes the tool-filtering code further down, not this block. Consider moving the two-line block above that header (or giving it its own one-line comment) so the header still accurately describes what follows it.

6. Process nit

Per CLAUDE.md, new features / migrations should get a changelog.d/ fragment. This PR doesn't add one (it wouldn't be caught by the pre-commit hook, since that hook only triggers on changes under changelog.d/).

---

Nothing here blocks the core idea — the design (append-after-persona-resolution via the existing computed_context queue, REPLACE as an unchanged default) is sound and well-tested where it's tested. Items 1 and 2 are the ones I'd actually want resolved (or explicitly deferred) before this goes out, since together they mean the feature is currently reachable by neither most invocation paths nor most users.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!