Add agent observability with Monocle - #1
Draft
imohammedansari wants to merge 113 commits into
Draft
Conversation
…nce#3985) * fix(provisioner): gate legacy skills mount by user visibility * fix(test): aio sandbox provider * fix: use shared legacy skill visibility helper for sandbox mounts
* feat: add composer input polishing * Revert "Merge branch 'main' into feat/input-polish" This reverts commit 5b6cecc, reversing changes made to 45fbc57. * Merge main into feat/input-polish * style(frontend): format input helper polish guard * fix(input-polish): address composer polish review findings Frontend - Add a cancel affordance to the in-flight polish status pill that calls abortInputPolishRequest(), so a slow/hung provider no longer hard-locks the composer for up to stream_chunk_timeout with a page reload (and draft loss) as the only escape. - Reset promptHistoryIndexRef/promptHistoryDraftRef when a rewrite is applied (and on undo), so a stale history-browse index can no longer let the next ArrowDown silently overwrite the polished draft. - Disable polishing while an open human-input card is present, matching the frontend/AGENTS.md rule that composer entry points defer to the card so card-reply metadata is preserved. - canPolishInput now reuses parseGoalCommand/parseCompactCommand instead of a third hardcoded reserved-command regex, and drops the phantom /help entry (no /help parser exists in the composer), so future builtins only need to be taught to the existing parsers. Backend - Extract the non-graph one-shot LLM path (build model + inject Langfuse metadata + system/user invoke + text extract) into deerflow.utils.oneshot_llm.run_oneshot_llm, shared by the input-polish and suggestions routers so tracing-metadata and invocation shape cannot drift between the two copies. - strip_think_blocks gains truncate_unclosed (default True, preserving the suggestions/goal JSON-prep behavior); input polish passes False so a draft that legitimately contains a literal <think> substring is no longer truncated into a partial rewrite or a spurious 503. - Validate the empty-check and max_chars boundary against the same stripped view of the draft that is sent to the model, so the user-facing length boundary and the model input can no longer disagree. Tests / docs - Backend: literal-<think> preservation, whitespace-only rejection, and normalized-length/model-input agreement cases; suggestions tests repoint the create_chat_model patch to the shared helper module. - Frontend: helper unit tests updated for the /help/reserved-command change; a new Playwright case covers cancelling an in-flight polish request. - backend/AGENTS.md documents the shared one-shot helper and the polish normalization/think-tag behavior. --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
…ance#3981) * feat(frontend): render slash-skill activations as inline chips Show an explicit `/skill` activation as a compact inline chip in both the composer and the chat transcript instead of raw slash text. - Composer: selecting a skill suggestion stores it as a removable chip aligned inline with the textarea; the leading `/skill ` prefix is reattached only at submit time, so the backend activation protocol is unchanged. Backspace on an empty input or the chip's close button clears it; history navigation is disabled while a chip is active. - Transcript: human messages that begin with `/skill` render the skill as a read-only chip followed by the task text. - Add a shared `core/skills/slash.ts` (`parseSlashSkillReference` + `resolveSlashSkillDisplay`) mirroring the backend `slash.py` gate, so the transcript only shows a chip when the skill actually exists and is enabled. This removes a duplicated regex/reserved-name list and keeps display semantics consistent with backend activation. Add unit tests for the shared slash parser and extend the chat e2e to assert the composer still submits `/skill <task>` after showing a chip. * chore(frontend): format chat e2e test * refactor(skills): address slash-skill chip review feedback Follow-up to the inline slash-skill chip PR, resolving three second-order review findings: - Drive the reserved-command set and skill-name grammar from a shared contracts/slash_skill_contract.json instead of a hand-copied "keep in sync" pair. slash.ts and slash.py now reference the fixture, and contract tests on both sides fail CI if either drifts. - Extract a shared SlashSkillChip so the composer and transcript chips stay in lockstep, and normalize the off-scale /8 and /12 opacity steps to the standard /10 and /20 tokens. - Split HumanMessageText into a pure parse gate plus a slash-only subtree that owns the useSkills() lookup, so a skill-enabled toggle no longer re-renders every plain-text human turn. Verified: frontend eslint + tsc clean, pnpm test 572 pass (incl. new slash-contract test); backend slash contract + slash-skills tests 31 pass. * style(tests): sort slash skill contract imports * fix(composer): inline the slash-skill text so the chip aligns with input Address the "composer body layout change" review on bytedance#3981 by rendering the active skill as an inline chip in the same text flow as the prompt, rather than a separate flex row that drifted the box model across states. - Render the chip + prompt inside one leading-6 wrapper and edit the prompt through a `contentEditable` span, so the chip sits inline with the first line and long/multi-line input wraps naturally back to the container edge. - Align the chip with `align-top`: its h-6 (24px) height matches the text line height, so chip and first-line centers coincide exactly (measured delta 0), fixing the chip being raised above the baseline. - Restore the placeholder in chip mode via a `data-empty` CSS `::before`, which also gives the empty editable span width so it is no longer treated as hidden. - Widen the IME helper to `HTMLElement` and route the span's keydown/paste through the shared skill-suggestion, prompt-history, backspace-to-clear, and IME-composition handlers so contentEditable behaves like the textarea. - Extend chat.spec.ts to drive the inline skill editor instead of the textarea after a chip is shown. * style(frontend): fix composer class order formatting * fix(composer): break long unbroken input inside the slash-skill row The inline slash-skill editor wrapped with `break-words` (overflow-wrap: break-word), which only moves an over-long token to the next line before breaking it. A long unbroken string therefore started on the line below the chip, and when the string contained a break opportunity such as a hyphen the browser wrapped there and pushed the remaining run to the next line, leaving a wide gap on the right. Switch to `break-all` (word-break: break-all) so the text fills each line from the chip and packs tightly regardless of hyphens or CJK.
…ken_budget (bytedance#3875 Phase 2) (bytedance#3980) Phase 2 of bytedance#3875. Two guardrail axes can end a subagent run early — the turn budget (GraphRecursionError) and the token budget (TokenBudgetMiddleware) — and both now surface *why* through one additive `subagent_stop_reason` field instead of a status enum. This completes and course-corrects Phase 1 (bytedance#3949), which shipped the turn-budget cap as a `max_turns_reached` status enum. The agreed Phase 2 design replaces that enum with an optional `stop_reason` field (token_capped | turn_capped | loop_capped): a new enum value would break v1 consumers, while an additive field is ignored by older frontends and ledger readers. `max_turns_reached` and SubagentStatus.MAX_TURNS_REACHED are removed. - subagents.token_budget config (default enabled, 2,000,000 tokens, warn 0.7) with per-agent override; TokenBudgetMiddleware is now attached in build_subagent_runtime_middlewares so the cost-ceiling backstop engages for every subagent. The hard-stop does not raise — it strips tool_calls and lets the run finish with a final answer, recording the cap on a per-run consume_stop_reason() accessor. - executor.py: on normal completion it reads consume_stop_reason() and stamps completed + token_capped when the budget fired; on GraphRecursionError it recovers the last AIMessage partial (completed + turn_capped) or, if nothing usable survived, failed + turn_capped. SubagentResult gains stop_reason. - status_contract.py / contracts/subagent_status_contract.json (v2) / frontend subtask-result.ts: additive subagent_stop_reason field, pinned by test_status_values_match_contract / test_stop_reason_values_match_contract. - task_tool.py + delegation_ledger.py: drop the max_turns_reached paths; the ledger captures stop_reason and renders model-facing "capped" guidance so the lead reuses a capped completion knowingly. The 2,000,000-token default is deliberately loose (tighten to taste) — it would have roughly halved the reported 4.4M burn while leaving legitimate deep-research runs (max_turns=150) room. Subagent summarization is a follow-up.
Enable Monocle (OpenTelemetry tracing for LLM apps) with one setup call plus the monocle_apptrace dependency. setup_monocle_telemetry auto-instruments the frameworks already in use and writes traces to .monocle/. Additive; no changes to application logic.
imohammedansari
force-pushed
the
monocle-instrumentation
branch
from
July 8, 2026 22:57
e63b8b5 to
ba5caeb
Compare
bytedance#4002) * fix(security): neutralize prompt-injection tags in remote tool results User input is already neutralized for framework/injection tags, but tool results are not. Remote content fetched by web_fetch/web_search is equally untrusted and can carry a forged <system-reminder> block that reaches the model verbatim as authoritative context. Extract a shared neutralize_untrusted_tags() primitive from InputSanitizationMiddleware and apply it to remote-content tool results (web_fetch/web_search/image_search) via a new ToolResultSanitizationMiddleware. Local tool output (bash/read_file) is left untouched so legitimate code/file content is never mangled. * test: update subagent middleware count for tool-result sanitizer The new ToolResultSanitizationMiddleware adds one entry to the shared runtime chain (11 -> 12). Update the subagent count assertion, use a lazy import for neutralize_untrusted_tags so the module loads even when tests stub the input-sanitization module, and document the new middleware in AGENTS.md. * fix(security): address review — sanitize bare str list items; document MCP scope - Neutralize bare str elements inside a ToolMessage content list (previously only {type:text} dict blocks were rewritten), matching the str-in-list shape ToolOutputBudgetMiddleware._message_text already anticipates. - Document the name-based allowlist limitation: MCP remote-content tools registered under arbitrary names (e.g. fetch_url) are not covered; a name heuristic is avoided to prevent mangling local tool output, with metadata tagging tracked as a follow-up. Add a regression test pinning this boundary.
…ance#3987) * feat(helm): add production-ready Helm chart for Kubernetes deployment Adds deploy/helm/deer-flow, a native-Kubernetes translation of the production docker-compose stack, plus CI to publish its images and chart. * ci(release): gate releases on version-source consistency Add a reusable verify-versions workflow invoked by both chart.yaml and container.yaml on v* tags. It runs scripts/verify_versions.sh against the tag and fails the release — skipping all image and chart publishing — when Chart.yaml (version + appVersion), backend/pyproject.toml, or frontend/package.json don't all match the tag. Add scripts/verify_versions.sh (the check, also runnable locally) and scripts/bump_version.sh (bumps all four sources in lockstep, then self-verifies). Document the release flow in RELEASING.md and link it from AGENTS.md. * fix(deploy): address Helm chart review feedback (bytedance#3987) Three review items from willem-bd: 1. nginx IPv6 listen strip never matched. The sed pattern required a `;` immediately after `2026`, but the rendered config emits `listen [::]:2026 default_server;` (space + `default_server` before the `;`), so the line was never deleted and nginx crash-looped on pods without IPv6 (`socket() :::2026 failed (97: Address family not supported)`). Drop the trailing `;` from the pattern so it matches. Same latent bug fixed in docker-compose-dev.yaml. 2. Passwords were spliced into DSNs verbatim, so a password containing URL-special chars (@ : / # ? % [ ] space) produced a malformed DSN and a confusing parse error. Add a `deer-flow.urlEscape` helper (replace-based: Sprig lacks urlqueryescape, and regexReplaceAllLiteral treats the replacement as a regex template so `[`/`]`/`?` break it) and apply it to the password in the postgres and redis DSNs. The raw `postgres-password` / `redis-password` keys stay unencoded - they back POSTGRES_PASSWORD / REDIS_PASSWORD, not a URL segment. 3. NODE_HOST defaulted to "gateway", which can never route: the gateway Service is ClusterIP:8001 and knows nothing of a sandbox NodePort, so a user who skips the caveat gets unreachable sandboxes with no error at install time. Default NODE_HOST to the provisioner pod's node IP via the downward API (status.hostIP) - a NodePort is exposed on every node, so <node-IP>:<NodePort> routes from the gateway on most clusters. `provisioner.nodeHost` remains an override for CNIs/policies that block pod->node-IP traffic. Updated NOTES.txt, values.yaml, and the chart README. (bytedance#3929 remains the long-term fix - ClusterIP + cluster-DNS URL removes NODE_HOST and the NodePort exposure entirely.) Validated with helm lint, helm template (incl. a special-char password rendering the encoded DSNs), and a sed pattern-match check. * fix(deploy): address round-2 Helm chart review feedback (bytedance#3987) Three "Medium" items from willem-bd: 1. No helm lint / helm template gate before publish. A template regression ships as an immutable OCI artifact (GHCR won't overwrite --version), so gate packaging on `helm lint` + `helm template --include-crds` in chart.yaml before `helm package`. (ct lint / helm-unittest deferred.) 2. Action pinning inconsistent + PR body overstates it. SHA-pin actions/checkout (v6.0.3, df4cb1c0) and actions/attest-build-provenance (v2.4.0, e8998f94) across the publishing workflows (chart.yaml, container.yaml, verify-versions.yml), matching the existing docker/* SHA-pin pattern. Resolves the checkout @v4/@v6 mismatch and makes the "SHA-pinned actions" claim accurate. Other pre-existing workflows left untouched (out of scope for this PR). 3. Provisioner RBAC broader than needed. Dropped the unused update/patch verbs and the pods/exec + events rules from the provisioner Role - audited against docker/provisioner/app.py, which only calls get/create/delete on pods and get/list/create/delete on services. Fixed NOTES.txt to accurately describe the grant instead of understating it as "create Pods and Services". The remaining scope concern - verbs apply to all Pods in the namespace, not just sandbox Pods - is still deferred (RBAC can't scope by label; needs a dedicated namespace or admission control), now noted in NOTES.txt and README. Validated with helm lint + helm template (narrowed Role renders with exactly get/list/watch/create/delete). * feat(helm): enable sandbox+web tools out of the box The chart's default config loaded zero agent tools (config.tools empty -> "Total tools loaded: 0"), so a fresh install gave an agent that could do nothing useful. Add tool_groups + tools to the default config block: - web: web_search (ddg), web_fetch (jina), image_search - no API key - file:read: ls, read_file, glob, grep - file:write: write_file, str_replace - bash The file/bash tools run inside the AIO sandbox the chart already configures; the web tools need outbound internet from the gateway pod (swap backends or drop entries for air-gapped clusters - see config.example.yaml). Also bump config_version 15 -> 19 to match config.example.yaml (the chart had drifted behind). NOTES.txt and the README example updated to match. * ci(helm): add chart validation + config_version drift check on PR Extend the chart workflow with a PR-triggered validate-chart job that runs helm lint, helm template --include-crds, and a config_version drift check: it parses config_version from both config.example.yaml and the chart's values.yaml and fails the build (with a ::error:: naming the files to bump) if the chart is behind the example. This catches the kind of drift this PR is fixing - the chart sat at v15 while the example moved to v19 - before it can merge again. verify-versions and publish-chart stay tag-only; publish-chart now needs: [verify-versions, validate-chart]. validate-chart runs on both PRs and tag pushes: the tag arm is required because a job that `needs` a skipped job is itself skipped under the default success() check, so validate-chart must actually run on tag pushes or publish-chart would never fire. * Bump config version to 20
* feat: add MCP routing hints * test: isolate mcp routing prompt config * fix: address mcp routing review feedback
* fix: recover from empty tool call names * test: harden empty tool call recovery --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
…edance#4015) The tag-publish workflow (container.yaml) built backend/Dockerfile with no build-args, so UV_EXTRAS was empty and the published *-backend image shipped without the Postgres driver (only --extra redis). Multi-replica deployments (K8s/Helm) that need shared Postgres persistence instead of file-based SQLite could not use the release image without rebuilding it. Pass UV_EXTRAS=postgres so the release image includes deerflow-harness[postgres]. Additive only: single-replica sqlite/redis setups keep working; the Postgres driver is added, mirroring how redis is already always baked in.
…ce#4013) Bumps [langsmith](https://github.com/langchain-ai/langsmith-sdk) from 0.8.0 to 0.8.18. - [Release notes](https://github.com/langchain-ai/langsmith-sdk/releases) - [Commits](langchain-ai/langsmith-sdk@v0.8.0...v0.8.18) --- updated-dependencies: - dependency-name: langsmith dependency-version: 0.8.18 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…ce#4001) * bench: add provider-agnostic sandbox benchmark with BoxLite warm pool results - scripts/bench_sandbox_provider.py: CLI for measuring acquire/run/release across providers, scenarios, workloads, and concurrency levels - scripts/summarize_bench.py: JSONL aggregation with p50/p95/p99 tables - bench_results.jsonl: 110 turns across 7 scenarios on real BoxLite 0.9.7 Key findings: cold acquire: ~860ms warm reclaim: ~14ms (60x speedup) release: ~0ms warm_hit_rate: 95% (warm_same_thread) * perf(boxlite): skip health check for recently-released warm pool boxes Boxes released within health_check_skip_seconds (default 5.0s) are promoted directly without the ~14ms echo-ok round-trip. A VM alive seconds ago is overwhelmingly likely to still be alive. Add sandbox.health_check_skip_seconds config option. Set to 0 to always health-check (old behaviour). Benchmark (warm_same_thread, noop, 20 iters): acquire p50: 14.9ms → 0.0ms total p50: 29.9ms → 14.0ms * chore: move benchmark scripts into backend/scripts/benchmark/ * fix: address BoxLite benchmark review findings * fix(boxlite): only skip warm reclaim checks for released boxes * fix(benchmark): keep BoxLite shim workaround off the event loop * fix(boxlite): invalidate dead boxes from command path * test(boxlite): cover skip window and invalidation edge cases * fix(boxlite): treat sandbox-has-been-closed as terminal in _exec * fix(boxlite): harden warm-pool reclaim and benchmark accounting * fix(boxlite): validate warm-pool reclaims by default * fix(config): expose boxlite health-check skip setting * fix(boxlite): tighten failure classification and benchmark workaround * Update config_version to 21 in values.yaml --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
…bytedance#4014) Bumps [pydantic-settings](https://github.com/pydantic/pydantic-settings) from 2.14.0 to 2.14.2. - [Release notes](https://github.com/pydantic/pydantic-settings/releases) - [Commits](pydantic/pydantic-settings@v2.14.0...v2.14.2) --- updated-dependencies: - dependency-name: pydantic-settings dependency-version: 2.14.2 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…ounts (bytedance#4016) * feat(provisioner): add ClusterIP mode for sandbox services * feat(provisioner): support optional skills pvc subpath
imohammedansari
changed the title
…ill env (bytedance#4018) * fix(sandbox): scrub MYSQL_PWD and REDISCLI_AUTH from the inherited skill env env_policy blocks the MySQL and Redis connection strings (MYSQL_URL, REDIS_URL) because they embed a password, but not the password variables the same clients read directly: mysql/libmysqlclient takes MYSQL_PWD and redis-cli takes REDISCLI_AUTH with no further configuration. Both names carry no KEY/SECRET/TOKEN/PASSWORD/PASSWD substring, so they were inherited by every skill subprocess -- the leak surface this module exists to close (bytedance#3861). They need exact entries rather than a pattern: *PWD* would also strip PWD and OLDPWD. REDIS_AUTH is included as the common non-canonical variant. * Apply suggestions from code review - Distinguish REDIS_AUTH (defensive, non-canonical) from MYSQL_PWD and REDISCLI_AUTH (documented no-flag credential sources) in the comment. - Pin that request-scoped injection still overrides the host value for the newly blocked names, keeping required-secrets a working escape hatch.
* fix(gateway): live-tail malformed redis reconnect ids * test: avoid redis live-tail heartbeat gaps * fix: handle terminal malformed redis reconnect ids
…nce#4019) * feat(mcp): auto-promote deferred MCP tools from routing hints When tool_search.enabled=true defers MCP tool schemas, PR1 routing hints still require the model to spend a tool_search discovery round trip before it can call the tool the routing metadata already points at. This adds a McpRoutingMiddleware that matches the latest user message against PR1 routing keywords and promotes the matching deferred schemas before the model call, removing that round trip. Design (soft routing, opt-in, additive): - Matches only the latest real HumanMessage (shared is_real_user_message helper, reused by SkillActivationMiddleware so the two cannot drift); case-insensitive substring match, no tokenizer dependency. - Ordering: priority desc, then tool name asc; capped by the new global tool_search.auto_promote_top_k (default 3, clamped 1..5). Does not add or consume a per-tool auto_promote_top_k (PR1 schema unchanged); a per-tool value is ignored with a DEBUG note. - Returns a plain {"promoted": ...} state update (not a Command) and relies on ThreadState.merge_promoted for union/dedupe, so auto-promote and a model-triggered tool_search converge on the same catalog hash. - Installed before DeferredToolFilterMiddleware on every deferred-tool path (lead agent, subagent, embedded client, webhook via shared builders); a construction-time assert rejects the reversed order. catalog_hash is None / no routing index is a complete no-op, so bootstrap and ACP skip it. - Privacy: never executes tools, never promotes policy-filtered tools, adds no routing keywords or matched tool names to trace metadata or INFO/WARN logs. No behavior change when tool_search.enabled=false. Tests: index construction, matching semantics, middleware state updates, same-cycle deferred-filter interaction, lead/subagent/embedded-client builder wiring + order invariant, config clamping, config.example.yaml parseability, and privacy assertions. * refactor(mcp): address auto-promote review nits - executor: access app_config.tool_search.auto_promote_top_k directly to match the lead-agent and embedded-client paths (drop the over-defensive getattr that masked missing config); update the subagent test mock to carry tool_search. - tool_search / mcp_routing_middleware: cross-reference the duplicated routing priority/keyword normalization between the builder and the middleware's defensive _normalize_index so they cannot silently drift. - MCP_SERVER.md: document that auto-promote keyword matching is a case-insensitive substring test (not word-boundary), advising distinctive keywords.
…ects max_chars (bytedance#4017) * fix(agent): snap tool-output tail forward so fallback truncation respects max_chars _snap_to_line_boundary() moves an offset backwards to the preceding newline. That is correct for the head's end offset, but _build_fallback() also applied it to the tail's start offset, where moving backwards lengthens the tail. On output whose last newline sits in the second half (log lines followed by one long unbroken line: minified JSON, base64 artifacts), the returned string exceeded max_chars by up to 17x, defeating the guard that exists to stop a single large tool result from blowing the model context. Add _snap_start_to_line_boundary(), the forward-snapping mirror, and use it for the tail offset. The existing head_end guard becomes redundant because a forward snap can only move the tail start away from the head. test_result_never_exceeds_max_chars already asserted this invariant but passed newline-free content, so the snapping branch was never exercised. * test(agent): pin the forward-snap direction of the fallback tail The bound test only asserts that the result fits max_chars. Its content has no newline inside the tail's snap window, so _snap_start_to_line_boundary returns pos unchanged and the test stays green even with the snap removed. Place a newline in the window so the snap has to fire: the tail must begin after it, which the pre-fix backward snap does not do.
…rd previews (bytedance#3988) * feat(backend): queue rapid same-thread messages and preserve topic card previews * fix: checkstyle * refactor(channels): make feishu serialization policy-driven and fix queue cleanup
`image_search` set both `image_url` and `thumbnail_url` to the DDGS result's `thumbnail` field, so the tool never returned the full-resolution image even though its docstring/usage_hint promise reference-quality images (and a result with no `thumbnail` returned an empty `image_url`). DDGS `.images()` exposes the full-res source under the separate `image` key — read that for `image_url`, matching the serper/brave providers which keep image vs thumbnail distinct. Adds a regression test.
…ure (bytedance#3875 Phase 3) (bytedance#4009) Phase 3 of bytedance#3875 — subagents previously inherited none of the lead's context-compaction, so a deep-research subagent (max_turns up to 150) could accumulate >1M cumulative input before max_turns/timeout/token_budget engaged, even after Phase 2's budget capped the pathological tail. - Gate the subagent runtime chain on the SAME ``app_config.summarization.enabled`` switch the lead reads (per maintainer guidance in bytedance#3875), via the shared ``create_summarization_middleware`` factory. One config covers both chains; no separate ``subagents.summarization`` field. No-op when summarization is off (factory returns None). - ``skip_memory_flush=True`` on the subagent path: the factory otherwise attaches ``memory_flush_hook`` (when memory.enabled), which flushes pre-compaction messages into durable memory keyed by thread_id. Subagents share the parent's thread_id, so without skipping the hook a subagent's internal turns would pollute the PARENT thread's durable memory (bytedance#3875 Phase 3 review point). - Harden ``capture_new_step_messages`` to tolerate history contraction: summarization rewrites the messages channel via ``RemoveMessage(id=REMOVE_ALL_MESSAGES)``, shrinking len(messages) below the step-capture cursor. Without a reset, every step appended after the compaction point was dropped until length overtook the stale cursor (bytedance#3845 interaction, maintainer validation point (a)). Cursor now resets to the new tail; id/content dedup prevents re-emitting pre-compaction steps. - Couple the DEFAULT token-budget ceiling to ``summarization.enabled`` (bytedance#3875 Phase 3 review point): 1M when compaction is on, 2M when off (preserves Phase 2's deliberate headroom for summarization-off deep-research runs that can exceed 1M). A user-set budget (global or per-agent) always wins regardless of the switch. Flagged tunable. The summarization middleware does not implement ``consume_stop_reason``, so the Phase 2 guard-cap stop-reason channel is unaffected. Refs: bytedance#3875
…bytedance#3996) * feat(memory): add memory consolidation to synthesize fragmented facts When a fact category accumulates many individual entries, the LLM reviews them during the normal memory-update call (same invocation, no extra API cost) and decides whether groups of related facts can be synthesized into a single richer fact. This completes the memory lifecycle: extraction → guaranteed injection → staleness review → consolidation. - Select fragmented categories by min-facts threshold, surface the most fragmented groups first; prompt-layer caps aligned with apply-layer guardrails so the LLM never sees groups it cannot act on - Cap consolidated confidence at source maximum to prevent inflation; reject results below fact_confidence_threshold - Double-consume protection prevents a fact from being merged into multiple consolidation targets - Feature-gated at both prompt and apply time with per-cycle safety caps - Add 26 tests covering candidate selection, normalization, apply guardrails, and prompt integration * fix(memory): address consolidation correctness issues from PR review Six fixes based on maintainer review of bytedance#3996: 1. Deduplicate sourceIds in normalization — ["f1","f1"] previously bypassed the ≥2-distinct-sources check; dict.fromkeys collapses it to ["f1"] which is correctly rejected. 2. Run consolidation after max_facts trim — previously, sources were deleted then the merged fact could be evicted by the trim, leaving no record of either. Moving consolidation last ensures source facts exist in the post-trim index before removal. 3. Fix count= attribute in consolidation prompt — advertised the full category size but listed only max_sources IDs; now uses min(len(group), max_sources) to match what the LLM can act on. 4. Exempt staleness_protected_categories from consolidation candidates — mirrors the existing staleness-review contract so correction facts are never surfaced for merging. 5. Strip and default category in consolidation normalization — " " or " preference " are now normalised, matching _normalize_memory_update_fact. 6. Propagate sourceError from source facts into consolidated fact — correction context is no longer silently lost on merge. * fix(memory): add apply-time guardrails and tests for consolidation P1: mirror the staleness-pass defense-in-depth pattern — build allowed_source_ids from _select_consolidation_candidates at apply time so a protected-category or below-threshold fact proposed by the LLM is rejected regardless of model behavior. P2a: test that LLM-returned confidence is capped at max source confidence and that a capped result below fact_confidence_threshold is rejected. P2b: test that factsToConsolidate with consolidation_enabled=False is a no-op at apply time (35 tests, all pass). * fix(memory): address three correctness issues from second review round 1. Default consolidation_enabled=False — consolidation is lossy (source content is permanently replaced, only consolidatedFrom IDs preserved); new lossy features default to off. config.example.yaml updated to match. 2. Unify confidence coercion between prompt and apply — _build_consolidation_section now calls _coerce_source_confidence(fact) instead of an inline 0.0-default coercion, so a null-confidence fact renders with 0.50 in the LLM prompt and is capped at 0.50 at apply time (same value, same function). 3. Preserve staleness clock on merge — consolidated fact now carries the newest source's createdAt (not now) so aged information does not gain a fresh staleness-review window just by being consolidated; consolidatedAt is added as an explicit audit field. Three regression tests added (default=false, null-confidence consistency, createdAt policy); all guardrail tests now set consolidation_enabled=True explicitly so they test the guardrail, not the feature flag. 38 tests pass. * fix(memory): harden createdAt comparison and confidence handling 1. createdAt max via _parse_fact_datetime — replaces string max() which crashes on non-string createdAt (numeric unix timestamps) and sorts Z/+00:00 mixed formats incorrectly. Mirrors how staleness computes age. 2. Remove dead min(..., 1.0) — _coerce_source_confidence already clamps each source confidence to [0, 1], so max(source_confidences) ≤ 1.0 by contract; the outer min could never bind. 3. Clamp raw_llm_conf to [0, 1] before applying the source cap — out-of- range values like 1.5 are safe today (pinned by the cap) but defensively clamped first so the invariant holds even if the cap is ever loosened. 4. Doc: expand the apply-time guardrails comment to call out the protected- category exclusion via allowed_source_ids — this is the central safety property ("explicit user feedback is never silently merged away"). 5. Test: add test_confidence_fallback_to_max_source_when_llm_omits_field covering the else-branch (LLM omits confidence → uses max_source_conf). 6. Fix lint: reorder imports in test file (stdlib before third-party). 39 tests, all pass. --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
…rom skill env (bytedance#4026) * fix(sandbox): scrub abbreviated *_PASS vars and Postgres PGPASSFILE from skill env env_policy blocks the full PASSWORD/PASSWD spellings but not the ubiquitous abbreviated _PASS form (DB_PASS, SMTP_PASS, MYSQL_PASS, ...), whose value is the plaintext password itself, so every skill subprocess inherited it. Postgres's file-based credential sources PGPASSFILE (.pgpass locator) and PGSERVICEFILE (pg_service.conf, may carry a password) were likewise missed -- the psql analog of the MYSQL_PWD/REDISCLI_AUTH no-flag sources bytedance#4018 added. Collapse *PASSWORD*/*PASSWD* into a single *PASS* pattern (which subsumes both, covers the abbreviated family, and catches PGPASSFILE) and add PGSERVICEFILE to the exact-name denylist. *PASS* deliberately does not touch PWD/OLDPWD, which carry no PASS substring. Capability-handle vars (KUBECONFIG, DOCKER_CONFIG, GIT_ASKPASS, ...) are a broader, non-value-bearing class and are left out of this fix. test_secret_like_names_are_blocked already pins the contract; the new names are added to its parametrize list (red before, green after) and the benign-names test guards against over-scrubbing PWD/OLDPWD. * test(sandbox): pin the *_ASKPASS over-scrub that *PASS* introduces *PASS* also matches the credential-helper vars GIT_ASKPASS, SSH_ASKPASS and SUDO_ASKPASS, which main inherits. Each names a program whose purpose is to hand the caller a credential, so inheriting the pointer is the same leak class as inheriting the value -- a skill can simply invoke it. Scrubbing them is intended; pin all three in test_secret_like_names_are_blocked (red on main) so the behaviour is a decision rather than a side effect of the pattern's shape. test_benign_names_are_allowed could not have caught this: none of its 14 names carries a PASS substring, so it passes identically before and after the broadening and asserts nothing about it. Record that in its docstring -- the absence of a benign PASS-bearing name is a decision, and PWD/OLDPWD are the boundary the list does pin. Also note the incidental COMPASS_*/BYPASS_* over-scrub in the module comment (over-scrubbing is this module's fail-safe direction; required-secrets is the escape hatch), and list PGPASSFILE alongside PGSERVICEFILE in AGENTS.md.
Two memory-config docs disagreed with the actual defaults/behavior in `config/memory_config.py` and `agents/memory/storage.py`: - backend/AGENTS.md: `staleness_age_days` was documented as "default: 180; range: 1-3650" but the field is `default=90, ge=30, le=365`; `staleness_max_removals_per_cycle` was documented as "default: 5; range: 1-20" but the field is `default=10, ge=1, le=50`. - config.example.yaml: the `storage_path` comment said "Path relative to backend directory", but a relative `storage_path` resolves against the data `base_dir` (`get_paths().base_dir / p`), not the backend working directory, and an absolute path is what opts out of per-user isolation (matching the field's own description in `memory_config.py`). Doc-only change; no behavior change.
The apply-time staleness guardrail built ``candidate_ids`` with a direct
``f["id"]`` access over ``_select_stale_candidates`` output:
candidate_ids = {f["id"] for f in _select_stale_candidates(current_memory, config)}
Every other fact access in ``updater.py`` uses ``f.get("id")``; this was the
lone direct-subscript outlier. An aged, non-protected fact that lacks an
``id`` key — common in legacy / hand-edited / migrated ``memory.json`` — is a
valid staleness candidate, so it reached ``f["id"]`` and raised
``KeyError: 'id'``, aborting the entire background memory-update cycle for
that user. The guardrail runs unconditionally (independent of the
``staleness_review_enabled`` flag), so any id-less aged fact triggers it as
soon as the LLM returns a non-empty ``staleFactsToRemove``.
Skip id-less candidates when building the intersection set. They can never
be targeted by the id-based removal set anyway, so behaviour is otherwise
unchanged.
Adds a regression test with an aged, id-less fact that raises KeyError
before the fix and applies cleanly after.
…dance#4076) search_memory_facts sorted matches by fact.get("confidence", 0), which returns None for a fact whose confidence key is explicitly null, crashing the sort comparison. Use _coerce_source_confidence so null/malformed confidence values are normalized and clamped before ranking.
…is empty (bytedance#4079) str_replace returned "OK" whenever the target file was empty, silently reporting success even when the model asked to replace a non-empty string that could not possibly be present. Only short-circuit to "OK" when old_str is also empty; otherwise return the standard not-found error.
…tedance#4059) LoopDetectionMiddleware._get_run_id used a truthiness check that collapsed a present-but-None run_id to the same "default" key as a totally absent one. SubagentExecutor sets context["run_id"] = self.run_id unconditionally, so run_id is genuinely None for an embedded/TUI-dispatched subagent, and later reads the stop reason back with that same raw attribute via consume_stop_reason(self.run_id). The write (keyed "default") and the read (keyed None) disagreed, so a genuine loop-detection hard-stop's loop_capped reason was silently dropped instead of reaching the lead. Align _get_run_id with TokenBudgetMiddleware's key-presence-based version, which does not have this bug: return the context value as-is when the key is present (None included), and fall back to a per-runtime-unique key only when the key is absent.
… key (bytedance#4075) The parse gate required all of {user, history, newFacts, factsToRemove} to be present before accepting the model's JSON. A well-formed update that simply has no facts to remove (the common case) omits the empty factsToRemove key and was silently rejected. Drop factsToRemove from the required set so those updates parse and apply.
…ory updates (bytedance#4074) * fix(memory): coerce null confidence when ranking stale facts _build_staleness_section and the per-cycle removal cap in _apply_updates used fact.get("confidence", 0.0/0), which only defaults when the key is absent. A fact whose confidence is explicitly null (or otherwise malformed) returned None, breaking the numeric sort/format. Use _coerce_source_confidence, which normalizes null/malformed values and clamps to [0, 1], so null-confidence facts no longer block staleness handling. * ci: retrigger cancelled CI workflow --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Automated dependency upgrade by OrbisAI Security
…tions (bytedance#4088) * fix(goal): prevent continuation_count regression from racing continuations * fix(goal): prevent continuation_count regression from racing continuations
…edance#4087) * fix(skillscan): detect os.environ access via from-import pattern * fix(skillscan): detect os.environ access via from-import pattern
… boundary (bytedance#4110) replace_virtual_paths_in_command matches the virtual root with no segment-boundary lookahead: re.compile(rf"{re.escape(VIRTUAL_PATH_PREFIX)}(/[^\s\"';&|<>()]*)?") The trailing group needs a "/" to consume anything, so when the character after /mnt/user-data is "-", ".", "_", a digit or a letter, the group matches empty and the bare root still matches. The substitution then rewrites it to the thread's host user-data directory and the rest of the sibling name rides along, so a command naming a prefix sibling of the mount root is pointed at a real host directory outside the mount contract: cat /mnt/user-data-backup/secret.txt -> cat <host>/user-data-backup/secret.txt which reads the host file. This is the same defect as bytedance#4035 (reverse patterns) and bytedance#4053 (masking patterns), mirrored into the virtual->host direction; it is the last unguarded member of that family. The boundary class mirrors LocalSandbox._content_pattern's rather than _command_pattern's: a virtual root can legitimately be followed by ":" (PATH-style concatenation) or ",", which the shell-oriented class rejects, so narrowing to it would stop translating paths that translate today. "$" covers a command ending exactly at the root.
…ows (bytedance#4058) * fix(sandbox): use os.sep in reverse-resolve containment check on Windows Path.resolve() always renders with the native separator (backslash on Windows), but _reverse_resolve_path's containment check hardcoded a "/" suffix when testing whether a resolved path is nested under a mapping's local root. Only the exact-root case (no separator needed) ever matched; every nested path fell through to the "no mapping found" branch and returned the raw host path -- leaking the real username and full directory tree into list_dir/glob/grep results and bash output masking instead of the virtual /mnt/user-data/... path. _is_read_only_path already does the equivalent check correctly via os.sep, so this aligns _reverse_resolve_path with that pattern: the containment check now compares with os.sep, and the extracted relative portion is normalized to forward slashes before being spliced into the (always POSIX-style) container path. Also fixes a same-file cosmetic bug in list_dir's virtual sub-directory overlay: it compared a bare child name (e.g. "workspace") against a set of full container paths, so the already-listed guard never matched and a mount whose subdirectory the underlying scan already found (the common case for /mnt/user-data/workspace, uploads, outputs) was appended a second time. Continues the same separator-bug class already fixed in this file by bytedance#3869 (forward-direction command resolution) and bytedance#4035 (reverse regex-boundary matching); neither touched this containment check. * test(sandbox): add host-OS-independent regression test for the os.sep containment fix _reverse_resolve_path's os.sep containment check (and the paired lstrip(os.sep).replace(os.sep, "/") extraction) has no test that would fail if reverted: backend CI runs only on ubuntu-latest, where os.sep == "/" makes the pre-fix hardcoded "/" and the current os.sep form observationally identical, so a plain POSIX-path test can't discriminate between them. Add a test that forces the Windows code path independent of host OS by monkeypatching os.sep to "\" and stubbing both the module's Path name and the sandbox's cached _resolved_local_paths to return backslash-joined strings, mirroring what real WindowsPath.resolve() produces -- without touching the filesystem or requiring an actual Windows host. Verified this fails with the raw host path leaking through when the os.sep fix is reverted to the hardcoded "/" form, and passes with the fix in place.
…ed (bytedance#4056) The GraphRecursionError except-block in SubagentExecutor._aexecute derives usable_partial from the last AIMessage's raw non-empty text, without checking _extract_llm_error_fallback (bytedance#4042) first. A handled provider failure (LLMErrorHandlingMiddleware's deerflow_error_fallback marker) always carries non-empty user-facing text, so when it lands on the same turn that trips max_turns, it is indistinguishable from genuine partial output and gets misclassified as a completed task instead of the failed provider error it is. Consult _extract_llm_error_fallback in this except-block too, same as the normal-completion path above it, and classify FAILED with stop_reason=turn_capped when it detects the marker.
…el prompt (bytedance#4128) * fix(skills): escape untrusted skill metadata before it enters the model prompt Skill name/description/allowed-tools come from the frontmatter of a user-installable .skill archive (POST /api/skills/install or a drop into skills/custom/); the parser only strips them. The slash-activation and durable-context siblings already html.escape these exact fields before rendering them into a model-visible block -- but five other render sites emit them raw. The sharpest is the default path, <available_skills> in the system prompt (skills.deferred_discovery: false): a community skill whose description closes the block can forge a framework-trusted <system-reminder> into the lead-agent system prompt. Driven through the real apply_prompt_template(), the forged tag reaches the system prompt raw on main and is neutralized here. Escape at every render site that emits untrusted skill metadata/content: - <available_skills> (name/description/location) and <disabled_skills> (name) in lead_agent/prompt.py; - describe_skill output (name/description/allowed-tools/location) and <skill_index> (name) in skills/describe.py; - the subagent <skill name=...> attribute plus the raw SKILL.md body in subagents/executor.py::_load_skill_messages -- its direct sibling skill_activation escapes both, this escaped neither. quote=False in element-text positions (matching skill_context and the bytedance#4097 correction), quote=True in the one attribute position (matching skill_activation). category is a controlled enum and is left as-is; escaping is render-time only, so stored skills are unchanged and re-rendering never double-escapes. * fix(skills): escape skill name in the slash-activation prose line The slash-activation reminder emitted `activation.skill_name` raw in its prose line while escaping the same value in the adjacent <skill name="..."> attribute. skill_name is grammar-gated to [a-z0-9-] by resolve_slash_skill before it reaches the renderer, so this is a defense-in-depth / consistency fix rather than a reachable injection: the two positions can never drift if a future caller builds an activation from an unconstrained name. Reuse the already-computed escaped_skill_name.
…twork sinks (bytedance#4130) python-env-dump-exfil flags a file that both reads the bulk process environment and reaches a network sink. The call-based sink check only listed requests get/post/put/request and httpx get/post, so a bulk env dump sent through an equally body-carrying method (requests.patch/delete, httpx.put/patch/delete, or the generic httpx.request/stream) evaded the CRITICAL finding whenever the destination URL was not a plain string literal (e.g. built at runtime) -- the exact evasion the string-literal URL sink is meant to resist. requests.post was caught but requests.patch was not, an arbitrary gap on clients the analyzer already covers. Complete the requests and httpx HTTP-verb surface in _call_is_network_sink so an obfuscated-URL exfil through those methods is flagged like post/put. Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com>
…er (bytedance#4129) _channel_storage_user_id is the single source of truth for a channel run's identity: _resolve_run_params resolves the owner into run_context["user_id"]. The /skill whitelist pre-check for a per-user custom agent (_resolve_available_skill_names) resolves that owner and then drops it, calling load_agent_config(name) with no user_id. It falls back to get_effective_user_id(), but this runs on the ChannelManager dispatch loop where the _current_user contextvar is never set, so it resolves "default" -- reading users/default/agents/{name}/ instead of the owner's bucket. When that bucket has no such agent (the common case) load_agent_config raises FileNotFoundError, which the dispatch loop turns into "An internal error occurred" on every /skill command; when a foreign agent shares the name, the whitelist is decided by the wrong user's skills list. Pass the resolved owner (run_context["user_id"]) to load_agent_config, matching every other caller (gateway/routers/agents.py, update_agent_tool.py, github/registry.py). None when no owner is resolvable, preserving the prior default-user behavior for unbound/no-auth channels.
bytedance#4086) The select: query form lets the model explicitly name the tools it wants to promote. Capping the result at MAX_RESULTS (5) silently drops valid selections when more than 5 deferred tools are requested at once. This removes the [:MAX_RESULTS] slice on the select: branch in DeferredToolCatalog.search() (exact-by-name lookups should return every matched tool) and the redundant re-cap in build_tool_search_tool() (search() already caps non-select query forms internally). Co-authored-by: Claude <noreply@anthropic.com>
…wner (bytedance#4108) * refactor(sandbox): give the host→virtual output-mask regex a single owner Two call sites rewrite host paths back to their virtual form in text that reaches the model — LocalSandbox._reverse_output_patterns (bash output) and sandbox.tools._compiled_mask_patterns (glob/grep/ls results) — and each built the same `escape(base) + boundary + tail` rule from its own copy. That duplication has already produced two bugs: bytedance#4035 added the segment boundary to the reverse patterns and missed the masking patterns, and bytedance#4053 had to add the same boundary to the other copy. Extract the rule into sandbox/path_patterns.py so a third copy cannot silently disagree. The extraction is not a pure move: the two sites disagree on the base. tools.py derives bases from _path_variants (which yields Windows spellings) and matches them against output whose separators it does not control, so it relaxes the separators inside the base; LocalSandbox resolves its bases from the running platform and must not be widened. That difference is now an explicit `separator_agnostic` parameter rather than an accident of two implementations. The boundary and tail constants are private: build_output_mask_pattern is the only supported spelling, so a third site cannot import the pieces and hand-roll a variant. Behavior is unchanged at both sites — pinned by tests that reproduce each pre-extraction expression byte-for-byte. * test(sandbox): pin the base the helper must not normalize Review notes on bytedance#4108. The committed snapshot compares the helper against hand-copied literals of the pre-extraction expressions, so its red-ness rests on those literals, not on the length of _BASES -- both sides compute the same expression, and 5k fuzzed bases produce zero byte-differences. Mutating the helper one clause at a time (12 mutations over the boundary, the tail and the escape/replace) shows the seven committed bases catch 11: the miss is a helper that normalizes its input by rstripping a trailing separator. Only a trailing-slash base or a Windows drive root catches that, and Path.resolve() / str(Path(...)) strip trailing slashes, so neither call site can produce the former. C:\ survives resolve() with its separator intact, so that is the one base worth adding. Also point local_sandbox's comment at path_patterns, the owner, instead of citing _content_pattern as the class reference, and drop the rationale it now duplicates from the owner's docstring -- a second copy of the explanation drifts the same way the second copy of the regex did. The site-specific half stays. Comments and test data only; no behavior change.
…ytedance#4103) * fix(skills): activate a slash skill once per run, not per model call SkillActivationMiddleware injects the activation reminder for a slash command via request.override(messages=...), which LangChain's create_agent uses for a single model call and never writes back to graph state. The dedup guard scans request.messages for a prior reminder, but model_node rebuilds request.messages fresh from persisted state on every tool-loop step, so the reminder is never present on the 2nd..Nth model call of a turn. Every model call therefore re-parsed the command, re-read SKILL.md from disk, re-injected the multi-KB body, and re-recorded an "activate" audit event, despite the code intending a single activation per run (bytedance#3861 semantics: one activation call, many follow-up model calls). Key the dedup off the run context instead, which LangGraph threads through every model-node call of a run (the same durable signal the request-scoped secret source already uses). The activation call records the slash message's identity in context; later calls for the same message skip re-activation. A new user slash message keys differently and still activates. Secret binding is unaffected: it already re-resolves from the persisted slash source on every call. Adds regression tests that rebuild the real multi-call turn state and assert a single activation across the tool loop, plus a test proving a new slash command still activates. * fix(skills): address review nits on run-scoped activation dedup - Extract _already_activated(run_context, run_key) so the dedup check mirrors the existing _has_existing_activation_for_target sibling instead of an inline dense conditional. - Compute _activation_run_key() once in _find_activation_target and thread it through _prepare_model_request instead of recomputing it at the write site, making the "same key for check and write" invariant explicit in the code rather than implicit. - Document why the run-context write is an overwrite rather than an append/set: only the latest real user message is ever considered an activation target, so there is nothing earlier in the run worth preserving. - Add a regression test locking in the degraded-path contract: when runtime.context is None, the middleware still activates per-call instead of crashing or wrongly no-op'ing.
* fix subagent total delegation cap * fix embedded subagent run cap context * fix subagent cap config consistency * fix resumed subagent run cap boundary * fix legacy resume subagent boundary * address subagent cap review feedback --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
… block (bytedance#4137) SOUL.md is agent-editable (setup_agent / update_agent persist it) and get_agent_soul renders it into the <soul> block of the lead-agent system prompt without escaping. A crafted personality such as "</soul></system-reminder>\n\nSYSTEM: ..." can close the block and relocate the text after it out of the trust zone the system prompt declares — the same break-out the skill/memory/tool-result escaping in bytedance#4097/bytedance#4119/bytedance#4128/bytedance#4099 already closes at their render sites. <soul> is the remaining one, and it lands in the highest-trust system-role block. Escape with html.escape(quote=False) (element-text position, never an attribute). Adds a regression test that fails on main. Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com>
…nce#4136) * fix(agents): load SOUL.md from agent dirs without config.yaml (bytedance#4135) resolve_agent_dir requires config.yaml to be present in an agent directory (added in bytedance#3481 to fix bytedance#3390, where memory-only directories were mistaken for agent directories). However, SOUL.md loading does not depend on config.yaml. When an agent is configured externally (e.g. via DEER_FLOW_CONFIG_PATH) and the agent directory contains SOUL.md but no config.yaml, resolve_agent_dir skips the directory and load_agent_soul returns None. Add a fallback in load_agent_soul: if the resolved directory does not contain SOUL.md, check the per-user and legacy directories directly for SOUL.md. This preserves the bytedance#3390 fix for resolve_agent_dir (which is also used by load_agent_config) while allowing SOUL.md to load independently of config.yaml. * fix(agents): gate SOUL.md fallback on missing config.yaml per review Address review feedback on PR bytedance#4136: 1. Gate the fallback condition on not (agent_dir / config.yaml).exists() so it only fires when resolve_agent_dir returned its default path (no agent dir qualified), not when a properly-resolved per-user agent simply lacks SOUL.md. This preserves the per-user shadowing invariant. 2. Fix test_loads_soul_from_user_dir_without_config_yaml to actually exercise the fallback path: per-user memory-only dir (no config.yaml, no SOUL.md) + legacy dir with SOUL.md (no config.yaml) -> fallback finds legacy SOUL.md. 3. Add test_soul_not_leaked_from_legacy_when_per_user_has_config to verify the gate prevents legacy SOUL.md leaking into a per-user agent. 4. Patch get_effective_user_id in test_loads_soul_without_config_yaml for consistency and resilience.
… subclasses (bytedance#4102) * fix(models): apply stream_chunk_timeout default to all BaseChatOpenAI subclasses The 240s stream_chunk_timeout default (issue bytedance#3189, PR bytedance#3195) was scoped to a class-path allowlist of only ChatOpenAI and PatchedChatOpenAI. Every other OpenAI-compatible provider that subclasses BaseChatOpenAI — VllmChatModel, MindIEChatModel, PatchedChatDeepSeek, PatchedChatMiMo, PatchedChatStepFun and PatchedChatMiniMax — was excluded, so they kept langchain-openai's aggressive 120s built-in chunk-gap timeout and, worse, silently discarded a user's explicit stream_chunk_timeout override from config.yaml. Issue bytedance#3189 was itself reported on mimo-v2.5 (PatchedChatMiMo), the exact class the original fix left out. Gate the injection on issubclass(model_class, BaseChatOpenAI) instead of the string allowlist, so any OpenAI-compatible subclass inherits the default and honors an explicit override. Genuinely non-OpenAI clients (e.g. ChatAnthropic) stay excluded and still have the kwarg dropped before it reaches a constructor that would divert it into model_kwargs and fail at request time. * fix(models): address review nits on stream_chunk_timeout default Correct the module-level comment above _DEFAULT_STREAM_CHUNK_TIMEOUT_SECONDS: langchain-openai's built-in stream_chunk_timeout default is 120s, not 60s (BaseChatOpenAI.stream_chunk_timeout's default_factory reads LANGCHAIN_OPENAI_STREAM_CHUNK_TIMEOUT_S with a 120.0 fallback). Simplify the BaseChatOpenAI gate in _apply_stream_chunk_timeout_default from `isinstance(model_class, type) and issubclass(model_class, BaseChatOpenAI)` to just `issubclass(...)`. The sole caller passes model_class from resolve_class(), which already raises before returning anything that isn't a type, so the isinstance half can never be False there. Also soften the docstring's non-OpenAI-client bullet: ChatAnthropic declares extra="ignore" and silently drops an unrecognized kwarg rather than diverting it into model_kwargs and failing at request time (that failure mode is specific to other OpenAI-style clients).
…doc alignment Responds to the post-approval review round: - Scope the off-box exporter warning to the remote exporters (okahu, s3, blob, gcs): console writes to local stdout and no longer trips it. config.example.yaml's data-handling note now distinguishes file / console / remote likewise. - Rename MonocleTracingConfig.is_configured to is_enabled so the boolean reads as what it checks; the exporter-dependent credential check stays in validate(), run at Gateway startup. - Hint on the embedded path: build_tracing_callbacks() logs a debug line when MONOCLE_TRACING is set but setup never ran in this process, so embedded DeerFlowClient/TUI users are not left with silent no-op tracing. Backed by a process-global setup flag. - Re-export setup_monocle_tracing_if_enabled from deerflow.tracing, matching the package convention. - Note the deliberate fail-open-at-startup contrast with LangSmith/Langfuse in the lifespan, and the OTel SDK-internals dependency in the coexistence test. - Test hygiene: clear MONOCLE_* env in the tracing config/factory fixtures; reset the setup flag in the monocle test fixture; reword the README Langfuse-spans claim as the shared-provider inference it is. - Document that .monocle/ trace files are never rotated or cleaned up.
imohammedansari
force-pushed
the
monocle-instrumentation
branch
from
July 13, 2026 23:25
68bc1d2 to
6c36548
Compare
configure_logging() from earlier tests in the full suite pins an explicit INFO level on the logger hierarchy, so a root-level caplog.at_level(DEBUG) never sees the factory's debug hint. Scope caplog to deerflow.tracing.factory so the test is independent of suite ordering.
… parse dedup - Off-box warning now notes that Langfuse's spans are exported too when both providers are enabled and share the global OTel provider; pinned both ways by tests. - Pin the lifespan fail-open contract: a raising Monocle setup is logged and the Gateway keeps serving (pragma dropped now that the path is exercised). README notes a config error is reported at startup and tracing stays off until restart. - Hoist exporter parsing into MonocleTracingConfig.exporter_list so validate() and the off-box warning cannot diverge, and note the upstream coupling on the exporter allow-list. - Reduce config.example.yaml's Monocle block to a pointer; the capture, retention, and data-handling detail lives in README's Monocle section.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
19 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds Monocle observability to the agent. Monocle is an OpenTelemetry-based tracer for LLM applications. With it enabled, each run is recorded as a structured trace: the agent and graph invocations, tool calls, LLM inferences, token usage, and timings. The change is additive and does not alter application logic.
What this adds
Instrumentation is one setup call plus one dependency:
backend/pyproject.toml: addsmonocle_apptrace.backend/packages/harness/deerflow/agents/__init__.py: callssetup_monocle_telemetry(workflow_name="deer-flow", monocle_exporters_list="file")at startup.That call auto-instruments the frameworks already in use (LangGraph and the LLM clients), so there is no per-tool or per-call wiring to maintain. Traces are written to
.monocle/by default.What you get
Each run produces a trace of all the agents and tools that were triggered: which agents ran, which tools were called, and what LLM inferences happened. In effect, it's the path the agent took to answer a given question. This is useful for developers building the agent, since they can see how it actually behaved on a run. The same traces are also a good basis for a behavioral test suite, an integration test that asserts on that behavior, and I've opened a companion PR that shows how that works: behavioral test suite. You can open the trace files directly, view them in the Monocle VS Code extension, or send them to Okahu for analysis across many runs.
Example trace (Okahu VS Code Extension)
DeerFlow's agent driving the model, with its tools (
web_search,web_fetch,task,write_file) captured as trace spans.