fix: four defensive fixes from a second-lens engine hunt - #113
Conversation
A refute-by-default adversarial hunt across concurrency, injection, SSRF,
DoS/ReDoS, edge-input, and the MCP/web boundary surfaced these. Each is a
reachable defect with a concrete repro; each fix carries a regression test.
- scrub: the private-key detector was O(n^2). The lazy BEGIN...END regex
scanned to end-of-string once per BEGIN header, so a crafted multi-MB blob
of BEGIN lines stalled the single event loop for seconds on one evidence
insert. A non-backtracking scanner that jumps BEGIN to END by index replaces
it; work is now linear.
- sync: a connector item whose source timestamp does not parse became a truthy
Invalid-Date watermark that pinned the cursor and crashed the unguarded
toISOString() outside the try/catch, wedging the connector on every run. It
is now treated as a missing timestamp.
- distill: two distills of the same evidence (a scheduled drain overlapping a
manual distill) both read an empty dedup set and each inserted the full node
set. A per-evidence advisory lock, mirroring the connector lock, serializes
them so the second sees the first's nodes and skips them.
- injection: the instruction-smell detector missed the anchor-as-object
override phrasing ("ignore the above and ...") because the pattern required
a trailing instructions/rules noun. Advisory only; now flagged.
Deferred to founder review, not shipped here: a HIGH Jira credential-exfil /
SSRF (jira.ts attaches the decrypted token to a config-controlled baseUrl with
no allowlist). Its exfil vector is the CSRF-able write endpoint that the
founder-gated PR #96 already closes, and the only exfil-stopping change in the
connector itself (a host allowlist) is a product call that would break
self-hosted Jira Data Center. Left for a human to land alongside #96.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 18bbef81f2
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| DISTILL_LOCK_NS, | ||
| evidenceId, | ||
| ]); | ||
| return await fn(); |
There was a problem hiding this comment.
Avoid exhausting the pool while holding a distill lock
Each call retains a pool client for the entire callback, but the callback passed by Marrow.distill immediately calls Store methods such as getNodesForEvidence and insert*, which acquire clients from that same pool. With the default pg pool size (10), ten concurrent distills for different evidence IDs can each acquire this lock client and then all wait indefinitely for a client to execute their first Store query; the same deadlock occurs with a one-connection pool. Keep the lock on the client used for the work, or reserve a separate connection/pool for advisory locks.
Useful? React with 👍 / 👎.
A second-lens adversarial hunt over the shipped engine (main #104), across the axes the first correctness hunt did not cover: concurrency/transactions, prompt injection, connector SSRF/secrets, DoS/ReDoS, edge-input integrity, and the MCP/web boundary. Every candidate had to survive an independent refute-by-default verifier. Five distinct defects were confirmed; the scarier
answer()double-decide candidate was correctly refuted by the skeptic and dropped.This PR ships the four rule-safe, defensive fixes (each with a regression test) and defers the one security-boundary finding to you (below).
Shipped here
scrub.tsBEGIN...ENDregex scanned to end-of-string once perBEGINheader, so a crafted multi-MB blob ofBEGINlines stalled the single event loop for seconds on one evidence insert (measured ~4.4s at 2MB, ~1000x amplification vs benign text).BEGINand jumps to the nextENDby index. Both cursors only move forward: linear.sync.tsNaN, so it pinned the cursor, then crashed the unguardedwatermark.toISOString()outside the try/catch. The cursor never advanced, so every later run refetched the same item and crashed identically: a permanent per-connector wedge on ordinary provider glitches.marrow.ts/store.tsseenset is built from an in-memory read and each insert mints a fresh UUID, so two distills of the same fresh evidence (a scheduleddistill --pendingoverlapping a manual distill) both read empty and each insert the full node set, duplicating every node. Retrieval and the neighbor walk then double-count.withDistillLock, its own namespace) mirroring the connector lock, so the second pass blocks, then sees the first's nodes and skips them. Distinct evidence still distills in parallel.injection.tsSacred rules preserved: no fix writes a status, sets
confidence.source='human', or promotes anything. The distill lock is advisory serialization only; the injection change is an advisory read-time badge; the sync and scrub fixes are pure input hardening.Deferred to you (not shipped): HIGH Jira credential-exfil / SSRF
connectors/jira.tsattaches the decryptedBasic email:apiTokento a fully config-controlledbaseUrlwith no host allowlist, so a config-only rewrite ofbaseUrl(the GET view never returns the token, but a re-upsert coalesces the old cipher) exfiltrates a live credential, and an internal https host turns the sink into SSRF.I did not autonomously ship this, for three reasons:
Recommend landing it as a human change alongside #96. Full repro is in the hunt output.
Verification (all from repo root, explicitly)
pnpm typecheckclean across all packagespnpm lintclean (eslint + prettier)pnpm test: core 367, mcp-server 34, web 116, cli 67; skipped 0pnpm db:migrateup to date (no migration needed),check-benchmark-driftclean,no-em-dash1/0,pnpm smoke:packedokwithDistillLockserialization x3; concurrent-distill no-duplicateOne patch changeset (
@marrowhq/core).Opened by the autonomous loop, which cannot merge its own PRs (two-party human review). Queued for your review.