fix(scale): fail closed when the asserted user root cannot be bound - #8657
Conversation
request_context_middleware caught ValueError from aset_user_root, logged a warning, and let the request proceed on the execution context's existing root, which is the system root. The system root bypasses the owner check and disables the traversal ACL gate, so any request path without a redundant downstream bind was fail-open at the tenant boundary. MissingAnchorError extends ValueError, so the swallow also covered a well-formed root id that resolves to no anchor, and the observable behavior was a 500 with a traceback from the second bind. An asserted caller identity that cannot be resolved now rejects the request with a 401 at the middleware; the exception detail stays in the server log for the operator. Requests that assert no identity are unchanged. Fixes jaseci-labs#8348
# Conflicts: # jac/jaclang/scale/server/native_server.jac
…8454 did not carry The middleware change this PR opened with landed on main as jaseci-labs#8454 (merged 2026-08-28), including `identity_unresolvable` in errors.jac and the (ValueError, RuntimeError) fail-closed branch in native_server.jac. Both production edits are dropped as duplicates. What is left is the tier jaseci-labs#8454 did not have: a real server subprocess, a real Postgres store, a SQL fault injected into identity_users, and the 401 asserted over HTTP with a positive control and an unaffected-user control. jaseci-labs#8454's test_middleware_root_bind_refused.jac covers the same decision one tier down, through make_client.
|
CI note: the red lanes on this PR are not from this branch. Recording the
1.
A markdown file cannot cause that. Both #8654 and #8664 went 2. Whole-tree
Same 913 files, same "912 passed, 1 failed", different victim each run, and Other people's PRs look green only because their runs predate 0.37 (newest is Happy to file the |
jac check's fork work pool assigns files to workers differently on every run, and a worker serving a file with an incomplete stub prelude reports stdlib types as Self/<Unknown>. One unrelated file fails per run. No source change here.
Summary
Fixes #8348.
request_context_middlewarecaughtValueErrorfromaset_user_root, logged"using system root", and let the request proceed on the execution context's
existing root - the system root, which bypasses the owner check
(
check_access_level) and disables the traversal ACL gate (_acl_gated).Today a redundant second bind in
_run_function_with_occhappens to kill suchrequests with a 500 traceback, so the tenant boundary holds by accident; any
request path without that second bind inherits a live fail-open. Because
MissingAnchorErrorextendsValueError, the swallow covered both a malformedroot id and a well-formed id that resolves to no anchor.
The middleware now fails closed: an asserted caller identity that cannot be
bound rejects the request with a deliberate 401 (
UNAUTHORIZEDenvelope), andthe exception detail goes to the server log for the operator. Requests that
assert no identity (guest,
:pub) are unchanged. The rejection is returnedinside the existing try/finally so context, auth, and tracing cleanup still run.
Test (real subprocess, real SQL, no mocks)
New
scale/tests/server/test_root_bind_fail_closed.jac: realjac run --servesubprocess; the fault is injected exactly the way the issuemeasured it - real SQL against the server's identity store
(
UPDATE identity_users SET doc = doc || ...) through a fixture endpoint.Covers both variants (well-formed-but-unresolvable UUID, malformed id), with a
positive control before the fault and a fresh-user control after.
Fails on pre-fix code with
unresolvable root must be rejected with 401, got 500; passes with the fix.Could this have been less code?
error_responseinruntime/context/errors.jac(added a 4-lineunauthorizedhelper besideservice_unavailable/gateway_timeout).aset_user_rootitself: rejected - it is aruntimelib primitive whose callers legitimately handle failure differently
(the OCC path heals via
reset_root); the middleware is where "assertedidentity" is known.
rejected - that reproduces today's observable bug (500 + traceback) rather
than a deliberate 401.
Deleted
itself. Nothing else became dead: the downstream bind in
_run_function_with_occstays, as first-line resolution for paths that donot traverse this middleware.
What else could this break
Grepped
aset_user_root,user_root_id,unauthorized,UNAUTHORIZEDcallsites:
traceback; they now get a structured 401. The issue names this as the
expected behavior.
reset_roothealing in_abegin_user_request_contextruns only under
JAC_SV_SIBLINGand binds through its own path, not thismiddleware branch; unaffected (the middleware only rejects when the id it
was handed cannot bind).
user_idunset skips the bind entirely;/healthz,/healthz/liveshort-circuit before this code. Unchanged.pod and in
jac run --serve; behavior is identical. kind vs EKS, embeddedvs external DB: not touched (the store only matters as the source of the
corrupt id). Multi-replica: per-pod middleware, same rejection on each.
when the bind fails - previously the masking crash was the only thing
standing between them and fail-open.