Skip to content

feat(llm-logs): name the virtual key a request used, and the user it stands for - #7499

Merged
joeyorlando merged 5 commits into
mainfrom
task-envs/3fcv5u
Aug 27, 2026
Merged

feat(llm-logs): name the virtual key a request used, and the user it stands for#7499
joeyorlando merged 5 commits into
mainfrom
task-envs/3fcv5u

Conversation

@archestra-task-envs

@archestra-task-envs archestra-task-envs Bot commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

Closes T-1172.

The bug

Virtual API keys exist so that proxied traffic can be attributed — to a
person, or to a named shared credential. The LLM Proxy logs never said
which key a request used.

interactions has carried the answer since virtual keys were
introduced: virtual_key_id and passthrough_virtual_key_id
(backend/src/database/schemas/interaction.ts). Nothing on the read
path resolved either. getSessions selects auth_method and joins
users on interactions.user_id, and never looks at the key columns,
so:

  • The sessions list Agent column rendered the agent over
    No user — shared key — correct, and useless. It says a virtual key
    identified nobody without saying which key that was.
  • The request detail Metadata card rendered Auth Method: Virtual Key and stopped there.

The user column is empty for a reason worth keeping straight. The proxy
takes a user identity from a virtual key in exactly one place
(llm-proxy-handler.ts):

if (virtualResult.virtualKeyScope === "personal") {
  regularVirtualKeyUserId = virtualResult.virtualKeyAuthorId ?? undefined;
}

A team- or org-scoped key attributes to nobody by design — which #7216
already explained with unattributedReason: "shared_virtual_key". What
was missing is the other half: which shared key, and for a personal
key, whose.

1. Resolve the key on the read path

New InteractionVirtualKeySchema — id, name, scope, key type, token
prefix, owner — filled by two read paths:

  • SessionSummary.virtualKeys: every key the session's requests used,
    both columns, deduplicated and ordered by name. The ids are
    aggregated in the existing phase-2 session query; the names are
    resolved by one batched VirtualApiKeyModel.findSummariesByIds in
    phase 3, next to the external-agent-name resolution. Batching rather
    than aliasing virtual_api_keys into the aggregate twice keeps a
    query that already groups over interactions from growing two more
    joins.
  • GET /api/interactions/:id returns virtualKey and
    passthroughVirtualKey, scoped to the caller's organization — the
    same tenant-boundary care the route already takes for
    connectorName, so a stale id cannot surface a key name from another
    organization.

The summary answers two questions, deliberately in separate fields:

  • Who does this key attribute traffic to? ownerUserId /
    ownerUserName, set only for a personal key's author, mirroring
    the proxy rule quoted above. A shared key resolves to null for both.
  • Who is this key shared with? teams for a team-scoped key (from
    virtual_api_key_team, which nothing on this path had ever queried),
    and createdByUserName for whoever set the key up, at any scope.

The split is the point. A shared key attributing to nobody is a fact
about attribution; it is not a reason to render the key as anonymous,
because the record already says who it belongs to. Keeping the creator
out of the owner field is what stops the second answer being read as the
first — a key's creator is not the person who made the request.

The schema is registered in z.globalRegistry, so the OpenAPI spec
emits one InteractionVirtualKey component instead of inlining the
object at each of the ~60 interaction response variants (7.2k generated
lines → 1.0k). Note also that .nullable() is fine here where #7216
needed z.union([Schema, z.null()]) — hey-api drops | null from
enums, not from object refs; both spellings were generated and
compared.

2. Name the key in the UI

VirtualKeyBadge renders the key name in the sessions list Agent
column and on the session detail, beside the user (or beside the badge
explaining why there is none). Extra keys collapse to +N; the tooltip
gives each key's kind, owner and token prefix.

The request detail gains a Virtual API Key row next to Auth Method,
listing the passthrough key before the standard one — the
passthrough key is what carried the acting user, the standard key only
supplied the provider credential. The label is deliberately not
"Virtual Key": that is already the auth-method value, and the two
would have rendered as the same string stacked on itself.

Each key renders one line naming what it belongs to:

personal              Virtual key · Alice Example
team                  Virtual key · shared with Platform, Security
team, no assignments  Virtual key · team key, no team assigned
org                   Virtual key · shared org-wide, created by Alice

The "no team assigned" case is called out rather than folded into a
generic "shared": a team key with every assignment removed is reachable
by nobody, which is worth seeing.

This also corrects copy the change made visible. The "No user — shared
key" tooltip claimed the request "used an organization-scoped virtual
key", which is wrong for a team-scoped one; same fix in the
shared_virtual_key doc comment.

Two layout fixes fall out of the cell this lands in: the Agent column
now stacks three lines so it takes ~25px more width, and
UnattributedUserBadge is capped and truncated — being w-fit shrink-0 it used to spill out of its cell into the Model column at
narrower viewports.

Notes

  • No migration. Both columns already exist and are already written by
    the proxy; this is read-path and UI only.
  • Tests pin the distinctions that are easy to get wrong: a shared key
    names itself and its teams but reports no owner, a personal key
    carries its owner and reports no teams even when junction rows survive
    a scope change, a request presenting both key kinds reports both, and
    a key belonging to another organization does not resolve.
  • Registering the schema as an OpenAPI component paid off on the second
    pass: adding teams and createdByUserName cost 53 generated lines
    rather than another thousand.
  • virtualKeys is not exposed as a log filter. Filtering sessions by
    key is a reasonable follow-up but wants an index on virtual_key_id,
    which interactions cannot get in a transactional migration.

Archestra Contributor

`interactions` has carried `virtual_key_id` and
`passthrough_virtual_key_id` since virtual keys existed, but nothing on
the read path ever resolved them. The logs could say `auth_method =
virtual_key` and no more, which is the least useful half of the answer
when per-user attribution is the reason a virtual key exists.

Adds `InteractionVirtualKeySchema` — id, name, scope, key type, token
prefix, and the user the key stands for — and two read paths that fill
it:

- `SessionSummary.virtualKeys`: every key a session's requests used,
  both columns, deduplicated and ordered by name. Aggregated as ids in
  the existing phase-2 session query and resolved in one batched lookup
  in phase 3, alongside the external-agent-name resolution, rather than
  aliasing `virtual_api_keys` into the aggregate twice.
- `GET /api/interactions/:id` returns `virtualKey` and
  `passthroughVirtualKey`, scoped to the caller's organization so a
  stale id cannot surface a key name across a tenant boundary.

`ownerUserId` / `ownerUserName` are populated only for a *personal*
key's author, matching the single place the proxy takes a user identity
from a virtual key (`llm-proxy-handler.ts`: `virtualKeyScope ===
"personal"`). A team- or org-scoped key is shared and stands for
nobody; naming whoever created it would read as an attribution the
proxy deliberately never makes — the same distinction
`unattributedReason: "shared_virtual_key"` already draws (#7216).

The schema is registered in `z.globalRegistry` so the spec emits one
component rather than inlining the object at each of the ~60 interaction
response variants.
The sessions list showed `LLM Proxy` over `No user — shared key`, and
the interaction detail's Metadata card showed `Auth Method: Virtual
Key`. Both say a virtual key was involved; neither says which one, or
who it stands for.

- `VirtualKeyBadge` renders the key's name in the sessions list Agent
  column and on the session detail, next to the user (or the
  unattributed badge explaining why there is none). Extra keys collapse
  to `+N`, and the tooltip spells out each key's kind, owner and token
  prefix.
- The interaction detail gains a `Virtual API Key` row beside `Auth
  Method`, listing the passthrough key before the standard one — the
  passthrough key is the one that carried the acting user, the standard
  key only supplied the provider credential. The label deliberately
  differs from the auth-method *value* "Virtual Key" so the two are not
  the same string stacked on top of each other.

A shared key reads "shared, no owner" rather than rendering blank: that
is the real answer, and it is what tells someone their traffic is
unattributable because the key is org- or team-scoped.

Also two layout fixes to the cell this lands in: the Agent column now
stacks three lines, so it gets ~25px more width, and
`UnattributedUserBadge` is capped and truncated — being `w-fit
shrink-0`, it used to spill out of the cell into the Model column at
narrower viewports.
The authentication page explains how each credential identifies a
caller but never says where that lands. Adds a short section on the
logs naming the virtual key, and on the difference a personal key makes
against a shared one.
The two key columns hold disjoint keys today — the Authorization path
rejects a passthrough key and the X-Archestra-Virtual-Key path rejects
a standard one — but concatenating them relied on that silently, and a
key reaching both would have rendered as two identical badges.
A shared key read "shared, no owner" and stopped there, which answers
the wrong question. It is true that a team- or org-scoped key attributes
to nobody — the proxy takes a user identity from a virtual key only when
the scope is `personal` — but that is a fact about *attribution*, and it
was standing in for a fact about *association* that the record already
holds. A shared key is not anonymous. It belongs to named teams, or to
the organization at somebody's hand.

`virtual_api_key_team` was simply never queried. The key summary now
carries:

- `teams` — the teams a team-scoped key is shared with, ordered by name.
  Empty for personal and org keys, and for a team key whose assignments
  were all removed (which renders as "no team assigned" rather than a
  bare "shared", since nobody can reach such a key).
- `createdByUserName` — who set the key up, at any scope.

Both come from the same batched lookup, so a page of sessions still
costs two queries for keys rather than one.

The two questions stay in separate fields on purpose. `ownerUserId` /
`ownerUserName` remain personal-only, so the creator of a shared key can
never be read as the person whose request was logged; the creator is
reported next to the sharing, not in the owner's place. Rendering
follows the split:

    personal              Virtual key · Alice Example
    team                  Virtual key · shared with Platform, Security
    team, no assignments  Virtual key · team key, no team assigned
    org                   Virtual key · shared org-wide, created by Alice

Also corrects copy this made visible: the "No user — shared key" tooltip
claimed the request "used an organization-scoped virtual key", which is
wrong for a team-scoped one. Same fix in the `shared_virtual_key` doc
comment on the backend.
@joeyorlando
joeyorlando added this pull request to the merge queue Aug 27, 2026
Merged via the queue into main with commit 5676c73 Aug 27, 2026
47 of 48 checks passed
@joeyorlando
joeyorlando deleted the task-envs/3fcv5u branch August 27, 2026 02:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant