feat(llm-logs): name the virtual key a request used, and the user it stands for - #7499
Merged
Conversation
`interactions` has carried `virtual_key_id` and `passthrough_virtual_key_id` since virtual keys existed, but nothing on the read path ever resolved them. The logs could say `auth_method = virtual_key` and no more, which is the least useful half of the answer when per-user attribution is the reason a virtual key exists. Adds `InteractionVirtualKeySchema` — id, name, scope, key type, token prefix, and the user the key stands for — and two read paths that fill it: - `SessionSummary.virtualKeys`: every key a session's requests used, both columns, deduplicated and ordered by name. Aggregated as ids in the existing phase-2 session query and resolved in one batched lookup in phase 3, alongside the external-agent-name resolution, rather than aliasing `virtual_api_keys` into the aggregate twice. - `GET /api/interactions/:id` returns `virtualKey` and `passthroughVirtualKey`, scoped to the caller's organization so a stale id cannot surface a key name across a tenant boundary. `ownerUserId` / `ownerUserName` are populated only for a *personal* key's author, matching the single place the proxy takes a user identity from a virtual key (`llm-proxy-handler.ts`: `virtualKeyScope === "personal"`). A team- or org-scoped key is shared and stands for nobody; naming whoever created it would read as an attribution the proxy deliberately never makes — the same distinction `unattributedReason: "shared_virtual_key"` already draws (#7216). The schema is registered in `z.globalRegistry` so the spec emits one component rather than inlining the object at each of the ~60 interaction response variants.
The sessions list showed `LLM Proxy` over `No user — shared key`, and the interaction detail's Metadata card showed `Auth Method: Virtual Key`. Both say a virtual key was involved; neither says which one, or who it stands for. - `VirtualKeyBadge` renders the key's name in the sessions list Agent column and on the session detail, next to the user (or the unattributed badge explaining why there is none). Extra keys collapse to `+N`, and the tooltip spells out each key's kind, owner and token prefix. - The interaction detail gains a `Virtual API Key` row beside `Auth Method`, listing the passthrough key before the standard one — the passthrough key is the one that carried the acting user, the standard key only supplied the provider credential. The label deliberately differs from the auth-method *value* "Virtual Key" so the two are not the same string stacked on top of each other. A shared key reads "shared, no owner" rather than rendering blank: that is the real answer, and it is what tells someone their traffic is unattributable because the key is org- or team-scoped. Also two layout fixes to the cell this lands in: the Agent column now stacks three lines, so it gets ~25px more width, and `UnattributedUserBadge` is capped and truncated — being `w-fit shrink-0`, it used to spill out of the cell into the Model column at narrower viewports.
The authentication page explains how each credential identifies a caller but never says where that lands. Adds a short section on the logs naming the virtual key, and on the difference a personal key makes against a shared one.
The two key columns hold disjoint keys today — the Authorization path rejects a passthrough key and the X-Archestra-Virtual-Key path rejects a standard one — but concatenating them relied on that silently, and a key reaching both would have rendered as two identical badges.
A shared key read "shared, no owner" and stopped there, which answers
the wrong question. It is true that a team- or org-scoped key attributes
to nobody — the proxy takes a user identity from a virtual key only when
the scope is `personal` — but that is a fact about *attribution*, and it
was standing in for a fact about *association* that the record already
holds. A shared key is not anonymous. It belongs to named teams, or to
the organization at somebody's hand.
`virtual_api_key_team` was simply never queried. The key summary now
carries:
- `teams` — the teams a team-scoped key is shared with, ordered by name.
Empty for personal and org keys, and for a team key whose assignments
were all removed (which renders as "no team assigned" rather than a
bare "shared", since nobody can reach such a key).
- `createdByUserName` — who set the key up, at any scope.
Both come from the same batched lookup, so a page of sessions still
costs two queries for keys rather than one.
The two questions stay in separate fields on purpose. `ownerUserId` /
`ownerUserName` remain personal-only, so the creator of a shared key can
never be read as the person whose request was logged; the creator is
reported next to the sharing, not in the owner's place. Rendering
follows the split:
personal Virtual key · Alice Example
team Virtual key · shared with Platform, Security
team, no assignments Virtual key · team key, no team assigned
org Virtual key · shared org-wide, created by Alice
Also corrects copy this made visible: the "No user — shared key" tooltip
claimed the request "used an organization-scoped virtual key", which is
wrong for a team-scoped one. Same fix in the `shared_virtual_key` doc
comment on the backend.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes T-1172.
The bug
Virtual API keys exist so that proxied traffic can be attributed — to a
person, or to a named shared credential. The LLM Proxy logs never said
which key a request used.
interactionshas carried the answer since virtual keys wereintroduced:
virtual_key_idandpassthrough_virtual_key_id(
backend/src/database/schemas/interaction.ts). Nothing on the readpath resolved either.
getSessionsselectsauth_methodand joinsusersoninteractions.user_id, and never looks at the key columns,so:
No user — shared key— correct, and useless. It says a virtual keyidentified nobody without saying which key that was.
Auth Method: Virtual Keyand stopped there.The user column is empty for a reason worth keeping straight. The proxy
takes a user identity from a virtual key in exactly one place
(
llm-proxy-handler.ts):A team- or org-scoped key attributes to nobody by design — which #7216
already explained with
unattributedReason: "shared_virtual_key". Whatwas missing is the other half: which shared key, and for a personal
key, whose.
1. Resolve the key on the read path
New
InteractionVirtualKeySchema— id, name, scope, key type, tokenprefix, owner — filled by two read paths:
SessionSummary.virtualKeys: every key the session's requests used,both columns, deduplicated and ordered by name. The ids are
aggregated in the existing phase-2 session query; the names are
resolved by one batched
VirtualApiKeyModel.findSummariesByIdsinphase 3, next to the external-agent-name resolution. Batching rather
than aliasing
virtual_api_keysinto the aggregate twice keeps aquery that already groups over
interactionsfrom growing two morejoins.
GET /api/interactions/:idreturnsvirtualKeyandpassthroughVirtualKey, scoped to the caller's organization — thesame tenant-boundary care the route already takes for
connectorName, so a stale id cannot surface a key name from anotherorganization.
The summary answers two questions, deliberately in separate fields:
ownerUserId/ownerUserName, set only for a personal key's author, mirroringthe proxy rule quoted above. A shared key resolves to
nullfor both.teamsfor a team-scoped key (fromvirtual_api_key_team, which nothing on this path had ever queried),and
createdByUserNamefor whoever set the key up, at any scope.The split is the point. A shared key attributing to nobody is a fact
about attribution; it is not a reason to render the key as anonymous,
because the record already says who it belongs to. Keeping the creator
out of the owner field is what stops the second answer being read as the
first — a key's creator is not the person who made the request.
The schema is registered in
z.globalRegistry, so the OpenAPI specemits one
InteractionVirtualKeycomponent instead of inlining theobject at each of the ~60 interaction response variants (7.2k generated
lines → 1.0k). Note also that
.nullable()is fine here where #7216needed
z.union([Schema, z.null()])— hey-api drops| nullfromenums, not from object refs; both spellings were generated and
compared.
2. Name the key in the UI
VirtualKeyBadgerenders the key name in the sessions list Agentcolumn and on the session detail, beside the user (or beside the badge
explaining why there is none). Extra keys collapse to
+N; the tooltipgives each key's kind, owner and token prefix.
The request detail gains a
Virtual API Keyrow next toAuth Method,listing the passthrough key before the standard one — the
passthrough key is what carried the acting user, the standard key only
supplied the provider credential. The label is deliberately not
"Virtual Key": that is already the auth-method value, and the two
would have rendered as the same string stacked on itself.
Each key renders one line naming what it belongs to:
The "no team assigned" case is called out rather than folded into a
generic "shared": a team key with every assignment removed is reachable
by nobody, which is worth seeing.
This also corrects copy the change made visible. The "No user — shared
key" tooltip claimed the request "used an organization-scoped virtual
key", which is wrong for a team-scoped one; same fix in the
shared_virtual_keydoc comment.Two layout fixes fall out of the cell this lands in: the Agent column
now stacks three lines so it takes ~25px more width, and
UnattributedUserBadgeis capped and truncated — beingw-fit shrink-0it used to spill out of its cell into the Model column atnarrower viewports.
Notes
the proxy; this is read-path and UI only.
names itself and its teams but reports no owner, a personal key
carries its owner and reports no teams even when junction rows survive
a scope change, a request presenting both key kinds reports both, and
a key belonging to another organization does not resolve.
pass: adding
teamsandcreatedByUserNamecost 53 generated linesrather than another thousand.
virtualKeysis not exposed as a log filter. Filtering sessions bykey is a reasonable follow-up but wants an index on
virtual_key_id,which
interactionscannot get in a transactional migration.