π΅οΈ Problem / motivation
The MCP server's audit journal proves that an entry was not altered after the fact: Ed25519 signature, hash chain, high-water-mark anchor. It does not prove who produced it. Every identity field in a tool_call entry is copied from what the client declares: client comes from clientInfo in the initialize params (MCP/src/main/java/org/sikuli/mcp/server/McpDispatcher.java:142), llm comes from _meta.llm (McpDispatcher.java:302), and both land untouched in the entry (MCP/src/main/java/org/sikuli/mcp/audit/AuditEntry.java:35-36). Nothing the server observes on its own is written anywhere. The bouncer writes down the name you give him, and never asks for ID.
I proved it today on my machine, same jar, two callers, two journals:
| caller |
client in the entry |
llm |
Claude Code, stdio run |
{"name":"claude-code","version":"2.1.258", β¦} |
null |
me, with curl.exe in a PowerShell, HTTP serve |
{"name":"humain-en-powershell","version":"1.0"} |
null |
The second name is the one I typed into the command. Had I written "name":"claude-code", the two entries would have been indistinguishable, and somebody could have blamed a language model for clicking my button. The README says "every action is attributable to a specific client, session and LLM backend". Today that reads: attributable to whatever the caller said it was. For a regulated deployment, that is a hole a curl walks through without ducking.
π§Ύ Proposed solution
Add a provenance block to every tool_call entry, filled exclusively from what the operating system and the transport hand the server, never from the request body, and placed under the entry hash and the signature like every other field. The declared client and llm stay as they are; provenance is the witness that can contradict them. The client tells its version, the OS tells its own, both are signed side by side. May the better story win.
What goes in
Common to both transports
transport: stdio or http
server_pid, server_user: the JVM's own pid and OS account, from ProcessHandle.current().info()
handshake: the protocolVersion the client proposed (McpDispatcher.java:149), the capabilities keys it declared, whether notifications/initialized was ever sent (McpDispatcher.java:124), and the delay in ms between initialize and the first tools/call. An SDK sends the notification; a human hand-rolling three curl calls forgets it nine times out of ten. I speak from experience, it was tonight.
stdio (run) β captured once at startup in cmdRun (MCP/src/main/java/org/sikuli/mcp/cli/Main.java:110), carried into McpServer and stored on the SessionContext minted at initialize (MCP/src/main/java/org/sikuli/mcp/server/SessionContext.java:41)
parent: pid, executable path, command line, OS user and start time of the process that spawned the server, plus the same for its parent, up to the session leader. Pure JDK, ProcessHandle.parent(). On my machine this yields claude.exe for Claude Code and pwsh.exe for a human. You can lie to a server; you cannot lie to your own parent process.
HTTP (serve) β captured per request in handlePost (MCP/src/main/java/org/sikuli/mcp/transport/HttpTransport.java:228), because the peer port changes with every connection
peer: source IP and port from HttpServerExchange.getSourceAddress(), recorded always, proxy or not
forwarded: the verbatim Forwarded, X-Forwarded-For, X-Forwarded-Proto and X-Real-IP headers when present. A proxy has an owner and an IP: the entry records both hops, the proxy's and the one the proxy vouches for, and an auditor decides which one to believe. Recorded regardless of OCULIX_MCP_TRUST_TLS_TERMINATION; that flag only changes which interface the server accepts to bind, not what it observes. The server may refuse to answer you; it never refuses to write you down.
user_agent and origin headers, verbatim (Origin is already examined at HttpTransport.java:95-105)
peer_process, loopback only: resolve the peer port to the owning process through the OS TCP table, then pid, executable, parent executable and OS user. Windows: GetExtendedTcpTable via the JNA already on the classpath. Linux: /proc/net/tcp inode lookup then /proc/<pid>/. macOS: proc_pidinfo / lsof -i. This is what turns "127.0.0.1:52144" into "curl.exe, parent pwsh.exe, user DELL". Off-loopback the field is null and peer + forwarded carry the attribution.
Tier 2 β the part nobody ships, and for good reason
Everything above says which process called. This tier says what that process is, who was at the machine, and what the machine was showing at the moment of the call. All of it read from the OS at call time, none of it declared, all of it under the signature. This is the tier that makes you want to read the terms of use before clicking.
caller_binary: SHA-256 of the caller's executable (parent in stdio, peer process on loopback HTTP) and, on Windows, its Authenticode signer subject (Anthropic, PBC for claude.exe, Microsoft Corporation for pwsh.exe, nothing for a home-built binary). A curl.exe renamed to claude.exe keeps its hash and its lack of signature. macOS: the code-signing Team ID via codesign. Linux: the package owner of the path when the distro knows it.
logon_session: the Windows logon session the caller belongs to, whether it is the interactive console session or a remote one, and for a remote one the RDP client name and client address (WTSQuerySessionInformation, WTSClientName / WTSClientAddress). A human over Remote Desktop leaves the name and IP of the machine they sat at. Linux/macOS: the tty and SSH_CLIENT / SSH_CONNECTION of the caller's environment when readable.
presence: milliseconds since the last physical keyboard or mouse input on the machine (GetLastInputInfo on Windows, CGEventSourceSecondsSinceLastEventType on macOS, /dev/input timestamps on Linux). A call issued 200 ms after a keystroke was ordered by someone at the keyboard; a call issued after 40 minutes of silence was not. This single number separates "attended" from "unattended" better than any declared field. The server will not know whether you were drinking coffee, but it will know nobody was touching the mouse.
foreground: the foreground window's process name and title at the instant of the call, so the entry says not only "click at 468,60" but "click at 468,60 while Xray Sample Project β Jira β Chrome had focus". Title only, no pixels: confidential mode stays confidential.
screen_before: a perceptual hash of the screen before the action (64-bit dHash, not an image), so an auditor can later prove the screen was, or was not, the one the action was meant for, without storing a single pixel in the journal.
cadence: per session, the median and jitter of inter-call intervals so far. Machine rhythm is regular to the millisecond; human rhythm is not, except for very well-drilled QA engineers, and I know a few.
host: hostname, OS build, machine identifier, and the server jar's own SHA-256, so an entry from a tampered or rebuilt server is distinguishable from one produced by the released binary. The journal audits its own author too; it is only polite.
None of this refuses anything. It records. The point is that a journal line stops being "a caller said X" and becomes "a signed binary, from this logon session, with a human at the keyboard or not, looking at this window, on this host, said X". That is the sentence a regulator wants, and no automation server writes it today. The blob eats everything, but it keeps books its accountant would envy.
Verdict, with the evidence next to it
caller_class: agent, interactive, or unknown, derived from the above (parent or peer executable known as an MCP host, capabilities non-empty, notifications/initialized seen, cadence). Purely a convenience for the reader: the raw fields are what an auditor relies on, and the verdict never overrides client. The server has an opinion; it states it at the bottom of the page, with its sources.
Where it plugs in
AuditEntry: one new field, provenance (AuditEntry.java:21-43), included in CanonicalJson so it sits under entry_hash and the signature.
JournalWriter.buildEntry already accepts an extra object (MCP/src/main/java/org/sikuli/mcp/audit/JournalWriter.java:205-208); appendToolCall passes null today (JournalWriter.java:116-118). Either route provenance through a dedicated field or through extra; a dedicated field keeps extra for markers as documented.
McpDispatcher.auditSafely (McpDispatcher.java:288-294) receives the session context; the per-request HTTP bits travel on the SessionHandle set in handlePost before dispatcher.dispatch (HttpTransport.java:308).
JournalVerifier: unchanged in logic; entries with and without provenance must both verify, so the field is optional in the canonical form and old journals stay green.
verify output gains one line per file: how many entries carry provenance, how many caller_class values of each kind.
Non-goals
- Not a replacement for the client credential (
OCULIX_MCP_TOKEN) or for mTLS. Those prove possession of a secret; provenance proves what the OS saw. The two together are what "attributable" should mean.
- No attempt to fingerprint the human.
os_user is an account, not a person; shared accounts stay shared, and the server has no webcam.
- No blocking: provenance is recorded, never used to refuse a call. Refusal stays with
ActionGate.
π Alternatives considered
- Per-client tokens with the
kid written in the entry. Proves which secret was used, not from where or by which process. Complementary, not sufficient: a leaked token still says "Claude Code", with great confidence.
- Trust
clientInfo and _meta.llm. Today's behaviour. Free-text, self-declared, indistinguishable from a forged declaration.
- Log at the reverse proxy. Only exists in HTTP deployments behind a proxy, and the correlation with the signed journal is manual. The journal should be self-sufficient.
π Where would this live?
MCP server.
π§© API impact
No: internal, additive audit field. Journals written before this change verify unchanged.
π― Use case / impact
Anyone running the server in serve mode on a shared host, and anyone who needs to answer "was this click done by the agent or by someone at the keyboard" after the fact. Today the answer is "whatever the caller typed in clientInfo". After this, the answer is the parent executable, the OS account, the peer address and every proxy hop, all under the same signature as the action itself.
π Additional context
Reproduction from today, both on oculix-mcp-server.jar built from the current tree: one oculix_find_text from Claude Code over stdio, one from a PowerShell curl.exe over HTTP with a made-up clientInfo.name. Both entries verify. Neither entry contains a single byte the caller did not choose.
π£οΈ Opinions wanted
@RaiMan β you spent fifteen years teaching SikuliX to read the screen; tell me if you see a reason the server should not also read who is talking to it.
@adriancostin6 β the peer_process part on Linux and macOS, /proc/net/tcp and proc_pidinfo, is your turf; if you know a case where port β process resolution lies, I want to hear it before I write it.
@davidyoung8196504567-sudo β you are the one who put JNA loading under the microscope on Legerix#20; this proposal is the same kind of thing, prove instead of declare. I am particularly interested in your feedback on this one, including telling me that one of the tier 2 fields is a bad idea.
π¦
π΅οΈ Problem / motivation
The MCP server's audit journal proves that an entry was not altered after the fact: Ed25519 signature, hash chain, high-water-mark anchor. It does not prove who produced it. Every identity field in a
tool_callentry is copied from what the client declares:clientcomes fromclientInfoin theinitializeparams (MCP/src/main/java/org/sikuli/mcp/server/McpDispatcher.java:142),llmcomes from_meta.llm(McpDispatcher.java:302), and both land untouched in the entry (MCP/src/main/java/org/sikuli/mcp/audit/AuditEntry.java:35-36). Nothing the server observes on its own is written anywhere. The bouncer writes down the name you give him, and never asks for ID.I proved it today on my machine, same jar, two callers, two journals:
clientin the entryllmrun{"name":"claude-code","version":"2.1.258", β¦}nullcurl.exein a PowerShell, HTTPserve{"name":"humain-en-powershell","version":"1.0"}nullThe second name is the one I typed into the command. Had I written
"name":"claude-code", the two entries would have been indistinguishable, and somebody could have blamed a language model for clicking my button. The README says "every action is attributable to a specific client, session and LLM backend". Today that reads: attributable to whatever the caller said it was. For a regulated deployment, that is a hole acurlwalks through without ducking.π§Ύ Proposed solution
Add a
provenanceblock to everytool_callentry, filled exclusively from what the operating system and the transport hand the server, never from the request body, and placed under the entry hash and the signature like every other field. The declaredclientandllmstay as they are;provenanceis the witness that can contradict them. The client tells its version, the OS tells its own, both are signed side by side. May the better story win.What goes in
Common to both transports
transport:stdioorhttpserver_pid,server_user: the JVM's own pid and OS account, fromProcessHandle.current().info()handshake: theprotocolVersionthe client proposed (McpDispatcher.java:149), thecapabilitieskeys it declared, whethernotifications/initializedwas ever sent (McpDispatcher.java:124), and the delay in ms betweeninitializeand the firsttools/call. An SDK sends the notification; a human hand-rolling threecurlcalls forgets it nine times out of ten. I speak from experience, it was tonight.stdio (
run) β captured once at startup incmdRun(MCP/src/main/java/org/sikuli/mcp/cli/Main.java:110), carried intoMcpServerand stored on theSessionContextminted at initialize (MCP/src/main/java/org/sikuli/mcp/server/SessionContext.java:41)parent: pid, executable path, command line, OS user and start time of the process that spawned the server, plus the same for its parent, up to the session leader. Pure JDK,ProcessHandle.parent(). On my machine this yieldsclaude.exefor Claude Code andpwsh.exefor a human. You can lie to a server; you cannot lie to your own parent process.HTTP (
serve) β captured per request inhandlePost(MCP/src/main/java/org/sikuli/mcp/transport/HttpTransport.java:228), because the peer port changes with every connectionpeer: source IP and port fromHttpServerExchange.getSourceAddress(), recorded always, proxy or notforwarded: the verbatimForwarded,X-Forwarded-For,X-Forwarded-ProtoandX-Real-IPheaders when present. A proxy has an owner and an IP: the entry records both hops, the proxy's and the one the proxy vouches for, and an auditor decides which one to believe. Recorded regardless ofOCULIX_MCP_TRUST_TLS_TERMINATION; that flag only changes which interface the server accepts to bind, not what it observes. The server may refuse to answer you; it never refuses to write you down.user_agentandoriginheaders, verbatim (Originis already examined atHttpTransport.java:95-105)peer_process, loopback only: resolve the peer port to the owning process through the OS TCP table, then pid, executable, parent executable and OS user. Windows:GetExtendedTcpTablevia the JNA already on the classpath. Linux:/proc/net/tcpinode lookup then/proc/<pid>/. macOS:proc_pidinfo/lsof -i. This is what turns "127.0.0.1:52144" into "curl.exe, parentpwsh.exe, userDELL". Off-loopback the field isnullandpeer+forwardedcarry the attribution.Tier 2 β the part nobody ships, and for good reason
Everything above says which process called. This tier says what that process is, who was at the machine, and what the machine was showing at the moment of the call. All of it read from the OS at call time, none of it declared, all of it under the signature. This is the tier that makes you want to read the terms of use before clicking.
caller_binary: SHA-256 of the caller's executable (parent in stdio, peer process on loopback HTTP) and, on Windows, its Authenticode signer subject (Anthropic, PBCforclaude.exe,Microsoft Corporationforpwsh.exe, nothing for a home-built binary). Acurl.exerenamed toclaude.exekeeps its hash and its lack of signature. macOS: the code-signing Team ID viacodesign. Linux: the package owner of the path when the distro knows it.logon_session: the Windows logon session the caller belongs to, whether it is the interactive console session or a remote one, and for a remote one the RDP client name and client address (WTSQuerySessionInformation,WTSClientName/WTSClientAddress). A human over Remote Desktop leaves the name and IP of the machine they sat at. Linux/macOS: the tty andSSH_CLIENT/SSH_CONNECTIONof the caller's environment when readable.presence: milliseconds since the last physical keyboard or mouse input on the machine (GetLastInputInfoon Windows,CGEventSourceSecondsSinceLastEventTypeon macOS,/dev/inputtimestamps on Linux). A call issued 200 ms after a keystroke was ordered by someone at the keyboard; a call issued after 40 minutes of silence was not. This single number separates "attended" from "unattended" better than any declared field. The server will not know whether you were drinking coffee, but it will know nobody was touching the mouse.foreground: the foreground window's process name and title at the instant of the call, so the entry says not only "click at 468,60" but "click at 468,60 whileXray Sample Project β Jira β Chromehad focus". Title only, no pixels: confidential mode stays confidential.screen_before: a perceptual hash of the screen before the action (64-bit dHash, not an image), so an auditor can later prove the screen was, or was not, the one the action was meant for, without storing a single pixel in the journal.cadence: per session, the median and jitter of inter-call intervals so far. Machine rhythm is regular to the millisecond; human rhythm is not, except for very well-drilled QA engineers, and I know a few.host: hostname, OS build, machine identifier, and the server jar's own SHA-256, so an entry from a tampered or rebuilt server is distinguishable from one produced by the released binary. The journal audits its own author too; it is only polite.None of this refuses anything. It records. The point is that a journal line stops being "a caller said X" and becomes "a signed binary, from this logon session, with a human at the keyboard or not, looking at this window, on this host, said X". That is the sentence a regulator wants, and no automation server writes it today. The blob eats everything, but it keeps books its accountant would envy.
Verdict, with the evidence next to it
caller_class:agent,interactive, orunknown, derived from the above (parent or peer executable known as an MCP host,capabilitiesnon-empty,notifications/initializedseen, cadence). Purely a convenience for the reader: the raw fields are what an auditor relies on, and the verdict never overridesclient. The server has an opinion; it states it at the bottom of the page, with its sources.Where it plugs in
AuditEntry: one new field,provenance(AuditEntry.java:21-43), included inCanonicalJsonso it sits underentry_hashand the signature.JournalWriter.buildEntryalready accepts anextraobject (MCP/src/main/java/org/sikuli/mcp/audit/JournalWriter.java:205-208);appendToolCallpassesnulltoday (JournalWriter.java:116-118). Either route provenance through a dedicated field or throughextra; a dedicated field keepsextrafor markers as documented.McpDispatcher.auditSafely(McpDispatcher.java:288-294) receives the session context; the per-request HTTP bits travel on theSessionHandleset inhandlePostbeforedispatcher.dispatch(HttpTransport.java:308).JournalVerifier: unchanged in logic; entries with and withoutprovenancemust both verify, so the field is optional in the canonical form and old journals stay green.verifyoutput gains one line per file: how many entries carry provenance, how manycaller_classvalues of each kind.Non-goals
OCULIX_MCP_TOKEN) or for mTLS. Those prove possession of a secret; provenance proves what the OS saw. The two together are what "attributable" should mean.os_useris an account, not a person; shared accounts stay shared, and the server has no webcam.ActionGate.π Alternatives considered
kidwritten in the entry. Proves which secret was used, not from where or by which process. Complementary, not sufficient: a leaked token still says "Claude Code", with great confidence.clientInfoand_meta.llm. Today's behaviour. Free-text, self-declared, indistinguishable from a forged declaration.π Where would this live?
MCP server.
π§© API impact
No: internal, additive audit field. Journals written before this change verify unchanged.
π― Use case / impact
Anyone running the server in
servemode on a shared host, and anyone who needs to answer "was this click done by the agent or by someone at the keyboard" after the fact. Today the answer is "whatever the caller typed inclientInfo". After this, the answer is the parent executable, the OS account, the peer address and every proxy hop, all under the same signature as the action itself.π Additional context
Reproduction from today, both on
oculix-mcp-server.jarbuilt from the current tree: oneoculix_find_textfrom Claude Code over stdio, one from a PowerShellcurl.exeover HTTP with a made-upclientInfo.name. Both entries verify. Neither entry contains a single byte the caller did not choose.π£οΈ Opinions wanted
@RaiMan β you spent fifteen years teaching SikuliX to read the screen; tell me if you see a reason the server should not also read who is talking to it.
@adriancostin6 β the
peer_processpart on Linux and macOS,/proc/net/tcpandproc_pidinfo, is your turf; if you know a case where port β process resolution lies, I want to hear it before I write it.@davidyoung8196504567-sudo β you are the one who put JNA loading under the microscope on Legerix#20; this proposal is the same kind of thing, prove instead of declare. I am particularly interested in your feedback on this one, including telling me that one of the tier 2 fields is a bad idea.
π¦