fix: harden provider account credential lifecycle - #432
Conversation
…safety-current-main
JasonYeYuhe
left a comment
There was a problem hiding this comment.
Reviewed the full diff, the tests, CI, and ran the suite locally. The engineering here is strong — the transaction design is right, the failure modes are real ones, and nearly every structural risk I went looking for was already handled and had a dedicated test waiting for me. Details below.
One finding needs fixing before merge. It is not visible in CI, and it is caused by the isolation work rather than by the transaction work.
Blocking: test: isolate Claude helper state from XCTest wedges the offline suite on a developer Mac
swift test --package-path "CLI Pulse Bar/CLIPulseCore" on this branch hangs indefinitely on a machine that has Claude Code installed. Observed here: the run stopped at test 312 of ~2,715 and sat wedged for 17 hours until I killed it. SecurityAgent came up in the same minute as the xctest process and stayed up.
Hung test: ClaudeCollectorTests.testIsAvailableWithCLIBinary — untouched by this PR.
Mechanism
ClaudeCredentials.resolveTokenDetails tries sources in order (ClaudeSourceStrategy.swift):
4. env vars — absent
5. readCredentialsFile() — reads realHomeDir + "/.claude/.credentials.json"
6. readKeychainCredentials() — cross-app login-Keychain read
This PR adds an XCTest guard at line 249, so realHomeDir now returns a per-PID temp directory that is never created. But readKeychainCredentials() / SecItemCopyMatching at line 332 has no such guard.
So the isolation is half applied, and the half that landed makes things worse:
| step 5 | step 6 | |
|---|---|---|
| before | real ~/.claude/.credentials.json found → returns |
never reached |
| after | temp dir, file absent → falls through | cross-app Keychain read |
The function's own comment two lines above the call says it: "The sandboxed app cannot access cross-app keychain items without triggering a macOS authorization dialog." Removing the file that used to satisfy the lookup first is what routes execution onto that dialog.
Verified on this machine: ~/.claude/.credentials.json exists (509 bytes), and readCredentialsFile() reads it via realHomeDir.
Why CI is green
A headless runner has neither the credentials file nor a Claude Code Keychain item, so both step 5 and step 6 return nil immediately. The hang needs a real developer machine with Claude Code signed in — which is every machine that will actually run this suite locally.
This is the failure shape this repo keeps paying for: the check is green and the real environment is wedged. It also means the PR body's "CLIPulseCore offline suite: 2,715 tests, 4 skipped, 0 failures" is a CI result, not a local one.
Suggested fix
Guard the Keychain path the same way ClaudeHelperContract.appGroupHelperDir is already guarded in this PR:
public static func readKeychainCredentials(...) -> Creds? {
if isolatedTestHomeDirectory != nil { return nil }
...
}Worth adding a test that asserts the Keychain path is not consulted under XCTest — otherwise this reopens silently.
What I checked that turned out fine
Four things looked risky on the diff and are all correct — I am recording them so nobody re-derives them later:
| concern | finding |
|---|---|
save() holds the persistence lock and re-enters withMutationLock → deadlock? |
No. GeminiCredentialMutationLock is NSRecursiveLock + recursionDepth, and skips the file lock when depth > 0. The inner plain NSLock is never nested — the code inside it uses the unlocked private helpers. Covered by testAppSaveTransactionLockAllowsReentrantOwnerMutation. |
outbox Intent goes into a Set and enqueue defaults a fresh generation UUID → duplicate accumulation? |
No. enqueue filters same-(owner, accountID) records before inserting. Intent: Hashable includes generation, so markCompleted matches exactly — which is what makes a stale completion unable to clear a newer intent. |
commitProviderCredential has no paired rollback in the transaction |
Fine — the stated contract is that each primitive compensates itself, and commitAuthorization already carries epoch + operationID write markers and rollbackCredentialWrite. |
attribution count == 1 ? … : nil → multi-account users lose helper data? |
Only the per-account split is dropped; provider-level usage survives (providerResults.count == 1 is asserted). Graceful degradation, not data loss. |
release() returning true for kinds without a shared source is easy to read as a weakened check, but it is load-bearing: the same commit turns the call site into a guard, so without it every non-Claude/Gemini account deletion would fail. testReleaseForProviderWithoutSharedSourceIsSuccessfulNoOp pins it.
Tests are genuine rather than tautological — event-ordering assertions, and .failedRollbackIncomplete asserted distinctly from .failedRolledBack. The QARuntimeSideEffectPolicyTests change tightens the guard (deletedKeys.count == 2 → exact 4-key list). No pbxproj wiring risk: both new files are inside the SPM package.
The two red SwiftLint checks are genuinely pre-existing — same check fails on merged #425 and #427 (3,115 violations across 326 files repo-wide). Unrelated to this PR, but a permanently-red "warning-only" check can never report a new problem, which makes it as uninformative as a permanently-green one. Worth baselining separately.
I also independently confirmed the isolation claim that did land: nothing under ~/Library/Group Containers/group.yyh.CLI-Pulse/ was written during the run.
Non-blocking
- Ambiguous attribution empties the per-account view with no signal. Provider totals survive, so the user sees numbers — but their per-account breakdown vanishes with no explanation. A log line, or a hint in the UI, would keep this from reading as a bug.
- XCTest detection is inconsistent.
CookieResolveruses seven signals (XCTestConfigurationFilePath,XCTestBundlePath, bundle path, process name…);ClaudeCredentialsuses one (NSClassFromString("XCTestCase")). If seven are warranted in one place, one is a gap in the other — and the blocking issue above lives in exactly that gap. Suggest a single shared predicate. isolatedTestHomeDirectoryis per-PID, not per-test, so everything in one run shares an isolated home. Probably intentional, worth confirming.
Agreed on holding the signed-in provider-account smoke test before merge — with the Keychain guard added first, so the suite can actually finish locally.
Review prepared with Claude Code; findings verified against the code and reproduced locally.
Summary
Scope and acceptance criteria
Non-goals
Validation
git diff --check: passed.Risk and rollback
main.Agent involvement
Codex and bounded local coding workers assisted with implementation, test execution, review preparation, and evidence collection. Agents did not approve scope expansion, merge, release, or deployment.
Human ownership and review
Unfinished work