Skip to content

chore: land WIP — vLLM cache pinning, cleanup-archive tool, workstation plan - #57

Open
David-Martel wants to merge 4 commits into
mainfrom
chore/land-wip-vllm-cleanup-workstation-20260718
Open

chore: land WIP — vLLM cache pinning, cleanup-archive tool, workstation plan#57
David-Martel wants to merge 4 commits into
mainfrom
chore/land-wip-vllm-cleanup-workstation-20260718

Conversation

@David-Martel

Copy link
Copy Markdown
Owner

Lands the outstanding uncommitted work-in-progress as four coherent, signed clusters. No behavior change to the inference/media stack.

Clusters

  • feat(vllm) — Pin HF cache, vLLM runtime cache, and FunctionGemma model weights to explicit T:\Models roots (out of opaque Docker volumes); read-only model mount; matching .env.example + README.
  • feat(tools) — Add Invoke-CleanupArchive.ps1: rclone → Google Drive dated archive with md5+size verification and verify-gated local delete (dry-run default). Commits the executed 2026-06-26 manifest as archive evidence; gitignores transient *.dryrun.jsonl + rclone-*.log.
  • chore(reports) — Refresh Process Lasso governor watchdog snapshot (evidence only, no policy change).
  • docs(reports) — Workstation identity + Codex integration plan (research/planning; Phase 0–6; flags P0 exposed-secret rotation).

Validation

  • Invoke-CleanupArchive.ps1: AST parse OK (796 tokens), PSScriptAnalyzer clean (0 Error/Warning).
  • All 4 commits SSH-signed (ED25519, verified Good).
  • lefthook pre-commit (ast-grep, rust-fmt, ps-manifest) passed.

⚠️ CI note

The repo-wide CI gate is currently red on main (PowerShell Lint + Rust Clippy + Cross-Platform fail identically on unrelated PRs, incl. all 5 Dependabot bumps) — a pre-existing environmental failure, not introduced by this PR. Diagnosis + fix tracked separately; that fix also unblocks the Dependabot PRs (#49#53).

🤖 Generated with Claude Code

David-Martel and others added 4 commits July 18, 2026 11:07
Move the Hugging Face cache, vLLM runtime cache, and FunctionGemma model
weights out of opaque Docker volumes and default ~/.cache onto explicit
T:\Models roots so Windows-native inference and the WSL/Docker vLLM stack
share one storage surface.

- Add HF_HOME and VLLM_CACHE_ROOT container env; mount an explicit
  VLLM_CACHE_DIR volume and make the model weights mount read-only (:ro).
- Default HF_CACHE_DIR/VLLM_CACHE_DIR/PCAI_MODEL_DIR to T:\Models in the
  .env example.
- Document the T:\Models layout and the pre-create mkdir step in the README.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add Invoke-CleanupArchive.ps1: uploads explicit cleanup candidates to a
dated Google Drive archive folder via rclone, verifies each upload with
rclone check (md5 + size), records a JSONL manifest, and deletes the local
copy only after verification succeeds. Dry-run is the default; -Execute
uploads and -DeleteAfterVerify gates local removal on verified evidence.

- Commit the executed 2026-06-26 manifest as archive evidence (records a
  verified 18 GB IPSW upload+delete and the workstation-cleanup doc archive).
- Ignore transient *.dryrun.jsonl manifests and rclone-*.log transfer logs.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Re-run evidence for the Process Lasso governor watchdog: ProcessGovernor.exe
already running and responding (2026-07-18 snapshot, new PID). No policy or
startup change - evidence refresh only, per the boot/reliability evidence
policy.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Research-and-planning report (no credential, profile, OAuth, or MCP mutation
performed) covering the desired Google identity model, Workspace OAuth reauth
and scope contract, Chrome profile separation, PowerShell profile local-first
cleanup, Windows Terminal pinning, and dtm-codex home reconciliation, with a
phased Phase 0-6 execution plan. Flags P0 exposed-secret rotation (Context7
key, Google OAuth/ADC scratch artifacts) as prerequisite work.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings July 18, 2026 15:09

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces workstation reliability and cleanup tooling, including a new PowerShell script Invoke-CleanupArchive.ps1 to archive files to Google Drive via rclone, updated Docker Compose configurations for vLLM caching, and a detailed integration plan for workstation profiles and Google identities. The review feedback focuses on improving the robustness and efficiency of the archiving script by ensuring deterministic path resolution using $PSScriptRoot, resolving relative manifest paths to absolute paths to prevent runtime errors, and optimizing remote file verification by consolidating rclone calls into a single lsjson invocation.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment on lines +69 to +72
$manifestDir = Split-Path -Path $ManifestPath -Parent
if ($manifestDir -and -not (Test-Path -LiteralPath $manifestDir)) {
New-Item -ItemType Directory -Path $manifestDir -Force | Out-Null
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

If a relative path without a directory component (e.g., manifest.jsonl) is passed to -ManifestPath, Split-Path -Parent returns an empty string. This causes Join-Path on line 87 to throw a terminating runtime error because the -Path parameter cannot be empty.

Resolving $ManifestPath to a fully qualified absolute path using $ExecutionContext.SessionState.Path.GetUnresolvedProviderPathFromPSPath guarantees that $ManifestPath is always absolute, ensuring $manifestDir is never empty and preventing any downstream Join-Path failures.

    $ManifestPath = $ExecutionContext.SessionState.Path.GetUnresolvedProviderPathFromPSPath($ManifestPath)
    $manifestDir = Split-Path -Path $ManifestPath -Parent
    if ($manifestDir -and -not (Test-Path -LiteralPath $manifestDir)) {
        New-Item -ItemType Directory -Path $manifestDir -Force | Out-Null
    }

[string]$BatchId = '2026-06-26-cleanup-lane',
[string]$RemoteRoot = 'Archives/Workstation Cleanup',
[string]$RcloneRemote = 'gdrive-personal:',
[string]$ManifestPath = (Join-Path (Resolve-Path .) 'Reports/cleanup-archive-manifest-20260626.jsonl'),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Using Resolve-Path . to define the default ManifestPath makes it dependent on the current working directory of the shell at the time of execution. If the script is run from a subdirectory (e.g., Tools/), it will resolve to Tools/Reports/... instead of the repository's root Reports/ directory, leading to unexpected folder creation and duplicate manifests.

Using $PSScriptRoot ensures the path is always resolved relative to the script's location, making it deterministic regardless of where the script is invoked from.

    [string]$ManifestPath = (Join-Path (Split-Path -Path $PSScriptRoot -Parent) 'Reports' 'cleanup-archive-manifest-20260626.jsonl'),

Comment on lines +131 to +144
$remoteHashLine = (& rclone md5sum $remoteFile) | Select-Object -First 1
if ($LASTEXITCODE -ne 0) {
throw "rclone md5sum failed for $remoteFile"
}

$remoteSizeJson = (& rclone size $remoteFile --json) -join "`n"
if ($LASTEXITCODE -ne 0) {
throw "rclone size failed for $remoteFile"
}

$remoteSize = $remoteSizeJson | ConvertFrom-Json
$record.local_md5 = (($localHashLine -split '\s+', 2)[0]).ToLowerInvariant()
$record.remote_md5 = (($remoteHashLine -split '\s+', 2)[0]).ToLowerInvariant()
$record.remote_size_bytes = [int64]$remoteSize.bytes

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Verifying the remote file currently performs two separate external process invocations and network roundtrips to Google Drive: rclone md5sum and rclone size. This introduces unnecessary latency and overhead, especially when processing multiple files.

Using rclone lsjson --hash allows retrieving both the file size and the MD5 checksum in a single API call and process invocation, significantly improving execution efficiency.

                    $remoteMetaJson = & rclone lsjson --hash $remoteFile
                    if ($LASTEXITCODE -ne 0 -or -not $remoteMetaJson) {
                        throw "Failed to retrieve remote metadata for $remoteFile"
                    }

                    $remoteMeta = $remoteMetaJson | ConvertFrom-Json | Select-Object -First 1
                    $record.local_md5 = (($localHashLine -split '\\s+', 2)[0]).ToLowerInvariant()
                    $record.remote_md5 = $remoteMeta.Hashes.md5.ToLowerInvariant()
                    $record.remote_size_bytes = [int64]$remoteMeta.Size

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c6c30e95d8

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

}

$remoteDir = "$RcloneRemote$RemoteRoot/$BatchId/$Category"
$remoteFile = "$remoteDir/$($item.Name)"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Reject colliding archive destinations before deletion

When two SourcePath values have the same basename in the same batch/category, both are assigned this identical remote path; each iteration can verify its own upload and delete its local source, leaving only the final file at the shared archive location while both manifest entries claim successful retention. Detect duplicate destinations or preserve source-relative/unique names before allowing -DeleteAfterVerify.

Useful? React with 👍 / 👎.

Comment on lines +172 to +174
if ($Execute -or $RecordDryRun) {
$json = [pscustomobject]$record | ConvertTo-Json -Compress -Depth 5
Add-Content -LiteralPath $ManifestPath -Value $json -Encoding UTF8

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Prevent WhatIf runs from writing the manifest

With -Execute -WhatIf, ShouldProcess suppresses the upload and deletion, but this unconditional Execute branch still appends an execute:true record to disk; the earlier directory creation can also mutate the filesystem. This contradicts both RecordDryRun's opt-in contract and the AGENTS.md requirement that dry-run behavior suppress file writes, so manifest creation/appending must honor the preview decision.

Useful? React with 👍 / 👎.

Comment on lines +54 to +56
[switch]$Execute,
[switch]$RecordDryRun,
[switch]$DeleteAfterVerify

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Expose the required dry-run and help switches

This new write-capable workstation script cannot accept -DryRun, -h, or --help, so callers using the repository's standardized safe-run interface fail parameter binding instead of previewing the operation or displaying help. AGENTS.md explicitly requires write-capable session scripts to expose all three interfaces, together with tests proving that dry run suppresses side effects.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants