Add usage-stats collection endpoint - #6
Merged
Conversation
Adds a proxy-authed `usage` POST endpoint (parallel to `manifest`) that ingests the monthly rollup rows produced by `rootstock usage report` (Garden-AI/rootstock#47), plus a public GET /usage on the dashboard app. Rollups are stored per cluster per month (/data/usage/<cluster>/<YYYY-MM>.json) so a push replaces only the months it contains — server-side history survives clusters whose local spool is purged (e.g. Perlmutter PSCRATCH). User hash lists are stripped on ingest: only derived counts leave a cluster. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
OwenPriceSkelly
added a commit
to Garden-AI/rootstock
that referenced
this pull request
Jul 28, 2026
…169) The collector (#159) aggregates the usage spool; this adds the last hop from issue #47 — pushing the aggregated monthly rollup rows to the rootstock-admin backend's new usage endpoint (Garden-AI/rootstock-admin#6). - `rootstock usage push [--dry-run]` pushes the same aggregation `report` shows, filed under the manifest's cluster name. The backend stores rollups per month and replaces only the months present in a push, so repeated pushes (e.g. from the smoke-test cron) are idempotent. - Only derived counts are sent; the salted user hashes stay in the spool (summarize_spool already strips them). - The usage endpoint URL is derived from api_url by swapping the standard rootstock-admin function name (-manifest -> -usage); a new optional usage_api_url config field / ROOTSTOCK_USAGE_API_URL env var overrides it for non-standard deployments. - The success line names the endpoint it pushed to: dev-vs-prod mixups previously looked identical to success (the delta manifest incident). Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Backend half of the usage-stats follow-up from Garden-AI/rootstock#47: the collector side (
rootstock usage report/compact, Garden-AI/rootstock#159) produces monthly rollup rows, and this adds the endpoint they get pushed to.usagePOST (Modal proxy-auth, parallel tomanifest): accepts{"cluster": <name>, "rows": [...]}where rows are rollup rows keyed bymonth/cluster/env/checkpoint/device/clientwithsessions,n_calculations,duration_s,unique_users./data/usage/<cluster>/<YYYY-MM>.json: a push replaces only the months it contains, so history stored here survives clusters whose local spool gets purged (e.g. Perlmutter PSCRATCH). The subdir also keeps these files out of the dashboard's/data/*.jsonmanifest glob.usershash list sent by mistake is dropped on ingest. Cluster names and months are validated (also blocks path traversal into the volume).GET /usageon the public dashboard app returns all stored rollups.The client-side
rootstock usage pushlands separately in the main repo.Test plan
.local()with the volume stubbed: happy path (two months in one push), partial re-push replacing only included months,usersstripping, and 422s for missing/invalid cluster, non-list rows, and bad/missing months.modal deploy modal_app.py(dev), verify a round-tripPOST usage→GET /usage, thenmodal deploy --env=prod modal_app.py.🤖 Generated with Claude Code