Skip to content

skill: opening asks only report language; infer the audit tier from code (+ AWS publish fix) - #28

Merged
Gandy2025 merged 2 commits into
mainfrom
fix/opening-language-only
Jul 7, 2026
Merged

skill: opening asks only report language; infer the audit tier from code (+ AWS publish fix)#28
Gandy2025 merged 2 commits into
mainfrom
fix/opening-language-only

Conversation

@Gandy2025

Copy link
Copy Markdown
Contributor

What

Two independent skill-only changes on the production-scan skill.

1. Opening asks only the report language; the tier is inferred from the code

The trial's first touch was two chat ask cards — language, then a 4-question calibration questionnaire (scale / data / stage / revenue) that picked the audit tier. Live testing showed this over-asks at the first touch.

Now: the opening is a single language ask; the audit tier is inferred from the code — default Launch-ready, raised to Scale on hard signals (a Stripe key, a charges/subscriptions table, thousands+ users), dropped to Hobby only on unmistakable toy-project evidence. The report states the tier was inferred and offers a one-line correction, instead of asking.

Why the questionnaire could go: the four questions were highly correlated (≈1.5 real dimensions), half-inert (Q3/Q4 barely moved the tier), and their one high-signal dimension — data/money — is inferable from the code anyway. The one genuinely-uninferable dimension (expected scale) mostly gates Scale-only extras a pre-launch vibe-coded app hasn't built yet, and the one-line correction recovers the rare miss. The language ask is kept because it's high-impact (the whole report renders in it), hard to infer (non-English devs routinely ship English repos), and trivial to answer.

Folded-in review fixes:

  • cache_key (step 1.2): dropped the calibration fields — they're inferred at step 1.4, after the cache check, so unavailable there. Key is now owner-repo-WT-report_lang-scan_depth; the inferred tier is a pure function of the code that WT already fingerprints, so it's deliberately not in the key.
  • tier_source: added downgraded-to-hobby so the Hobby-inference case has a truthful label.
  • language ask keeps a narrow exception: if the user already wrote in a language before the scan, match it; otherwise it MUST ask (no regression of the scan-without-asking bug).
  • README + tiers.md + secrets.md + deploy-config.md synced to the inferred model.

2. Publish points the AWS CLI at the real credentials (not $HOME/.aws)

The hosted runtime sets HOME to the agent workspace, so the CLI's default ~/.aws lookup missed the real /home/ubuntu/.aws and the S3 publish failed with "Unable to locate credentials". Step 4.1 now exports AWS_SHARED_CREDENTIALS_FILE / AWS_CONFIG_FILE explicitly (file paths, not secrets), guarded on the file existing so off-runtime resolution is untouched. Verified live: HTML + JSON both returned HTTP 200.

Verification

  • Render smoke tests 17/17; full fixture render clean (no unfilled tokens).
  • cache_key bash run in isolation across en/zh/日本語/Français → valid single-path-segment keys, deterministic, language-separated.
  • AWS guard run in both branches (file absent → skip, no error; present → export); sh -n clean.
  • 4 independent adversarial review rounds (consistency / LLM-obedience / downstream-contract / the 5 fixes) — all SHIP.
  • Not covered here (needs a live trial): whether the LLM agent actually asks only language and obeys the inferred-tier flow end-to-end.

Deferred (not in this PR)

When private repos enter scan scope, report_key should become an opaque ~128-bit hash (drop the owner-repo-date prefix) and the pre-consent auto-publish should be revisited — the current legible key + 32-bit hash is appropriate only while scans are public-repo-only.

Gandy2025 and others added 2 commits July 8, 2026 00:49
Cut the 4-question calibration questionnaire from the opening — the trial's
first touch is now a single language ask. The audit tier is inferred from the
code (default Launch-ready; raised to Scale on hard signals like a Stripe key or
a users/password table, dropped to Hobby only on unmistakable toy evidence). The
report states the tier was inferred and offers a one-line correction, instead of
asking the user up front. The four calibration questions were redundant (highly
correlated), half-inert (Q3/Q4 barely moved the tier), and their one high-signal
dimension (data/money) is inferable from the code anyway.

Review fixes folded in:
- cache_key (step 1.2): drop the calibration fields — they are inferred at step
  1.4, AFTER this cache check, so unavailable here. Key is now
  owner-repo-WT-report_lang-scan_depth. The inferred tier is a pure function of
  the code that WT already fingerprints, so it is deliberately not in the key.
- tier_source: add "downgraded-to-hobby" so the Hobby-inference case has a
  truthful label (schema + step 0.2 + step 1.4).
- language ask keeps a narrow exception: if the user already wrote a real message
  in a language before the scan, match it; otherwise MUST ask (no regression of
  the scan-without-asking bug).
- README + tiers.md + secrets.md + deploy-config.md synced to the inferred model.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
….aws

The hosted trial runtime sets HOME to the agent workspace, so the AWS CLI's
default ~/.aws lookup resolves under the workspace and misses the real
credentials at /home/ubuntu/.aws — the S3 publish then fails with "Unable to
locate credentials" even though the file is present. Export
AWS_SHARED_CREDENTIALS_FILE / AWS_CONFIG_FILE explicitly before the uploads
(file paths, not secrets). Guarded on the file existing so off-runtime the
default ~/.aws resolution is left intact. Verified live: HTML + JSON both
returned HTTP 200 after this.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@Gandy2025
Gandy2025 merged commit 0021aa4 into main Jul 7, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant