You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Status: draft, pending review by @havardjohansen13 before circulation. No assignees, no broader @-mentions yet.
Context
Three consecutive Night Shift runs on the merkur-frontend + distriktssenteret-frontend sister sites have shown the same five patterns. Documented per-run in /Users/havardjohansen/Dev/night-shift/IMPROVEMENTS.md entries 2026-05-07, 2026-05-08, 2026-05-11 (private workspace file — happy to copy excerpts here on request).
Pattern 1 — Stale-base detection
Night Shift's per-task PR is correct relative to the branch checkout at run-start, but it doesn't see what the previous run already shipped. Result: DS produced word-for-word regenerations of fixes already on develop three days running.
Concrete examples from 2026-05-11:
DS #66 (security): 4th regeneration of the draft-mode auth + open-redirect fix (after docs: improvements plan (audit-driven roadmap) #24, #36, #49). develop already has SANITY_PREVIEW_SECRET validation + isRelativeUrl() in src/app/api/draft-mode/enable/route.ts. The PR would have regressed to SANITY_REVALIDATE_SECRET (wrong env var name).
DS #68 (perf): 100% duplicate. Every image sizes prop and the iframe loading=\"lazy\" already on develop from the 2026-05-08 bundle.
DS #67 (seo): most fixes already shipped (metadataBase, title.template, OG block, font swap). Only net-new item was the latin-ext Rubik subset.
Proposed fix: before opening a PR for a task, diff against not just develop but the most recent merged night-shift/* bundle (or the last N bundles if cadence is unclear). If the proposed change is structurally already present, skip the task with a brief log line.
Pattern 2 — DO-NOT-CHANGE policy blocks ignored
The 2026-05-11 DS #67 PR would have reverted <html lang=\"no\"> → lang=\"nb\" and og:locale: nn_NO → nb_NO. Both are explicitly forbidden by a block titled ## Language policy — DO NOT CHANGE near the top of the repo's CLAUDE.md:
Do not "fix" lang=\"no\" to lang=\"nn\" or lang=\"nb\" — that would misrepresent the rendered page.
This is the first time we've seen Night Shift actively regress a documented policy.
Proposed fix: when reading CLAUDE.md / AGENTS.md, parse ## (or deeper) headings whose text contains DO NOT CHANGE (or similar policy markers) and treat the contents as hard constraints. SEO task in particular should never modify lang/locale attributes if such a block exists.
Pattern 3 — Test PRs open with red CI
DS #64 (2026-05-11) and DS #48 (2026-05-08) both opened with red Cypress E2E. The new specs referenced components/tokens that exist on merkur but not yet on DS ([data-cy=\"ai-search-block\"], .u-head-sm, <h2.u-head-lg>).
Proposed fix: in the test-add task, after opening the PR, poll gh pr checks for N minutes (or gh run watch). If E2E fails, automatically close the PR with a comment explaining the failure — don't leave it for a human to triage. Better still: run the new tests locally against the project's dev server before opening the PR.
Three days, 12+ closed PRs across this category, zero merged. Pattern is consistent: README refresh, CHANGELOG init, ADR-0001 (shared codebase decision), "3 suggestions in docs/SUGGESTIONS.md". Reviewers don't merge them because the content doesn't change between runs — Night Shift keeps regenerating the same suggestions.
Proposed fix: move docs/changelog/ADR/suggestions tasks to a weekly (not nightly) cadence, OR make them opt-in per repo, OR have them edit existing files instead of always proposing new ones. Currently they're pure noise on the nightly review surface.
Pattern 5 — Security task gating on already-fixed targets
Security task fires nightly. It correctly identifies open-redirect / unauthenticated-route patterns, but its sense of "what's already fixed" is wrong (see pattern 1). The "Risk: Medium" tag on each PR has been the only thing keeping reviewers from accidentally merging regressions of already-applied fixes. Without that tag, the cost of mishandling these PRs would be high.
Proposed fix: either (a) skip the security task entirely if pattern 1 (stale-base detection) lands and would catch these, or (b) move security to a less frequent cadence (weekly) with assign-for-review rather than "Risk: Medium" auto-PRs, until pattern 1 ships.
Suggested triage
High priority: pattern 1 (stale-base detection) — root cause for patterns 5 and roughly half of pattern 3.
Medium: pattern 2 (DO-NOT-CHANGE parser).
Medium: pattern 4 (cadence reduction for docs/suggestions).
Low (but cheap): pattern 3 (self-CI-check on test PRs).
Context
Three consecutive Night Shift runs on the
merkur-frontend+distriktssenteret-frontendsister sites have shown the same five patterns. Documented per-run in/Users/havardjohansen/Dev/night-shift/IMPROVEMENTS.mdentries2026-05-07,2026-05-08,2026-05-11(private workspace file — happy to copy excerpts here on request).Pattern 1 — Stale-base detection
Night Shift's per-task PR is correct relative to the branch checkout at run-start, but it doesn't see what the previous run already shipped. Result: DS produced word-for-word regenerations of fixes already on
developthree days running.Concrete examples from 2026-05-11:
developalready hasSANITY_PREVIEW_SECRETvalidation +isRelativeUrl()insrc/app/api/draft-mode/enable/route.ts. The PR would have regressed toSANITY_REVALIDATE_SECRET(wrong env var name).sizesprop and the iframeloading=\"lazy\"already on develop from the 2026-05-08 bundle.latin-extRubik subset.Proposed fix: before opening a PR for a task, diff against not just
developbut the most recent mergednight-shift/*bundle (or the last N bundles if cadence is unclear). If the proposed change is structurally already present, skip the task with a brief log line.Pattern 2 — DO-NOT-CHANGE policy blocks ignored
The 2026-05-11 DS #67 PR would have reverted
<html lang=\"no\">→lang=\"nb\"andog:locale: nn_NO→nb_NO. Both are explicitly forbidden by a block titled## Language policy — DO NOT CHANGEnear the top of the repo'sCLAUDE.md:This is the first time we've seen Night Shift actively regress a documented policy.
Proposed fix: when reading
CLAUDE.md/AGENTS.md, parse##(or deeper) headings whose text containsDO NOT CHANGE(or similar policy markers) and treat the contents as hard constraints. SEO task in particular should never modifylang/localeattributes if such a block exists.Pattern 3 — Test PRs open with red CI
DS #64 (2026-05-11) and DS #48 (2026-05-08) both opened with red Cypress E2E. The new specs referenced components/tokens that exist on merkur but not yet on DS (
[data-cy=\"ai-search-block\"],.u-head-sm,<h2.u-head-lg>).Proposed fix: in the test-add task, after opening the PR, poll
gh pr checksfor N minutes (orgh run watch). If E2E fails, automatically close the PR with a comment explaining the failure — don't leave it for a human to triage. Better still: run the new tests locally against the project's dev server before opening the PR.Pattern 4 — Docs/changelog/ADR/suggestions cadence
Three days, 12+ closed PRs across this category, zero merged. Pattern is consistent: README refresh, CHANGELOG init, ADR-0001 (shared codebase decision), "3 suggestions in
docs/SUGGESTIONS.md". Reviewers don't merge them because the content doesn't change between runs — Night Shift keeps regenerating the same suggestions.Proposed fix: move docs/changelog/ADR/suggestions tasks to a weekly (not nightly) cadence, OR make them opt-in per repo, OR have them edit existing files instead of always proposing new ones. Currently they're pure noise on the nightly review surface.
Pattern 5 — Security task gating on already-fixed targets
Security task fires nightly. It correctly identifies open-redirect / unauthenticated-route patterns, but its sense of "what's already fixed" is wrong (see pattern 1). The "Risk: Medium" tag on each PR has been the only thing keeping reviewers from accidentally merging regressions of already-applied fixes. Without that tag, the cost of mishandling these PRs would be high.
Proposed fix: either (a) skip the security task entirely if pattern 1 (stale-base detection) lands and would catch these, or (b) move security to a less frequent cadence (weekly) with
assign-for-reviewrather than "Risk: Medium" auto-PRs, until pattern 1 ships.Suggested triage
Pattern 5 dissolves once pattern 1 lands.
Cross-reference
Sister sites:
frontkom/merkur-frontend,frontkom/distriktssenteret-frontend. Today's bundled output:The cost of bundling is currently ~30–45 min of human review per repo per day. If patterns 1, 4, and 5 land, that should drop to ~10 min.
🤖 Drafted by Claude Code; awaiting Håvard's review before circulation.