Skip to content

PLAN.md

Latest commit

 

History

History
238 lines (198 loc) · 13.2 KB

PLAN.md

File metadata and controls

238 lines (198 loc) · 13.2 KB
Raw

data-differences-tool — Current Work Plan

The current arc of work. Updated when the arc changes, not every session. For session-by-session state, see HANDOFF.md.

Goal

Make the Data Differences Tool presentable enough to hand a CEO, CFO, or consultant a naked link with no explanation needed.

Why this arc, why now

v1.0 is functionally complete. The gap is first-impression credibility: no favicon, no link preview, no branding, and the /improve audit is overdue. This is a 30–60 minute pass, not a feature arc.

Tasks

1. Favicon + Apple touch icon

  • Add Lailara favicon (SVG 32×32 + ICO fallback)
  • Add to index.html <head>
  • Apple touch icon 180×180 PNG (needs conversion tool — deferred)

2. HTML meta tags

  • <title>: "Data Differences Tool — Lailara LLC"
  • <meta name="description">
  • OG tags (og:title, og:description, og:type=website, og:url)
  • Twitter card (summary + title + description)

3. Lailara wordmark in footer

  • "Built by Lailara LLC" text link in footer
  • Links to lailarallc.com
  • Subtle — warm-gray, underlined, centered

4. Visual QA pass

  • Full click-through on desktop: upload CSV pair, XLSX pair, mixed
  • Verify auto-detect copy is clear
  • Verify download buttons work (Excel + CSV)
  • Check mobile viewport (iPhone SE width minimum) — zones stack, no overflow at 375px
  • Confirm fonts load (Playfair Display headings, Source Sans 3 body)

5. /improve audit-only

  • Run /improve (full pass, 2026-07-27) + full code review + UI review
  • Fix P1 findings (CSV formula injection)
  • Fix P2 findings that were safe (perf, export errors, warnings, stale auto-detect, dead reordered data)
  • Blank-key matching semantics: deliberately NOT changed — intentional + tested. See open question in Improvement History.

6. Redeploy

  • npm run deploy
  • Verify live at diff.lailarallc.com
  • Spot-check OG preview (paste URL in Slack or https://metatags.io)

7. Un-pinned defect — test is waiting, fix is not written

Added 2026-07-28 by the FIX-LIST cross-repo test sweep. it.fails is vitest's strict xfail: the suite fails loudly the moment the fix lands and the marker has to come off. Do not remove a marker without doing the fix.

  • Identical verdict ignores column changes. Fixed 2026-07-28: the gate now consults columnChanges and leads with the column finding. Both un-pinned tests are live, plus three new cases. src/lib/summary-generator.ts:13 — the identical-verdict gate checks only added/removed/modified row counts, so a file whose rows all match but which gained a column reports "Files are identical. No differences found." Add result.columnChanges.length === 0 to the gate and lead with the column finding ("No row values changed. 1 column added: discount."). The same sentence is written to Excel at src/lib/export.ts:60 — fix both in one commit. Tests: tests/lib/summary-generator.test.ts — does not claim identical when a column was added but no row changed, leads with the column finding when rows match but columns changed.

8. Tier C review list — 2026-07-28

Seven of eight findings fixed this session, one commit each.

  • [Critical] Identical verdict ignores column changes (item 7 above).
  • [Medium] "Files are identical" overclaims — now "No material differences after tolerant matching." See DECISIONS.md.
  • [Medium] wasNormalized rendered nowhere — now tagged in the expanded row table and carried as a Normalized column in both exports.
  • [Medium] dayjs(str) formatted the fallback date parse in local time — now dayjs.utc(str).
  • [Medium] Test case for zero row changes with non-empty columnChanges (plus three more).
  • [Low] Column type inferred from at most 100 rows — now every row.
  • [Low] Footer used London-40; now London-35, the footer token.
  • [Low] README test count, CLAUDE.md "Primary language: TBD".
  • [Product gap] No figure is denominated in money and no column is flagged as financial. Summing the numeric delta on detected currency columns would close the largest CFO gap. NOT started — this is a feature, out of scope for the v1.1 polish arc. Candidate for the next arc.

9. Open defect — dates shift east of UTC in the Date-object branch

Known since 2026-05-22, recorded only as a doc caveat until now. Filing it as work: a documented-but-untracked defect has the same half-life as a prevention rule that was written and never widened — which is exactly what this project just spent a session learning.

  • src/lib/normalizer.ts:61 — SheetJS parses slash-format dates as local midnight. East of UTC, local midnight is the previous UTC day (midnight Jan 15 in Tokyo is 15:00 Jan 14 UTC), so the Date-object branch's getUTC* extraction returns the wrong calendar date there. Negative offsets are unaffected, which is why it survived: every test and every session has run in US Eastern. Both string branches are already UTC-safe (verified under TZ=Asia/Tokyo); this is the last branch that is not. The fix means not trusting SheetJS's timezone anchoring at all — likely reading the raw cell text rather than the parsed Date. Add a TZ=Asia/Tokyo case to the suite as part of it.

10. Found while verifying — date cells render as raw JS Date strings

A ship date of 2024-02-10 displays as "Fri Feb 09 2024 19:00:00 GMT-0500" — wrong day, in a finance-facing report. The differ stores the raw cell value and both the UI and the exports call String() on it.

  • src/components/RowChanges.tsx:95 (old/new value cells) and src/lib/export.ts:38 — format Date values as YYYY-MM-DD for display. The normalizer already has the UTC extraction logic; the display path never used it. Add a test for the export row content. Done 2026-07-28. A third surface the entry did not name: src/lib/summary-generator.ts:153 stringifies the same values to build the plain-English transition sentence, so the summary carried the bug too. All three now call formatCellValue in src/lib/display.ts; 9 tests added. Display is UTC-anchored and therefore inherits task 9's east-of-UTC exposure by design — when 9 lands, screen, exports and summary all move with it rather than disagreeing.

Definition of done for this arc

  • Link preview in Slack/iMessage shows title + description (not blank)
  • Favicon visible in browser tab
  • Footer credits Lailara
  • No visual regressions from v1.0
  • /improve audit current

Out of scope

  • New features (v1.1 is polish only)
  • SheetJS replacement
  • Analytics / tracking
  • Auth or persistence

Arc history

When an arc completes, archive its goal, completion date, and outcome here. Then start a new arc above. Provides continuity without bloating the active plan.

2026-05-22 — Ship a deployed tabular diff web app

  • Outcome: All 12 definition-of-done items complete. App live at https://diff.lailarallc.com/. First /improve pass completed (8 fixes). 70 tests, 297KB initial bundle.
  • Tag: v1.0

Improvement history

Track when this project was reviewed and improved via /improve. Each entry records what was found, what was fixed, and when to check again.

2026-07-31 — Improvement pass (improve + ce code review + UI review)

  • Trigger: User-initiated combined review. Goal: confirm the tool is CEO/CFO-ready, self-explanatory in 30s, and that recent Claude Code work is correct (user reported quality concerns lately).
  • What was reviewed: 5 ce personas over src/ (correctness, maintainability, testing, TypeScript, security) + ui-review-skill against the live site. Baseline healthy: clean tree, 100 tests, tsc + build clean, in sync with remote.
  • UI verdict: Passes the 30-second executive test — headline, tagline, concrete use cases, and the example-result card make the job obvious. Live site renders on-brand (warm canvas, Playfair, Lailara footer). Most ui-review FAILs were noise (checker measured <body>; credential/staleness hits were in minified dist/ bundles) — tightened review.yaml so future runs are clean.
  • What was fixed (11 commits):
    • [correctness] Date-typed KEY columns rendered as raw JS Date strings on screen + both exports (RowChanges ×2, export.ts formatKey) — the same defect 16537dd fixed for cell values but scoped to the repro, not the surface. Now routed through formatCellValue. +test.
    • [correctness] wasNormalized tagged genuine cased text changes as "formatting normalized" in case-insensitive mode. +test.
    • [correctness] Two all-blank columns scored as a rename (jaccard 0/0→1). +test.
    • [correctness] Numeric comma-strip collapsed "1,2,3"→123 (false match); now strips only valid thousands separators, shared helper. +tests.
    • [correctness] CSV lone-CR not quoted → row-boundary corruption. +tests.
    • [type-safety] Removed 3 as any casts in differ.ts (typed metadata as ColumnMetadata); modeled ColumnChange as a proper discriminated union.
    • [testing] exportToExcel had ZERO tests + two false-confidence CSV tests (asserted blob.size>0 only) — added real read-back assertions. 100→112.
    • [security] Bounded decompression-bomb cell counts + long-header Levenshtein cost (both self-DoS, tab-only). +tests. Final: 115 tests.
    • [UI] Darkened green/amber result tokens to WCAG AA (green 3.6→5.5:1, amber 2.3→4.7:1) using documented DS steps (HK-25, SG-35). Browser-verified.
  • Security posture: Strong. No Critical/High. CSV formula-injection neutralization verified complete. No XSS/innerHTML/eval.
  • Deferred / not touched: SheetJS xlsx@0.18.5 CVEs (accepted, client-side); east-of-UTC Date-object branch (PLAN task 9, still open); blank-key matching (intentional). NOT pushed/deployed — local commits only, awaiting user.
  • Next review: 2026-08-30 (30 days; stable after this pass).

2026-07-27 — Improvement pass (improve + code review + UI review)

  • Trigger: User-initiated combined review; /improve audit was overdue (due 2026-06-19)
  • What was reviewed: Full src/ code review (background agent), security, UI/first-impression clarity, tests, git hygiene, deps. Baseline healthy: clean tree, 75 tests, TSC clean, no secrets, design system correct.
  • What was fixed (9 items, 7 commits):
    • [P1] CSV formula injection — exported report cells beginning with =,+,-,@ could execute in Excel; now neutralized with a leading apostrophe (plain negatives untouched). +6 tests.
    • [P2] Rename-detection O(n²) — B column value-sets rebuilt per iteration; now precomputed once.
    • [P2] Silent export failures — added catch handlers + inline error message.
    • [P2] Missing key column in File B — now warns instead of silently collapsing all rows.
    • [P2] Duplicate-row warning count — now reports rows dropped (agrees with excludedRowCount).
    • [P2] Stale key auto-detect — ColumnPicker re-runs detection when files change.
    • [cleanup] Reordered column change — removed misleading dead indices.
    • [UI] Landing copy — now states audience + stakes for the 30-second test.
    • [UI] Example result card + mobile stacking (verified no overflow at 375px).
  • Deferred / not touched: SheetJS xlsx@0.18.5 CVEs (already an accepted, documented decision — client-side blast radius). Row-cap-after-parse (low value). Full real-file click-through QA (browser pane couldn't upload files this session).
  • OPEN QUESTION — blank-key matching: The code review flagged that two unrelated all-blank-key rows (one in each file) get matched and reported as "modified." This is currently INTENTIONAL and encoded in differ.test.ts ("treats null/empty key values as empty-string key") with excludedRowCount as the honesty mitigation. NOT changed unilaterally. Decide next session: keep as-is, or make blank composite keys non-matchable (would rewrite that test).
  • Next review: 2026-08-26 (30 days; project stable after this pass)

2026-05-22 — Improvement pass

  • Trigger: First-ever /improve run (project was overdue)
  • What was reviewed: Security (automated), code quality, dependencies, tests, docs, git hygiene
  • What was fixed:
    • Fixed Excel serial date offset bug (off-by-one, was 25570 should be 25569)
    • Added 50MB file size limit on uploads
    • Added 200K row count cap to prevent UI freezes
    • Added magic-byte validation for XLSX files
    • Code-split SheetJS and ExcelJS into lazy-loaded chunks (1.5MB → 297KB initial bundle)
    • Error boundary now shows generic message, logs internals to console
    • Updated README from stub to full documentation
    • Documented SheetJS vulnerability acceptance in DECISIONS.md
  • Deferred: SheetJS replacement (significant refactor, low practical risk for client-side tool)
  • Next review: 2026-06-19