diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index d6b03e0e..acbc5f76 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -228,8 +228,9 @@ jobs: # Two steps, one program: the base config is loose (strict/noImplicitAny off) # and covers all of src/; the ratchet re-runs it with noImplicitAny: true and # fails only on the paths listed in scripts/checkjs-ratchet.js (the data-layer - # row contracts + the repositories annotated against them). See that script's - # header for why this is not a second directory-scoped tsconfig. + # row contracts, every repository, and the service subdirectories annotated + # against them). See that script's header for why this is not a second + # directory-scoped tsconfig, and its RATCHETED list for what is still out. # ───────────────────────────────────────────────────────────────────────────── typecheck-backend: name: Type Check (Backend) diff --git a/TODO.md b/TODO.md index 3fdd1b5d..c6d2e1ec 100644 --- a/TODO.md +++ b/TODO.md @@ -796,11 +796,201 @@ look-changing one. - `getCategoryBreakdown` and `getCategoryPivot` (infoRepositoryStatistics.js) use `COALESCE(t.category_id, r.default_category_id)` with no primary-recipient fallback: a row recorded under an alias whose primary carries the default is categorised in the transactions list but UNCATEGORISED in the breakdown, and absent from the pivot (its WHERE requires the 2-level COALESCE non-NULL). Corollary: the mock suite's "missing category_id → Uncategorised" pivot case is unreachable through the real query. Pinned in `tests/infoRepoStatistics.db.test.js`. - Fix: extend both queries' COALESCE with the primary-recipient default (matching transactionRepository's 3-level pattern); flip the pinned test. -- [ ] **Remaining 2-level category-resolution surfaces: monthly summary (live + MV), recurring detection, sankey** 🔽 +- [x] **Remaining 2-level category-resolution surfaces: monthly summary (live + MV), recurring detection, sankey** 🔽 ✅ 2026-07-28 · 0defad0 (all four sites now canonical 3-level with the required `pr` join; migration 0085 drops mv_monthly_summary for rebuild-at-boot (0084 mechanism — totals output-invariant for the MV's only reader, so this aligns the stored grain; the user-visible fixes are sankey attribution + recurring detection's category label). Pinned by 5 live-DB tests (aliasCategoryResolution.db.test.js, 3 fail pre-fix) + SQL-shape guards in the three mock suites; adversarially verified on live PG16 incl. alias chains / default-less primary / own-category override, migration round-trip, and mvAvailable fallback. Verifier + implementer residues filed as the five findings below) - ↪ _from: Orchestration session 2026-07-28 · category-resolution fix pass (same pattern, outside that pass's scope)_ - `services/materializedViewService.js:71` (`mv_monthly_summary` definition) and the monthly live path `repositories/infoRepositoryMonthly.js:184`, `services/recurringDetectionService.js:157`, and `services/calculations/aggregation/sankey.js:51,73` all still use the 2-level `COALESCE(t.category_id, r.default_category_id)` — alias rows categorised via their primary's default are treated as uncategorised on these surfaces, now inconsistent with the fixed transactions/breakdown/pivot 3-level resolution. - Fix: extend each to the canonical 3-level pattern; the MV change needs the same drop-and-rebuild migration treatment as 0084 (mv_monthly_summary drop). +- [x] **Sankey exclusion clauses are NULL-unsafe and not alias-aware — excluding any category silently erases every uncategorised row (income renders as €0 with flows still leaving it)** 🔼 ✅ 2026-07-28 · c5f3d66 (routed through the shared buildExclusionClauses — sentinel + alias-aware forms, param offsetting handled; the €3000 row now survives exclusion (Income 3000, not 0) and a primary-recipient exclusion removes alias rows. Characterization test split into four real pins; fail-against-old reproduces the filed shapes exactly) + - ↪ _from: Orchestration session 2026-07-28 · category-resolution verifier (reproduced live on the real service)_ + - `apps/node-backend/src/services/calculations/aggregation/sankey.js:58` — category exclusion is `COALESCE(...) != ALL($n)` with no `-1` NULL sentinel: a NULL effective category fails the comparison and the row is dropped, so excluding one category removed a €3000 uncategorised income row entirely — the graph showed `Income 0` with €40 still flowing out of it. The canonical form (with a comment documenting this exact bug) is `filterBuilder.js:356`: `COALESCE(..., -1) NOT IN (...)`. Sibling defect at `sankey.js:63`: recipient exclusion is `t.recipient_id != ALL(...)` — neither NULL-safe nor alias-aware (canonical: `COALESCE(r.primary_recipient_id, t.recipient_id, -1) NOT IN`), so excluding a primary recipient leaves its alias rows in the graph while the category exclusion beside it is now alias-aware. + - Fix: move both clauses to the canonical sentinel forms (or route through `buildExclusionClauses`); pin with a live-DB test that has an uncategorised row + an active exclusion (the existing aliasCategoryResolution.db.test.js sankey exclusion assertion is characterization only and passes either way — its comment says so). + +- [x] **Sankey has no `is_transfer` filter — internal transfers counted as income/spending in the flow graph** 🔼 ✅ 2026-07-28 · c5f3d66 (ADR-083 confirmed: excluded by default in ALL aggregations under the single runtime includeTransfers setting, explicitly not a separate node — sankey now applies the sibling conditional predicate; transfer pair off → Income 3000, on → 3900 with the legs visible, both pinned live) + - ↪ _from: Orchestration session 2026-07-28 · category-resolution fix pass (noticed, out of scope)_ + - `apps/node-backend/src/services/calculations/aggregation/sankey.js:74` (the aggregation query's WHERE) filters `is_active` and date only — unlike every other money aggregation (ADR-083; e.g. mv_monthly_summary's `is_transfer = false`). A savings transfer inflates both the income and spending sides of the graph. + - Fix: add `AND t.is_transfer = false` (confirm against ADR-083's exclusion semantics before assuming — sankey may arguably want transfers as a visible flow, in which case they should be a distinct node, not silent income/spending). + +- [x] **`transactionRepository` resolves effective category id and displayed name with OPPOSITE precedence — id says one category, name shows another** 🔼 ✅ 2026-07-28 · 53af9bc (rc-before-pc in all four CASE copies — CATEGORY_NAME_SQL, the category sort column, and the getAll/getAllWithCount inlines; id and name verified agreeing across getById/getAll/getAllWithCount/create/update/update+tags and the category sort on the alias-with-own-default topology. The same wrong or weaker CASE on export/splits/planned surfaces is filed below) + - ↪ _from: Orchestration session 2026-07-28 · category-resolution verifier (reproduced live: id=Food:Groceries, name="Bills:Utilities" on the same row)_ + - `apps/node-backend/src/repositories/transactionRepository.js:66` (`EFFECTIVE_CATEGORY_ID_SQL`: own → `rc` recipient default → `pc` primary default) vs `:67-72` (`CATEGORY_NAME_SQL`: `c` → **`pc` → `rc`**). For an alias recipient that has its own default AND a primary with a different default, the transactions list reports the recipient-default id but displays the primary-default name. The newly-fixed aggregation surfaces follow the id (matching `EFFECTIVE_CATEGORY_ID_SQL`), so they disagree with the reference surface's displayed name in this topology. + - Fix: make `CATEGORY_NAME_SQL` join through the same COALESCE the id uses (`rc` before `pc`); pin with a DB test on the alias-with-own-default-and-differing-primary-default topology. + +- [ ] **Recurring detection emits raw `categoryId` beside the resolved `categoryName`, and groups/labels patterns by alias rather than primary recipient** 🔽 🔎 partial-5084de5 2026-07-28 (categoryId half DONE: the pattern now emits the same 3-level effective id the name resolves from, pinned live + mock; creating a planned payment from a pattern now carries the displayed category instead of silently uncategorised. LEFT: the alias grouping/labeling half — whether patterns should group by primary recipient and label with the canonical name is an explicit product decision, deliberately not driven from the category fix) + - ↪ _from: Orchestration session 2026-07-28 · category-resolution fix pass (noticed, out of scope; the categoryId half pre-existing and widened by the 3-level fix)_ + - `apps/node-backend/src/services/recurringDetectionService.js:280` — emitted `categoryId` is raw `t.category_id` while `categoryName` is the resolved effective category, so any recipient-default-categorised pattern reports `categoryId: null` with a non-null `categoryName` (consumers keying on the id see uncategorised). And `:152` — `recipient_name` is `r.name` with grouping on raw `recipient_id`, not `COALESCE(pr.name, r.name)`/primary id, so an alias and its primary yield two separate patterns which now carry the same category label. The recipient half changes grouping semantics — decide deliberately, don't drive it from the category fix. + - Fix: emit the same effective category id the name is resolved from; separately decide whether patterns should group by primary recipient (alias-merge semantics) and label with the canonical name. + +- [ ] **Stale comment: `routes/recipients.js:20-21` still documents the MV category attribution as 2-level** ⏬ + - ↪ _from: Orchestration session 2026-07-28 · category-resolution fix pass (noticed in passing)_ + - The comment describes `COALESCE(t.category_id, r.default_category_id)`; as of the 3-level fix (0defad0) every category-bearing surface resolves three levels. + - Fix: update the comment to the 3-level expression. + +- [x] **Bank list DATE columns leak raw pg Date objects — first/last_transaction shift a day east of UTC** 🔼 ✅ 2026-07-28 · 9c267bf (both routed through toWireDate; verifier reproduced the TZ=Asia/Tokyo day-shift on HEAD and its absence after; frontend type already declared string so the contract now holds at runtime) + - ↪ _from: Orchestration session 2026-07-28 · real-DB harness third increment (pinned in tests/infoRepoBanks.db.test.js "PIN: first_transaction/last_transaction come back as raw pg Date objects")_ + - `apps/node-backend/src/repositories/infoRepositoryBanks.js:182-183` passes both DATE columns through raw while `anchor_date` on the same object is a wire string; `lib/dateFormat.js:38-44` states the project rule (raw pg DATE JSON-serializes to the previous day east of UTC — the systemic date-shift class). + - Fix: route both through `toWireDate`; flip the pin. + +- [x] **`getBankBalances` sums a multi-currency account's amounts across currencies, then converts the total at one rate** 🔼 ✅ 2026-07-28 · 9c267bf (new per-currency partition lateral: a stamped balance anchors only its own currency's partition, each partition converts at its own rate, summed after conversion — 100 EUR + 100 USD @0.5 → 150; anchored+mixed case 1025 pinned; single-currency byte-identical; COMPUTED_BALANCE_LATERAL untouched so the hub/reconcile/cross-workspace consumers are unaffected. Adversarially verified incl. 36 random multi-currency ledgers vs independent brute force. The same defect on the four OTHER surfaces is filed below) + - ↪ _from: Orchestration session 2026-07-28 · real-DB harness third increment (pinned in tests/infoRepoBanks.db.test.js "PIN: a multi-currency account adds EUR and USD amounts")_ + - `apps/node-backend/src/repositories/accountBalanceSql.js` — `SUM(t2.amount)` has no currency partitioning; `infoRepositoryBanks.js:69` picks the conversion currency from the single most-recent row. 100 EUR + 100 USD @0.5 → balance 100, correct is 150. The repository's own comment defers multi-currency partitioning as "D2" — this makes the deferral a wrong headline number, not a missing feature. + - Fix: partition the balance computation per currency and convert each partition; flip the pin. + +- [x] **Manual-only accounts reach `total_net_position` but never appear in balance history — headline disagrees with its own chart** 🔼 ✅ 2026-07-28 · 9c267bf (history gate dropped via a set-based span-expansion series (per-currency, matching the headline); headline == today's chart point pinned on mixed and all-manual ledgers INCLUDING moving FX curves — the verifier caught the headline still keying FX at last-activity (32% divergence on foreign-currency accounts) and an O(days²) planner degradation (27× slower); both fixed pre-commit, end-to-end ~2× HEAD while emitting the previously-dropped accounts' points) + - ↪ _from: Orchestration session 2026-07-28 · real-DB harness third increment (pinned in tests/infoRepoBanks.db.test.js "PIN: manual-only accounts reach total_net_position")_ + - The current-balance query dropped its `balance IS NOT NULL` gate (WP-A1) so manual-only accounts count in the headline, but the history query at `infoRepositoryBanks.js:129` still filters `WHERE lb.balance IS NOT NULL`: a mixed ledger renders total_net_position ≠ today's total_history point; an all-manual ledger renders a non-zero headline above an empty chart. + - Fix: apply the same WP-A1 gate removal to the history query. + +- [x] **Forecast module never excludes transfers — all ten queries violate ADR-083 while sibling surfaces honor it** 🔼 ✅ 2026-07-28 · 948980d (all 8 transactions-hitting queries got the sibling-identical conditional predicate; the 5 planned_transactions overlays deliberately did not — no is_transfer column exists there, pinned against a future 42703. includeTransfers added to the MC forecast cache key so toggling misses the 6h cache. Adversarially verified: trade-leg −5050→−50 live before/after; runtime-toggle proven; no balance-anchored query needed a carve-out) + - ↪ _from: Orchestration session 2026-07-28 · real-DB harness third increment (pinned in tests/infoRepoForecast.db.test.js "PIN: transfers inflate the cashflow comparison")_ + - Every query in `apps/node-backend/src/repositories/infoRepositoryForecast.js` (93-116, 261-298, 362-390, 462-479) filters `is_active` only; the module never calls `getIncludeTransfers()`. Siblings `infoRepositoryAverageVsCurrent.js:22-23` and `infoRepositoryMonthly.js:38,194` do. Same fixture on one dashboard: cashflow comparison −1000 vs avg-vs-current 100. + - Fix: thread the ADR-083 transfer-exclusion setting through the forecast queries; flip the pin. + +- [x] **Forecast 24-month average divides by populated months only, not the window** 🔼 ✅ 2026-07-28 · 948980d (divisor = months from the ledger's first in-window transaction through the last complete month, from an UNFILTERED MIN(date) ledger probe: empty months inside history are real zeros (240→10), pre-ledger months are not charged (3-month install divides by 3, not 24). Verifier-found defects fixed before commit: a stale never-executed planned row can no longer deflate the divisor 24×, and category/recipient/transfer filters move the numerator only — both pinned in both orientations) + - ↪ _from: Orchestration session 2026-07-28 · real-DB harness third increment (pinned in tests/infoRepoForecast.db.test.js "PIN: the 24-month average divides by populated months only")_ + - `infoRepositoryForecast.js:41-42` divides by `monthKeys.length`: one populated month of 240 in a 24-month window reports a monthly average of 240 (correct: 10). Inflates every forecast fed by it. + - Fix: divide by the window length (months elapsed), treating empty months as zero; flip the pin. + +- [x] **Top merchants convert at the LATEST FX rate while by-year and pivot use the historical per-date rate — same recipient, three different EUR figures** 🔼 ✅ 2026-07-28 · 53af9bc (aggregates per (recipient,date,currency), converts per date, reduces per recipient — the by-year/pivot shape; all three surfaces agree on a hostile multi-currency/same-day/alias fixture; counts/averages/seen-bounds re-reduce correctly. MoM on the same page still converts at latest — filed below) + - ↪ _from: Orchestration session 2026-07-28 · real-DB harness third increment (pinned in tests/infoRepositoryRecipients.db.test.js "PIN: top merchants convert at the LATEST rate")_ + - `apps/node-backend/src/repositories/infoRepositoryRecipients.js:64-67` passes no options to `convertRowsToEur`; `:217-221` and `:329-333` pass `useHistoricalRatesByDate`. One USD purchase → 90 in Top merchants, 25 in by-year and pivot. + - Fix: pass `useHistoricalRatesByDate` on the top-merchants path too; flip the pin. + +- [ ] **Recipient pivot: picking an alias returns a series labeled with the PRIMARY's name but containing only the alias's spend** 🔽 + - ↪ _from: Orchestration session 2026-07-28 · real-DB harness third increment (pinned in tests/infoRepositoryRecipients.db.test.js "PIN: picking an alias in the pivot")_ + - `infoRepositoryRecipients.js:293-296` expands member resolution downward only, then `GROUP BY COALESCE(pr.id, r.id)` (`:309-310`, `:324`) relabels upward: picking "Delhaize Wilrijk" returns a series named "Delhaize" worth 25 of the primary's 125 — the label promises the family, the number delivers one member. + - Fix: either expand the selection to the full alias family (label honest) or keep the alias's own label (number honest) — decide which semantics the pivot wants. + +- [x] **`avg_monthly_spending` divides by populated months (income-only months count), disagreeing with its own `avg_daily_spending` sibling** 🔼 ✅ 2026-07-28 · 53af9bc (both fields now share one denominator — the forecast's elapsed-months counting rule on this card's 6-month window via the same unfiltered ledger probe; income-only months no longer halve the average, a single busy month is 1/1 not full-weight-as-6mo, young installs undeflated. avg_daily deliberately moved off the fixed 183-day calendar — verifier enumerated every consumer, none assumed the old window. The old pin was non-discriminating and was replaced by three tests that fail pre-fix; edge clamps (current-month-only, future-only, inactive-oldest) probed live) + - ↪ _from: Orchestration session 2026-07-28 · real-DB harness third increment (pinned in tests/infoRepository.db.test.js "PIN: avg_monthly_spending divides by populated months")_ + - `apps/node-backend/src/repositories/infoRepositoryAverageVsCurrent.js:66-75` seeds the month key before the `eur < 0` test: one 240-spend month → average 240; adding an income-only month halves it to 120 with zero extra spend. `avg_daily_spending` on the same object uses the true calendar denominator, so the two fields disagree by construction. + - Fix: use the calendar window as the denominator (matching the daily sibling); flip the pin. + +- [x] **Net-worth history: manual-only accounts appear only in the LAST point — the chart steps up overnight and reports it as a monthly gain** 🔼 ✅ 2026-07-28 · 9c267bf (same gate dropped with the same series builder, cross-currency to match ITS headline; phantom step gone (580,560,560,560,560 with monthlyChange −20); invariant pinned; 20 random net-worth ledgers cross-checked vs independent brute force) + - ↪ _from: Orchestration session 2026-07-28 · real-DB harness third increment (pinned in tests/infoRepository.db.test.js "PIN: a manual-only account appears only in the LAST net-worth point")_ + - `apps/node-backend/src/repositories/infoRepositoryNetWorth.js:137` gates the history walk on `lb.balance IS NOT NULL` while the current point comes from the unstamped-tolerant lateral (`:147-162`, applied `:303-314`); the flow fallback only rescues "nothing stamped anywhere". Chart: 1000 → 1200 with no transaction, surfaced as a real gain. Same gate class as the banks history finding above — fix them consistently. + - Fix: apply the unstamped-tolerant balance resolution to the history walk too; flip the pin. + +- [x] **`infoRepositoryPlanned` builds next-month from LOCAL date parts but reads them back with UTC getters — names the wrong month east of UTC (the app's own default TZ)** 🔼 ✅ 2026-07-28 · e40e96f (window rebuilt on the ADR-009 string helpers, month/year parsed from the window start — host-TZ independent, verifier-confirmed at UTC+14 and brute-forced over a decade of boundaries; pinned under forced TZ=Asia/Tokyo incl. the Dec→Jan boundary. The SAME file's expandRecurringOccurrences has its own pre-existing TZ bug — filed below) + - ↪ _from: Orchestration session 2026-07-28 · real-DB harness third increment (noticed; NOT pinned as a failure — container and CI run TZ=UTC where it cannot reproduce; the suite asserts the month/period_start invariant instead)_ + - `apps/node-backend/src/repositories/infoRepositoryPlanned.js:98-99` + `:214-215` — `new Date(today.year, today.month, 1)` constructs with LOCAL getters but `month`/`year` are read back via `getUTCMonth()`/`getUTCFullYear()`. Under a server TZ east of UTC — including the default `APP_TIMEZONE=Europe/Brussels` — local midnight of the 1st is the previous month in UTC: the response names the wrong month while `period_start` names the right one. + - Fix: route through the ADR-009 APP_TIMEZONE date helpers instead of mixed local/UTC `Date` getters. + +- [ ] **Minor infoRepo residue from the harness migration: stale barrel doc, `infoRepositoryTags` not assembled, float `+=` on the monthly live path** ⏬ + - ↪ _from: Orchestration session 2026-07-28 · real-DB harness third increment (noticed in passing)_ + - (a) `infoRepository.js:9-10` advertises `getStatistics` and `getTransactionSummary`; neither exists (the barrel's exact key-set is now pinned). (b) `infoRepositoryTags.js` exists but is not assembled into the barrel — confirm whether intentional. (c) `infoRepositoryMonthly.js:277-279` accumulates money with raw `+=` while the MV path uses Decimal — no observable drift at fixture scale, but inconsistent. + - Fix: correct the doc comment; decide (b); align (c) on Decimal. + +- [x] **`batch_id` is a string on `POST /api/import/csv` but a number on the review-commit route — same JSON field, two wire types (both import pipelines)** 🔼 ✅ 2026-07-28 · 6bc3ace (NUMBER chosen — matches coercedIdSchema, frontend guards, and MSW; documented with the >2^53 escape hatch. Number(id) at both stage boundaries, typedefs tightened, openapi.yaml string→integer, frontend generated types + imports.ts followed. Reader audit found a LIVE bug this fixes: the SSE review_required event relayed the string that z.number() rejects — imports needing review failed the progress stream. Cross-route strict-equality pins through the real createBatch; fail-against-old proven with 8 failures. MSW response-shape divergence filed below) + - ↪ _from: Orchestration session 2026-07-28 · checkJs ratchet services pass (noticed while typing importPipeline; typed as current reality `string|number`)_ + - `apps/node-backend/src/services/importPipeline/stage.js:29` returns `result.rows[0].id` — pg emits BIGSERIAL as a **string** — while route-driven calls pass a `Number()` (`lib/importBatchIds.js:18` `coercedIdSchema`). Wire consequence: `routes/importRoutes.js:48` responds `batch_id: ""`, `routes/importRoutes.js:570` responds `batch_id: `. Identical split in the portfolio pipeline (`portfolioImportPipeline/stage.js:33` vs its index). Any client doing strict-equality on batch ids across the two responses breaks. + - Fix: normalize at the boundary — pick one wire type (number, matching the coerced input schema, unless ids can exceed 2^53) and convert at stage/response; then tighten the `ImportBatchId` typedef from `string|number` to the chosen type. + +- [ ] **`buildHistoricalRateIndex` is fed two different `rate_date` shapes — the raw-DATE path is one refactor from the documented day-shift bug** 🔽 + - ↪ _from: Orchestration session 2026-07-28 · checkJs ratchet services pass (noticed while typing currency/)_ + - `apps/node-backend/src/services/currency/currencyConversionService.js:171` selects raw DATE (pg → local-midnight `Date`) while `services/portfolio/portfolioSummaryService.js:178` projects `to_char(rate_date,'YYYY-MM-DD')` (string). Both work only because `normalizeDateInput` special-cases `Date`; removing or bypassing that special case on the raw path reproduces the systemic UTC-extraction day-shift its own comment block documents. + - Fix: project `to_char(...)` on the currencyConversionService path too, so the index is built from wire strings on both feeds. + +- [ ] **`getSnapshots` emits three money fields as `string|number` while every sibling is a NUMERIC string** ⏬ + - ↪ _from: Orchestration session 2026-07-28 · checkJs ratchet services pass (noticed while typing portfolioPerformanceSnapshotService)_ + - `apps/node-backend/src/services/portfolioPerformanceSnapshotService.js:41-43` applies `?? 0` to `stocks_etfs_invested`/`crypto_invested`/`metals_invested`, making them `number` when NULL and `string` otherwise (typed honestly as `string|number` in `PerformanceSnapshot`). Harmless today (`toDecimal` accepts both) but an inconsistency trap. + - Fix: default with `?? '0'` to keep the NUMERIC-string contract uniform; tighten the typedef. + +- [x] **`createErrorHandler` honours `err.status` only on AppError instances — malformed JSON and oversized payloads surface as 500, the latter with its reason hidden in production** 🔼 ✅ 2026-07-28 · 6bc3ace (two-part rule documented in the handler: integer 4xx status/statusCode on non-AppErrors is forwarded, but the message is echoed only for body-parser allowlist types — other forwarded 4xx get a generic per-status phrase, so a status-bearing internal error can't leak wording. Malformed JSON → 400, >1MB → 413 with its reason surviving production; loan-schedule errors 500→400. All non-AppError status carriers in src/ audited (Ollama upstream statuses stay wrapped/generic). Both pins flipped + 14 new tests incl. production mode; fail-against-old proven with 12 failures) + - ↪ _from: Orchestration session 2026-07-28 · supertest harness migration (pinned in tests/routes/transactions.test.js: "PIN: a malformed JSON body yields a 500..." and "PIN: an over-limit body yields a 500...")_ + - `apps/node-backend/src/middleware/errorHandler.js:117-119` — body-parser's HttpErrors carry a correct `.status` (SyntaxError 400 for truncated JSON, PayloadTooLargeError 413 for a >1MB body) but are not AppError instances, so both are reported as 500 INTERNAL_SERVER_ERROR. Because the 413 is thereby 5xx, the production branch (`errorHandler.js:139-141`) replaces "request entity too large" with the generic message — a client posting an oversized bulk/import payload cannot tell why it failed, and client typos count against server-error monitoring. + - Fix: honour a numeric `err.status`/`err.statusCode` in the 4xx range for non-AppError errors (mapping to a sensible code, e.g. BAD_REQUEST/PAYLOAD_TOO_LARGE) before falling through to 500; flip both pins. + +- [ ] **Forecast month/day boundaries mix APP_TIMEZONE with Postgres `CURRENT_DATE` — off-by-one month for ~2h around month rollover** 🔽 + - ↪ _from: Orchestration session 2026-07-28 · forecast fix pass (noticed by implementer, drift confirmed live by verifier; the new divisor clamps only the extremes)_ + - `apps/node-backend/src/repositories/infoRepositoryForecast.js:120-129` — `daysInMonth`/`currentDay`/`lastCompleteMonthIdx` derive from `todayAppDateString()` (APP_TIMEZONE, default Europe/Brussels) while every window predicate derives from Postgres `CURRENT_DATE` (session UTC). **Broadened 2026-07-28:** `infoRepositoryAverageVsCurrent.js:164-175` now has the identical split (its new divisor compares a probe-derived `CURRENT_DATE` month against an app-TZ last-complete-month; acknowledged in its own comment) — fix both when this is taken. Between ~22:00 UTC and midnight on a month's last day the app clock is already next-month: window edges and month arithmetic disagree by one, inflating the average divisor by one month mid-range (clamp catches only 0/over-window extremes). Bounded (~2h/month) but systematic. + - Fix: derive both sides from one clock — either pass the app-timezone "today" into the SQL as a parameter, or compute the JS-side month math from the DB's `CURRENT_DATE` returned by the query. + +- [ ] **`infoRepositoryForecast` string-interpolates range-validated integers into SQL instead of binding them** ⏬ + - ↪ _from: Orchestration session 2026-07-28 · forecast fix pass (noticed; defense-in-depth, no live injection vector)_ + - `apps/node-backend/src/repositories/infoRepositoryForecast.js:194,205,300,314,466,477` (+ the new ledger-start probe, which follows the module convention for the `HISTORY_MONTHS` constant) — `historyMonths`/`daysBack`/`daysForward` are `Number.isInteger`+range-validated then template-interpolated. The validation is the only thing between a future caller and injection; the module is the last one with un-parameterised interpolation. + - Fix: bind them as parameters (`interval '1 month' * $n` or `make_interval`), or leave the validation as the documented invariant if binding fights the interval syntax — decide once, comment it. + +- [ ] **Cross-currency `SUM(t2.amount)` still converts at one rate on four other surfaces — the fixed getBankBalances defect lives on in the hub, reconcile, cross-workspace, and net-worth current point** 🔼 + - ↪ _from: Orchestration session 2026-07-28 · balances fix pass (out of that finding's scope, which named getBankBalances; all four call sites read by the verifier)_ + - `apps/node-backend/src/repositories/accountRepository.js:67` (hub `computed_balance` + drift), `services/reconcileService.js:85` (drift vs statement), `services/crossWorkspaceDataService.js:61` (converts the raw cross-currency total at `a.currency`), `repositories/infoRepositoryNetWorth.js:159` (current point) — all still use `COMPUTED_BALANCE_LATERAL`'s single cross-currency sum. For a multi-currency account each reports the 100+100@0.5→100 wrong number. The per-currency builder (`computedBalanceByCurrencyLateral`) now exists — adopting it per surface needs each surface's conversion point audited (the hub emits native currency; reconcile compares against a single-currency statement figure — decide what a multi-currency drift even means there). + - Fix: migrate each surface deliberately onto the per-currency builder (or document why a surface is single-currency by contract); keep hub/drift parity in mind. + +- [ ] **Account drift badge mixes conventions on multi-currency accounts: native-currency drift beside a per-currency converted balance** 🔽 + - ↪ _from: Orchestration session 2026-07-28 · balances fix pass (noticed; verifier reproduced balance 150 / drift −50 where statement−balance=0)_ + - `apps/node-backend/src/repositories/infoRepositoryBanks.js:71-73` — `drift` deliberately still reads the cross-currency `lb.balance` for hub-badge parity, while `balance` on the same row is per-currency. Inconsistent at HEAD too (different numbers, same contradiction), but the fix makes the two figures on one badge visibly disagree in a new way. Depends on the hub finding above — resolve together. + - Fix: once the hub goes per-currency, re-derive drift per-currency (statement figure vs its own currency's partition) and the contradiction dissolves. + +- [ ] **Future-dated rows: banks headline counts them now, the chart only from their date — and an all-future account is 123 in banks but 0 in net worth** ⏬ + - ↪ _from: Orchestration session 2026-07-28 · balances fix pass (pre-existing, pinned as an explicit known-divergence test in tests/infoRepoBanks.db.test.js rather than silently changed — bounding the headline would break hub agreement)_ + - The headline is unbounded (hub parity) while the chart stops at today. Corollary the verifier added: an account whose rows are ALL future-dated yields banks headline 123 with net worth reporting `snapshots=0, current.liquid=0` (empty grid skips the current-point override). + - Fix: decide the product stance on future-dated rows (exclude from headline until effective? include in both?) and align banks + net worth + hub on it. + +- [x] **Net-worth transaction-flow fallback sums tracking-only transactions — a ledger whose only active accounts are `in_net_worth=false` reports THEIR running total as net worth instead of 0** 🔼 ✅ 2026-07-28 · e40e96f (NOT EXISTS predicate on both fallback CTEs excludes rows positively attributed to in_net_worth=false accounts; unattributed/un-migrated rows stay counted (pinned via UPDATE-relabel since the sync trigger creates accounts on INSERT). Verifier: predicate matches the walk's resolution exactly, FK/NOT-NULL edge cases unrepresentable, and the multi-currency variant (tracking-only USD polluting the converted sum) is covered. Residues filed below: firstDataDate span, no is_liability split on the fallback) + - ↪ _from: Orchestration session 2026-07-28 · balances fix pass (pre-existing block, reproduced by the verifier: liquid −133.25 where 0 is correct; comment at the site now states the real trigger honestly)_ + - `apps/node-backend/src/repositories/infoRepositoryNetWorth.js:193-201` — the fallback fires when the balance walk returns no rows, which happens not only for un-migrated ledgers but whenever every account with activity is excluded from net worth; it then sums ALL transactions with no account/`in_net_worth` predicate. + - Fix: constrain the fallback's sum to in-net-worth accounts (join through the account resolution the walk uses); an all-tracking ledger should report 0. + +- [ ] **`tests/infoRepository.test.js` mock dispatcher's bare `SELECT 1 FROM` guard is one branch-reorder away from silently swallowing production queries** ⏬ + - ↪ _from: Orchestration session 2026-07-28 · balances fix pass (noticed; the balance work deliberately kept its new SQL clear of the literal instead of widening the blast radius)_ + - `tests/infoRepository.test.js:57,68,111,150,172,189,212,237,267,300,420` — `sql.includes('SELECT 1 FROM')` is meant to catch only `mvAvailable`'s probe but also matches `NOT EXISTS (SELECT 1 FROM anchor)` inside `COMPUTED_BALANCE_LATERAL`; only branch ordering saves it today (acknowledged in the comment at `:293-296`). + - Fix: tighten the guard to `SELECT 1 FROM mv_` at all 11 sites. + +- [x] **Recipient month-over-month still converts at the LATEST rate — now inconsistent with the fixed top-merchants/by-year/pivot on the same page** 🔽 ✅ 2026-07-28 · e40e96f (per-(recipient,period,date,currency) aggregation + historical per-date conversion, the top-merchants pattern; a stressed 2.9× cross-surface disagreement closes to one rounding cent; Decimal accumulator guards the multiplied row count; no-DB contract guard added on the convertRowsToEur options) + - ↪ _from: Orchestration session 2026-07-28 · recipients-FX fix pass (same class, one function away; out of that finding's top-merchants scope)_ + - `apps/node-backend/src/repositories/infoRepositoryRecipients.js:152-155` — `momConverted` passes no options to `convertRowsToEur` while grouping by `(recipient, period, currency)`: a rate move between the two compared months is invisible, and MoM's EUR figures won't match the now-historical sibling surfaces. + - Fix: same treatment — add `t.date` (or the period's representative dates) to the grouping and pass `useHistoricalRatesByDate`. + +- [x] **Remaining category-NAME resolution defects: export + splits use pc-before-rc, planned surfaces resolve 2-level or 1-level** 🔼 ✅ 2026-07-28 · 5dba368 (all five sites on the canonical order: export/bulk-export + owed-splits CASE swapped; plannedTransactionRepository 2→3 levels via a shared constant across its six query sites; infoRepositoryPlanned 1→3. Adjacent correction surfaced by the fix's own tests: planned search now matches the resolved display label (the old OR'd-columns form both missed pc-categorised rows and over-matched via the primary). 5 pins fail pre-fix on the alias-with-own-default topology, asserting name==resolved-id against transactionRepository. Residue filed below: planned surfaces show the alias's recipient_name where transactions show the primary's) + - ↪ _from: Orchestration session 2026-07-28 · category-precedence fix pass (implementer noticed 4 sites; verifier reproduced all live, found a 5th, and confirmed the enumeration is otherwise exhaustive)_ + - (a) `apps/node-backend/src/services/transactionExport.js:75-80` and `repositories/splitRepository.js:374-379` carry the pre-fix `pc`-before-`rc` CASE — the CSV/NDJSON export and the owed-splits export show "Bills:Utilities" for the row the transactions list now labels "Zzz:Last" (reproduced live). (b) `repositories/plannedTransactionRepository.js:41-45` and `:617-621` resolve `c → rc` only (no `pc` branch, no `pr` join). (c) `repositories/infoRepositoryPlanned.js:104-107` resolves `c` ONLY (joins neither `rc` nor `pc`) — `getPlannedExpensesNextMonth` reports `category_name: null` for rows its sibling plannedTransactionRepository categorises. Every other name-rendering site verified structurally consistent. + - Fix: (a) swap to the canonical rc-before-pc order; (b)+(c) extend to the full 3-level pattern with the `pr` join; pin each on the alias-with-own-default topology. + +- [ ] **Denominator-fix residues: `countObservedMonths` duplicated verbatim across two money files; mock probe guard doesn't pin its load-bearing predicates** ⏬ + - ↪ _from: Orchestration session 2026-07-28 · denominator fix verification (non-blocking notes)_ + - (a) `countObservedMonths` + `monthKeyFromDbDate` now live verbatim in both `infoRepositoryForecast.js:76-99` and `infoRepositoryAverageVsCurrent.js:26-73` — two copies of a money denominator is the drift risk the precedence finding is about, one layer up; a shared helper in `infoRepositoryHelpers.js` (parameterised on windowMonths) closes it. (b) `tests/infoRepository.test.js:855-862` asserts the ledger probe lacks `is_transfer`/`LIMIT` but not that it KEEPS `is_active = true` and the window floor — both load-bearing; only the DB suite catches their removal. + - Fix: extract the shared helper; strengthen the mock guard to assert the two retained predicates. + +- [ ] **Planned surfaces show the ALIAS's `recipient_name` where transactions show the PRIMARY's — same recipient, two labels across the dashboard** ⏬ + - ↪ _from: Orchestration session 2026-07-28 · category-name fix pass (noticed; consistent across both planned repos, so possibly deliberate — decide, don't assume)_ + - `apps/node-backend/src/repositories/plannedTransactionRepository.js` and `infoRepositoryPlanned.js` project `r.name` while transactions use `COALESCE(pr.name, r.name)`: a planned row under an alias shows the alias name, the equivalent transaction shows the primary's. + - Fix: decide the intended display (probably the transactions convention) and align both planned repos; pin on the alias topology. + +- [ ] **MSW mock for `POST /api/import/csv` returns a response shape the backend never emits — and neither import route has a contract test, which is how the batch_id wire split survived** ⏬ + - ↪ _from: Orchestration session 2026-07-28 · batch_id fix pass (noticed; root cause of the missed detection)_ + - `apps/frontend/src/test/msw/handlers.ts:488,491` return `{ batch_id, rows, status: "queued" }` where the real route emits `total/imported/duplicates/errors/batch_id/auto_linked_count/status/error_message/links` (`routes/importRoutes.js:35-63`); no `contracts.test.ts` entry covers either csv route. + - Fix: align the MSW shapes with the real responses and add contract entries for both import routes. + +- [ ] **`loanSchedule` throws plain `Error` with `statusCode` instead of `ValidationError` — its useful message is now genericized to "Bad Request"** ⏬ + - ↪ _from: Orchestration session 2026-07-28 · errorHandler fix pass (noticed; the fix upgraded these 500→400 but the safe "Invalid loan configuration: …" text is replaced by the generic phrase under the message allowlist)_ + - `apps/node-backend/src/services/calculations/loanSchedule.js:70,102`, surfaced via `routes/plannedTransactions.js:247`. + - Fix: throw `ValidationError` (authored message, full fidelity through the handler's AppError path). + +- [ ] **Sankey display residues: a real category named "Uncategorised" merges with the NULL bucket; an overspent year renders no deficit signal** ⏬ + - ↪ _from: Orchestration session 2026-07-28 · sankey fix pass (noticed, out of scope; the label collision also exists in the pivot — pre-existing and consistent)_ + - (a) `sankey.js` `COALESCE(c.general || ': ' || c.detail, 'Uncategorised')` merges a genuinely-named "Uncategorised" category with the NULL bucket (same shape in the pivot — fix together or accept). (b) `savings = Math.max(0, income − spending)`: an overspending year just omits the savings node instead of showing a deficit — display gap, possibly deliberate. + - Fix: (a) distinguish the NULL bucket with a sentinel id rather than a display-string collision; (b) decide whether the flow graph should render a deficit node. + +- [x] **`expandRecurringOccurrences` mixes a pg local-midnight Date with APP_TIMEZONE strings and UTC comparisons — every occurrence shifts a day back where host offset > app offset** 🔼 ✅ 2026-07-28 · acd913c (expansion rebuilt as pure YMD-string calendar math on the ADR-009 helpers; clamp cascade preserved byte-for-byte (Jan 31 monthly → …Jul 28); TZ×cadence it.each matrix pins identical occurrences under UTC/Tokyo/LA, fail-against-old reproduces the 6-day and clamp-shift failures; planned suites green under UTC/Tokyo/LA/UTC+14. Residue filed below: the pattern grammar is now encoded in two places) + - ↪ _from: Orchestration session 2026-07-28 · planned-month fix verification (verifier reproduced on HEAD and on the fixed tree under TZ=Asia/Tokyo: a weekly cadence lands 6 days off — pre-existing, NOT a regression of the window fix in the same file)_ + - `apps/node-backend/src/repositories/infoRepositoryPlanned.js:65-88` — `:81` feeds a pg local-midnight `Date` to `toAppDateString` (APP_TIMEZONE) and `:71-73` compares `current.getTime()` against `Date.UTC(...)`. Same TZ-mix class the window fix just removed from this file's other function. + - Fix: expand recurrences in YMD-string arithmetic via the ADR-009 helpers (addDaysYmd etc.), no Date getters; pin under forced TZ like the window tests. + +- [ ] **Net-worth fallback residues: series length driven by excluded rows; everything lands in `liquid` (no `is_liability` split)** 🔽 + - ↪ _from: Orchestration session 2026-07-28 · net-worth fallback fix (noticed by implementer, both reproduced by verifier)_ + - (a) `infoRepositoryNetWorth.js:73-80` — `firstDataDateYmd` is MIN(date) over ALL active transactions incl. tracking-only: an all-tracking ledger returns a 401-day all-zero snapshots array whose span comes entirely from excluded rows. (b) `:228-278` — the fallback has no `is_liability` split, so un-migrated liability rows land in `liquid` and `liabilities` is always 0 on that path (verifier probe: −5000 → liquid −5000, liabilities 0). + - Fix: (a) apply the same tracking-exclusion to the date probe; (b) split the fallback by the liability resolution the walk uses where attributable. + +- [ ] **`convertRowsToEur` emits `used_fallback_rate`/`fallback_reason` per row but no repository surfaces them — fallback-rate conversions are indistinguishable in every recipient/money view** ⏬ + - ↪ _from: Orchestration session 2026-07-28 · MoM fix pass (noticed; verifier confirmed no reader repo-wide; more reachable now that all four recipient surfaces use historical rates)_ + - `apps/node-backend/src/services/currency/currencyConversionService.js:389-391`. (The portfolio path's `usedFallbackRate` is a separate unrelated field.) + - Fix: thread the flags into the emitted rows where a surface wants to badge estimated conversions, or document that they are internal-only. + +- [ ] **Recurrence pattern grammar is now encoded in two steppers — adding a pattern to one silently diverges the other** ⏬ + - ↪ _from: Orchestration session 2026-07-28 · recurrence-TZ fix (noticed; documented in the new stepper's JSDoc)_ + - `infoRepositoryPlanned.js` `nextOccurrenceYmd` (string-space, with fast-forward) vs `lib/` `calculateNextDate` / `lib/calculations/recurrence.js` `expandOccurrences` (Date-space, no fast-forward, 500-cap that truncates stale daily rows before the window — the reason it couldn't be reused). + - Fix: extract one shared string-space stepper in `lib/` with an optional fast-forward, migrate both call sites onto it. + @@ -1020,7 +1210,7 @@ look-changing one. - In packaged mode every `composeStartOrUp` path returns `built:false` (`:1287` — `!skipBuild && (!app.isPackaged || useRepoMode)`), so an image-pull update polls with the 60s budget, not the 3-minute build budget. A migration pushing backend-listen past ~56s → `pollReady` rejects → `loadErrorPage()` + blocking "taking longer than expected" dialog *mid-migration*; when alembic later finishes, nothing re-navigates (watchdog only starts after a successful `pollReady`, `:1128`) — the user is stranded on the error page until manual Retry. - Fix: treat "new image pulled / pending migrations" as `building`, or expose a backend "migrating" signal (splash status line) and extend the poll budget while it's set. -- [ ] **Transaction search requests are never aborted — superseded keystrokes leave the expensive OR-chain scans running to completion server-side, and nothing gates 1-character queries client-side** ⏫ 🔎 verified-present 2026-07-11 🔎 partial-c6b2327 2026-07-14 (abort signal threaded) ✅ 2026-07-28 · f5fb531 (user signed off with a THREE-character threshold: shared VirtualDataTable forwards the trimmed term only at >=3 chars (SERVER_SEARCH_MIN_LENGTH) and resets to '' below — unfiltered, never stale, input text preserved; RecipientsPage's server search gets the same gate; cross-referenced with backend MIN_SEARCH_LENGTH for the planned server-side companion) +- [x] **Transaction search requests are never aborted — superseded keystrokes leave the expensive OR-chain scans running to completion server-side, and nothing gates 1-character queries client-side** ⏫ 🔎 verified-present 2026-07-11 🔎 partial-c6b2327 2026-07-14 (abort signal threaded) ✅ 2026-07-28 · f5fb531 (user signed off with a THREE-character threshold: shared VirtualDataTable forwards the trimmed term only at >=3 chars (SERVER_SEARCH_MIN_LENGTH) and resets to '' below — unfiltered, never stale, input text preserved; RecipientsPage's server search gets the same gate; cross-referenced with backend MIN_SEARCH_LENGTH for the planned server-side companion) - ↪ _from: Performance research 2026-07-09 · Wave F3 (frontend residues)_ - `apps/frontend/src/features/transactions/hooks/useTransactionListData.ts:102-121` (main query), `:149-168` (loadMore) — `queryFn` never receives/forwards React Query's abort `signal`; `apps/frontend/src/lib/api/transactions.ts:16-52` (`getTransactions`) has no `signal` param even though `client.ts:229-259` fully supports `options.signal` - Each debounced (300ms) keystroke changes the queryKey and starts a fresh backend search; React Query drops the stale query client-side but the HTTP request is NOT aborted, so the known-unindexable OR-chain scan (see the filed ⏫ search finding) runs to completion for every intermediate term — fast typing stacks N concurrent full scans server-side, competing for the 10-connection pool. Compounding it, `VirtualDataTable.tsx:164-175` forwards ANY non-empty value with no minimum-length gate, so typing "a" issues a search matching nearly every row (the filed backend search finding already lists a server-side min-length as part of its fix — the client half is missing too). @@ -3869,17 +4059,20 @@ look-changing one. - evidence: adding a section touches ~6 places: `sections/.js`; reports/index.js **three** spots (import :15-34, renderer map :414/:435/:454, default-order list :424/:444/:464); the matching data fetcher (dataFetcher*.js); the frontend's independently hand-maintained mirror lists (ExportDialog.tsx:58-90 `FINANCIAL/PORTFOLIO/TAX_SECTIONS`); i18n `export.section.*` en+nl. There is no `GET /sections` catalog, and unknown IDs are *silently filtered* server-side (`requested.filter(id => id in RENDERERS)` — reports/index.js:480,:503,:526), so a backend/frontend ID typo yields a PDF that quietly omits the section rather than erroring. - fix: per report type, export one `[{id, render, default}]` array from reports/index.js (derive renderer map + default list from it) and either expose it via a small catalog endpoint for ExportDialog or at least make generateReport reject unknown section IDs instead of silently dropping them. -- [ ] **Real-DB test harness built but adoption stalled at 1 suite; TEST_DATABASE_URL set nowhere, so its tests silently skip everywhere** ⏫ 🔎 verified-present 2026-07-11 🔎 partial-c2152e5 2026-07-27 (the DECIDED wiring is done and was executed against a live PG16 cluster stood up in-session: CI test-backend now runs a postgres:18 service + alembic migration with TEST_DATABASE_URL set (via new scripts/db-migrate.js, which delegates to runMigrations() because bare `alembic upgrade head` cannot migrate a fresh DB — VARCHAR(32) version column vs longer revision ids); local `bun run test:db` runs a disposable tmpfs container. Turning it on exposed that the one 'adopted' suite had NEVER run — 3 of aggregationRefresh's 4 DB cases asserted pre-squash schema artifacts (fixed from the real schema) and 2 cashflowForecastMethods tests passed only when Postgres was unreachable (repo now mocked with the documented 42P01 fallback). First migrated suite landed: transferReconciliation.db.test.js, 24 live-DB tests for the previously-untested 254-line service. Also fixed: .gitignore `*.db*` silently swallowed any *.db.test.js file. 3108 pass with DB (deterministic across runs), 3080 without. LEFT at that point: infoRepo*/transactionRepository* mock suites) 🔎 partial-c155d78 2026-07-28 (second increment: transactionRepository (34 live-DB tests) + infoRepoStatistics (8) migrated; DB suites now serialize via a pg advisory lock in tests/setup/db.js (parallel workers deleted each other's fixtures once a 2nd suite existed) and with-test-db.sh probes readiness over TCP (socket probe raced initdb's temporary server). The real DB exposed 3 mock-masked discrepancies, pinned + filed as findings below — incl. the ⏫ 0076 ON CONFLICT onboarding regression. test:db 3175/3175. LEFT: infoRepoBanks, infoRepoForecast, infoRepoMonthly, infoRepository (854 lines), infoRepositoryRecipients — advisory-lock pattern makes each drop-in) +- [x] **Real-DB test harness built but adoption stalled at 1 suite; TEST_DATABASE_URL set nowhere, so its tests silently skip everywhere** ⏫ ✅ 2026-07-28 · 28bd320 (third and final increment: infoRepoBanks (11), infoRepoForecast (10), infoRepoMonthly (11, executes BOTH the live generate_series path and a really-created mv_monthly_summary incl. live-vs-MV equivalence), infoRepository barrel (17), infoRepositoryRecipients (15) — every suite named in the DECIDED fix line is now on the harness (mock suites kept per precedent). The real DB exposed 9 more mock-masked discrepancies, each pinned by a marked 'PIN:' test and filed as findings (see the harness-third-increment block in Correctness): banks raw-Date leakage / cross-currency summation / manual-only history gap, forecast transfer-inclusion + populated-months denominator, recipients FX-rate inconsistency + alias-pivot mismatch, avg-vs-current denominator, net-worth manual-only step. 3247 pass with DB (deterministic ×3), 3108/139-skip without) 🔎 verified-present 2026-07-11 🔎 partial-c2152e5 2026-07-27 (the DECIDED wiring is done and was executed against a live PG16 cluster stood up in-session: CI test-backend now runs a postgres:18 service + alembic migration with TEST_DATABASE_URL set (via new scripts/db-migrate.js, which delegates to runMigrations() because bare `alembic upgrade head` cannot migrate a fresh DB — VARCHAR(32) version column vs longer revision ids); local `bun run test:db` runs a disposable tmpfs container. Turning it on exposed that the one 'adopted' suite had NEVER run — 3 of aggregationRefresh's 4 DB cases asserted pre-squash schema artifacts (fixed from the real schema) and 2 cashflowForecastMethods tests passed only when Postgres was unreachable (repo now mocked with the documented 42P01 fallback). First migrated suite landed: transferReconciliation.db.test.js, 24 live-DB tests for the previously-untested 254-line service. Also fixed: .gitignore `*.db*` silently swallowed any *.db.test.js file. 3108 pass with DB (deterministic across runs), 3080 without. LEFT at that point: infoRepo*/transactionRepository* mock suites) 🔎 partial-c155d78 2026-07-28 (second increment: transactionRepository (34 live-DB tests) + infoRepoStatistics (8) migrated; DB suites now serialize via a pg advisory lock in tests/setup/db.js (parallel workers deleted each other's fixtures once a 2nd suite existed) and with-test-db.sh probes readiness over TCP (socket probe raced initdb's temporary server). The real DB exposed 3 mock-masked discrepancies, pinned + filed as findings below — incl. the ⏫ 0076 ON CONFLICT onboarding regression. test:db 3175/3175. LEFT: infoRepoBanks, infoRepoForecast, infoRepoMonthly, infoRepository (854 lines), infoRepositoryRecipients — advisory-lock pattern makes each drop-in) - ↪ _from: Architecture & code design 2026-07-06 · Wave W4 (test architecture)_ - evidence: `apps/node-backend/tests/setup/db.js:1-53` is a deliberate opt-in real-PG seam ("Phase 0 step 6": `getTestPool()` + `it.skipIf(!hasTestDatabase())`), but only one production suite uses it (`tests/services/aggregationRefresh.test.js:37` `describe.skipIf(!hasTestDatabase())`) plus the harness self-test (`tests/setup/db.test.js`). `TEST_DATABASE_URL` appears in zero workflows and zero package.json scripts — the gated cases skip in CI *and* in every default local run. Meanwhile 60 suites mock the pool (`vi.mock('../src/database/connection.js')` 48× + 12× two-levels-deep), with 48 SQL-string assertions across 11 files, and multi-step `mockResolvedValueOnce` choreographies encoding exact query order (e.g. `tests/infoRepoMonthly.test.js:33-50`: MV probe → currency probe → data query, pinned fake clock). DB-bound orchestration like `src/services/transferReconciliationService.js` (254 lines) has no direct test at all — only its pure-calc extract is golden-tested (`tests/services/transfers.golden.test.js` → `calculations/transfers.js`, 87 lines) and the service is mocked in 3 bulk-route tests. These are exactly the tests the harness was built for. - fix: **DECIDED 2026-07-10: wire the real-DB harness.** Add a Postgres service to CI (compose service or testcontainers) + a local script wiring `TEST_DATABASE_URL`, then migrate the SQL-order-choreography suites (infoRepo*, transactionRepository*, transferReconciliation) onto it incrementally. The mock-only alternative is rejected — do not delete `setup/db.js`. -- [ ] **Route tests mock Express itself and execute only the last handler — middleware chains are never exercised; no supertest anywhere** ⏫ 🔎 verified-present 2026-07-11 +- [x] **Route tests mock Express itself and execute only the last handler — middleware chains are never exercised; no supertest anywhere** ⏫ ✅ 2026-07-28 · 6bce22a (final batch: settingsStorage's router-driven blocks + transactionPatchValidation onto the real harness (promise-rejection assertions became real 400 VALIDATION_ERROR envelopes through the actual error handler); rateLimiter/validation kept as middleware unit tests with local res stubs; main.test.js's express mock was dead scaffolding, removed; routeHarness.js DELETED at zero consumers. Whole migration: 35 suites across 5 batches, every route suite on supertest against the real router with the production middleware chain; previously-unreachable guards (admin auth, enforceAiChatEnabled, validateIdParam everywhere, csvUploadErrorTranslator) now genuinely exercised with pins; suite 3186 pass/182 skip, typecheck clean) 🔎 verified-present 2026-07-11 🔎 partial-d66368d 2026-07-28 (the preferred fix's harness is built and the first batch is on it: tests/helpers/routeApp.js mounts the real router on a throwaway express() app with the production middleware chain reproduced per main.js line-by-line (requestId, json limit, CSRF guard, ADR-026 wrapResponse, per-mount before slot, 404 funnel, createErrorHandler with injectable isProduction); deliberate omissions (CORS/gzip/app-level limiters/security headers) documented inline; legacy routeHarness.js header-marked. 8 money-path suites migrated (transactions + its 4 bulk suites + validation pins, plannedTransactions with real validateIdParam, importValidationPins with csvUploadErrorTranslator newly on-path); real CSRF 403s / 404 envelopes / id-param 400s / export headers now exercised; the previously-fake production-sanitization tests now really test it. Exposed the errorHandler HttpError→500 bug, filed below. Actual file count is 30, not the finding's 20. LEFT: 22 files on the legacy harness — accounts, admin, aggregations, aggregationsForecastBacktest, ai, attachments, categories, crossWorkspace, import, info, investments, marketLookup, portfolioImport, portfolioImportValidationPins, recipientBankAccounts, recipients, research, savedCharts, settings, splits, tags, watchlist — plus main.test.js, settingsStorage.test.js, transactionPatchValidation.test.js) 🔎 partial-6055196 2026-07-28 (batch 2a: 12 more suites migrated — accounts, aggregations, aggregationsForecastBacktest, attachments, categories, crossWorkspace, recipientBankAccounts, recipients, research, settings, tags, watchlist; validateIdParam no longer stubbed anywhere in them, 8 new on-path pins (guard 400s + attachments missing-file 400 previously unreachable); attachments multer stub now drives req.file through the real chain; watchlist Infinity test carried over as JSON-representable string "Infinity". No new defects exposed. Suite 3181 pass/182 skip. LEFT: batch 2b — admin, ai, import, info, investments, marketLookup, portfolioImport, portfolioImportValidationPins, savedCharts, splits — plus main.test.js, settingsStorage.test.js, transactionPatchValidation.test.js; then routeHarness.js can be deleted) 🔎 partial-a0fdec0 2026-07-28 (batch 2b: admin, ai, marketLookup, portfolioImport, portfolioImportValidationPins, savedCharts migrated. The admin auth guard (main.js:324 chain reproduced via `before` with the real createAdminAuthMiddleware; adminCsrfGuard redundant with the harness's global /api guard; app-level limiters per the documented fidelity gap) and ai's router.use(enforceAiChatEnabled) were UNREACHABLE under the legacy harness — 6 new pins (admin 401/200 bearer-token matrix, ai 503 when disabled, reachable malformed-id 400). SSE tests assert the buffered text/event-stream body. ai empty-id delete pin documented as a 404-funnel case (Express can't route an empty :id segment). Suite 3187 pass/182 skip. LEFT: final batch — import, info, investments, splits route suites + main.test.js, settingsStorage.test.js, transactionPatchValidation.test.js; then delete routeHarness.js) 🔎 partial-e67b348 2026-07-28 (batch 2c: import, info, investments, splits migrated — every tests/routes/ suite (all 30) is now on the real-Express harness. import's leak-test now really pins the production-sanitized branch (it previously only asserted rejects.toThrow, never the message); multer error paths run the real csvUploadErrorTranslator chain; POSTs without a body need explicit .send({}) since express.json() leaves req.body undefined. One structural test dropped with in-file justification (mock-router handler-map registration check; both routes keep dedicated request tests) — suite 3186 pass/182 skip. LEFT: the non-route routeHarness consumers — main.test.js, settingsStorage.test.js, transactionPatchValidation.test.js, PLUS rateLimiter.test.js and validation.test.js (two consumers the earlier LEFT lists missed); then delete routeHarness.js) - ↪ _from: Architecture & code design 2026-07-06 · Wave W4 (test architecture)_ - evidence: `tests/routes/transactions.test.js:7-19` — hand-rolled `mockRouter` stores `handlers[handlers.length - 1]`, so any validation/guard middleware registered before the handler is silently dropped from the test path; handlers are invoked with bare `{ query: {} }` req objects (line 69-71), bypassing Express query parsing, error-handler integration, and the ADR-026 envelope middleware path. `grep supertest` = zero hits in tests and package.json. `vi.mock('../../src/middleware/validation.js')` in 4 files stubs validation outright. The pattern is duplicated, not shared: `mockRouter` defined in 20 route files (3-4 hits each) and `mockResponse` re-declared in 22 files; no helper module exists (`tests/` has only `setup/db.js` and `golden/runGolden.js`). - fix: switch route suites to supertest against the real router mounted on a throwaway `express()` app (repos/services still mocked) — this restores middleware, status codes, and envelope behavior to the tested path; at minimum extract `mockRouter`/`mockResponse` into `tests/helpers/` and capture the full handler chain instead of `[length - 1]`. -- [x] **Backend coverage gate measures only files tests happen to load — new untested modules never trip the 85/88 thresholds, and the config comment overstates e2e route coverage** ⏫ ✅ 2026-07-27 · 0b9642a (coverage.include src/**/*.js with three justified excludes (test code, src/main.js process entrypoint, src/database/migrate.js alembic exec glue); thresholds honestly re-ratcheted to measured 80.25/70.35/78.5/82.16 → 78/68/76/80 (floor−2); the false Playwright-route-coverage comment replaced with what the gate actually guarantees (routes now counted — 30 route files in the report). 36 genuinely-untested files newly surfaced at 0% and the gate still passes. Verified: both figures re-measured exact, CI 🎯 targets read from this config via vite-config-path — no workflow edit needed) +- [ ] **info route tests run a deep unmocked service cascade under a blanket `query → {rows: []}` mock; import route tests carry an inert bankAdapters mock** ⏬ + - ↪ _from: Orchestration session 2026-07-28 · route-harness migration batch 2c (noticed, out of scope)_ + - `tests/routes/info.test.js` — `services/portfolio/portfolioSummaryService.js getPortfolioSummary` (reached transitively from `/api/info/net-worth`, `/portfolio-performance`, and `warmInfoCaches`) is real/unmocked and does non-trivial work (settings lookup, portfolio-transaction aggregation) driven only by the blanket pool mock returning empty rows. Pre-existed the harness migration (only Express was mocked before too), but it is a deep unmocked cascade for a route test — behavior changes in that service surface as confusing info-route failures. And `tests/routes/import.test.js` still mocks `services/bankAdapters.js`, which `importRoutes.js` no longer imports at all (the dead `GET /supported-adapters` route was removed) — the mock is inert. + - Fix: mock `portfolioSummaryService` at the service boundary in info.test.js (its own tests cover the internals); delete the inert bankAdapters mock from import.test.js. — new untested modules never trip the 85/88 thresholds, and the config comment overstates e2e route coverage** ⏫ ✅ 2026-07-27 · 0b9642a (coverage.include src/**/*.js with three justified excludes (test code, src/main.js process entrypoint, src/database/migrate.js alembic exec glue); thresholds honestly re-ratcheted to measured 80.25/70.35/78.5/82.16 → 78/68/76/80 (floor−2); the false Playwright-route-coverage comment replaced with what the gate actually guarantees (routes now counted — 30 route files in the report). 36 genuinely-untested files newly surfaced at 0% and the gate still passes. Verified: both figures re-measured exact, CI 🎯 targets read from this config via vite-config-path — no workflow edit needed) - ↪ _from: Architecture & code design 2026-07-06 · Wave W4 (test architecture)_ - evidence: `apps/node-backend/vitest.config.js:8-19` — Vitest 4 (`vitest@^4.1.7`, package.json:24) with no `coverage.include`: only files imported during the run are counted, so an entirely untested new service/repository is invisible to the threshold math (thresholds become "quality of tested files", not "coverage of codebase"). The comment at line 11-12 justifies this with "Routes are exercised end-to-end by frontend Playwright suites" — but the e2e suite is 630 lines of page-load smoke, dialog UX, a11y scans, and 4 CRUD creates (`e2e/mutations-parity.spec.ts:21-96`); it exercises a handful of GET paths and 3 POSTs, nothing like route coverage. Contrast the frontend, which does this honestly: explicit `coverage.include` + ratchet thresholds with measured baseline documented (`apps/frontend/vite.config.ts:118-147`). - fix: add `coverage.include: ['src/**/*.js']` (with targeted excludes) and re-ratchet thresholds to the real measured numbers, frontend-style; rewrite the comment to state what the gate actually guarantees. @@ -3919,11 +4112,16 @@ look-changing one. - evidence: `src/lib/api/__tests__/coverage-clients.test.ts` (537 lines) + `-2` (494) + `-3` (339) batch-import dozens of thin fetch-wrapper functions and exercise them against MSW. They aren't worthless (they route through the contract-checked handlers), but the naming is candid: these exist to feed the threshold, and they pad the "statements covered" figure with the easiest code in the app while the harder gaps (features/, contexts/ — see the include-list finding) sit unmeasured. - fix: no urgent action; when the coverage include-list is fixed and re-ratcheted, consider folding these into per-module client tests so file names describe behavior, not the metric. -- [ ] **checkJs CI gate is nominally load-bearing: `strict:false` + `noImplicitAny:false` and the data layer is untyped** ⏫ 🔎 verified-present 2026-07-11 🔎 partial-6345da9 2026-07-27 (the fix's structure is in place: src/types/rows.js with 29 pg-accurate row typedefs (raw vs formatted shapes distinct; NUMERIC/BIGINT=string, DATE=Date — no setTypeParser exists), repositories/ annotated 101→324 @param / 51→206 @returns, and a working noImplicitAny ratchet (tsconfig.check.strict.json + scripts/checkjs-ratchet.js, wired into CI + pre-push + package.json; verified to fail on a removed annotation; a directory-scoped 2nd tsconfig was measured and rejected — doesn't narrow the program, breaks ambient types under bun). 24 files ratcheted at 0 errors (src/types + 23 of 40 repositories). Ratchet options were measured: global noImplicitAny = 3349 errors. LEFT at that point: 17 repositories/ files) 🔎 partial-a1c3a8d 2026-07-27 (repositories/ COMPLETE: all remaining 17 files annotated to 0 errors (335→0; incl. infoRepositoryForecast 55, infoRepositoryHelpers 36), 8 new pg-accurate typedefs in src/types/rows.js verified against the Alembic migrations, and the ratchet list collapsed to directory prefixes src/types/ + src/repositories/ so new repository files are ratcheted from birth — ratchet verified to still bite (removed annotation → exit 1). 41 files clean. LEFT: further directories per the fix line (services/, routes/, lib/ remain un-ratcheted)) +- [ ] **checkJs CI gate is nominally load-bearing: `strict:false` + `noImplicitAny:false` and the data layer is untyped** ⏫ 🔎 verified-present 2026-07-11 🔎 partial-6345da9 2026-07-27 (the fix's structure is in place: src/types/rows.js with 29 pg-accurate row typedefs (raw vs formatted shapes distinct; NUMERIC/BIGINT=string, DATE=Date — no setTypeParser exists), repositories/ annotated 101→324 @param / 51→206 @returns, and a working noImplicitAny ratchet (tsconfig.check.strict.json + scripts/checkjs-ratchet.js, wired into CI + pre-push + package.json; verified to fail on a removed annotation; a directory-scoped 2nd tsconfig was measured and rejected — doesn't narrow the program, breaks ambient types under bun). 24 files ratcheted at 0 errors (src/types + 23 of 40 repositories). Ratchet options were measured: global noImplicitAny = 3349 errors. LEFT at that point: 17 repositories/ files) 🔎 partial-a1c3a8d 2026-07-27 (repositories/ COMPLETE: all remaining 17 files annotated to 0 errors (335→0; incl. infoRepositoryForecast 55, infoRepositoryHelpers 36), 8 new pg-accurate typedefs in src/types/rows.js verified against the Alembic migrations, and the ratchet list collapsed to directory prefixes src/types/ + src/repositories/ so new repository files are ratcheted from birth — ratchet verified to still bite (removed annotation → exit 1). 41 files clean. LEFT: further directories per the fix line (services/, routes/, lib/ remain un-ratcheted)) 🔎 partial-ea9d8dd 2026-07-28 (services/ measured at 1576 implicit-any errors in 152 files — 4.7× the repositories round — so taken subdirectory-wise: importPipeline, portfolioImportPipeline, prices, info, currency, tax annotated to 0 (347→0, 34 files) and added as ratchet prefixes; 6 new pg-accurate table-row typedefs verified against the Alembic migrations (ExchangeRateRow, PortfolioPerformanceSnapshotRow, AssetPriceHistoryRow, ImportStagingRow, PortfolioImportBatchRow, PortfolioImportStagingRow) plus ParsedBankTransaction/BankCsvAdapter contracts for the 9 bank adapters; HINT_SCOPE → src/services/; bite re-proven; 75 files clean. Typing surfaced 3 real mismatches, filed as findings (batch-id wire type split, dual rate_date shapes into buildHistoricalRateIndex, snapshot ?? 0 mixed types). LEFT: ~33 top-level services/*.js + calculations/ (275 err), research/ (204), reports/ (178), portfolio/ (127), aiChat/ (90); then routes/, lib/) 🔎 partial-76bc150 2026-07-28 (top-level services slice: measured 349 errors in the 33 ungated top-level files; the 26 smallest annotated 144→0 and listed, plus the 31 already-clean files the gate reported ready (18 top-level + 13 under calculations//research//reports//portfolio/) — 133 files now gated, bite re-proven. New: src/types/thirdPartyModules.d.ts ambient multer shim (value import → unconditional TS7016; pg/express stay structural-JSDoc since they appear only in type position), RecipientMatchPatternRow verified against migration 0015. One faithful-shape note: accountService assertStatementBalanceHasDate takes number|string (zod-sanitized number vs pg NUMERIC string) — presence check only, annotated as the union, no consequence. LEFT: 7 top-level files, 205 errors — dbEditor 51, belgianInflationService 38, priceProviderService 34, deduplication 24, aiChatService 21, transactionExport 21, marketLookupService 16; then the still-dirty bulk of calculations/, research/, reports/, portfolio/, aiChat/; then routes/, lib/) 🔎 partial-d644eba 2026-07-28 (src/services/ TOP LEVEL COMPLETE: the 7 remaining files annotated 205→0 — 51/51 top-level files gated as individual entries (prefix-collapse tested and rejected: it gates dirty subdir files); dbEditor typed faithfully as a generic table editor (Record rows, one boundary cast on introspection rows); BelgianInflationRateRow/BelgianInflationRate verified against migration 0001; bite re-proven via scratch-backup restore. 140 files clean. One wire-type divergence surfaced and filed below (transactionExport running_balance number vs repository NUMERIC string). LEFT: subdirectories calculations/, research/, reports/, portfolio/, aiChat/; then routes/, lib/) - ↪ _from: Architecture & code design 2026-07-06 · Wave W5 (cross-cutting concerns)_ - evidence: `apps/node-backend/tsconfig.check.json:10-11` (`"strict": false, "noImplicitAny": false`), run in CI at `.github/workflows/ci.yml:200`. Under these settings untyped params are silent `any`, so the check only catches gross misuse. Core data layer is where typing is absent: `src/repositories/transactionRepository.js` has 14 exports and **0** `@param`/`@returns`; all of `src/repositories/` (41 files) totals 71 `@param` vs 411 in `services/`. Only 15 `@typedef` sites repo-wide and **no shared domain-row typedefs** (no `TransactionRow`/`PlannedRow` contracts — `@vision/types` covers only error codes; `packages/shared-utils/src/*.d.ts` covers money/portfolio helpers). The clean `@ts-ignore` count (0 in src/) reflects the loose checker, not clean types. Net: row shapes flow into services as implicit `any`; typing is decorative exactly where SQL-shape drift is likeliest. - fix: add a shared `src/types/rows.js` (or `.d.ts` in `@vision/types`) with `@typedef` contracts for the ~10 core row shapes; annotate repository returns against them; then ratchet `noImplicitAny: true` per-directory (start `repositories/`), keeping `strict:false` elsewhere. +- [ ] **`running_balance` wire type diverges between the CSV export stream and the repository window query** ⏬ + - ↪ _from: Orchestration session 2026-07-28 · ratchet top-level services slice (typing surfaced, out of scope)_ + - `apps/node-backend/src/services/transactionExport.js:229` — `streamCsvExport` computes `running_balance` locally as a JS number (`next.toNumber()`), while the same-named field from `transactionRepository`'s `SUM(...) OVER (...)` window (`EnrichedTransactionRow.running_balance`) is a pg NUMERIC string. Not a bug today — CSV formatting handles both — but the two paths disagree on the type of the same conceptual field, and a future programmatic consumer of the export path could assume the repository's string shape. + - Fix: pick one wire shape for `running_balance` (the repository's NUMERIC-string is the established convention) and convert at the formatting boundary only; pin with a type-level assertion or test. + - [ ] **requestId reaches only 3 of ~286 logger call sites — no child-logger/ALS seam, so service/repo logs are uncorrelated** 🔼 🔎 verified-present 2026-07-11 - ↪ _from: Architecture & code design 2026-07-06 · Wave W5 (cross-cutting concerns)_ - evidence: `config/logger.js` is a module-level singleton with no `child()`/context mechanism (lines 37-50). `req.id` appears in log payloads only at `src/main.js:220` (entry debug line), `src/middleware/errorHandler.js:110`, and `src/routes/ai.js:288`. The other ~283 `logger.*` calls across 69 files (e.g. every service/repository warn/error) carry no correlation id, so a mid-request warning can't be tied to the request that the errorHandler later logs. The requestId middleware docstring (`src/middleware/requestId.js:4-8`) only promises envelope propagation, not log propagation — the gap is architectural, not accidental. diff --git a/alembic/versions/0085_mv_monthly_summary_alias_category.py b/alembic/versions/0085_mv_monthly_summary_alias_category.py new file mode 100644 index 00000000..28532d49 --- /dev/null +++ b/alembic/versions/0085_mv_monthly_summary_alias_category.py @@ -0,0 +1,59 @@ +"""Rebuild mv_monthly_summary with the 3-level effective-category resolution. + +Revision ID: 0085_mv_monthly_summary_alias_category +Revises: 0084_mv_category_totals_alias_category +Create Date: 2026-07-28 + +`mv_monthly_summary` (the getMonthlyFinancialSummary fast path) resolved the +effective category over TWO levels only — `COALESCE(t.category_id, +r.default_category_id)` — while the transactions surfaces resolve THREE +(`…, pr.default_category_id`): a row recorded under an alias recipient whose +PRIMARY carries the default category was categorised in the transactions list +but grouped under the UNCATEGORISED (`category_id_key = -1`) row of the monthly +view. The monthly live path in `infoRepositoryMonthly.getMonthlyFinancialSummary` +is fixed to the 3-level pattern in the same release; this migration drops the MV +so the fast path is rebuilt from the corrected definition and the two paths +cannot disagree on the category grain. + +The view's only current reader sums across categories, so the monthly totals +themselves are unchanged — this aligns the stored category grain (and the +`category_id_key` unique-index column) with every other money surface rather +than fixing a wrong total. + +`CREATE MATERIALIZED VIEW IF NOT EXISTS` never redefines an existing view, so +without this drop already-migrated installs would keep the old 2-level SQL +forever (REFRESH re-runs the *stored* definition). Same drop-and-let-boot- +recreate precedent as migrations 0045 and 0084: materializedViewService. +createMaterializedViews runs right after migrations on the same boot +(main.js), recreates the view with the new definition, and the startup / +post-mutation refresh populates it. Until then `mvAvailable()` sees it +unpopulated and getMonthlyFinancialSummary serves the (corrected) live query — +no window of wrong answers. + +Downgrade drops the view again so the older code recreates its prior 2-level +definition on next boot. Derived data only; nothing to preserve. +""" + +from typing import Sequence, Union + +from alembic import op + + +revision: str = "0085_mv_monthly_summary_alias_category" +down_revision: Union[str, Sequence[str], None] = "0084_mv_category_totals_alias_category" +branch_labels: Union[str, Sequence[str], None] = None +depends_on: Union[str, Sequence[str], None] = None + + +def upgrade() -> None: + # destructive-ok: derived data only, rebuilt from transactions by + # materializedViewService.createMaterializedViews on the same boot that applies this + # migration (precedent: 0045, 0084) — the recreation with the 3-level effective-category + # definition is the point of the drop. Its unique index drops with it via CASCADE. + op.execute("DROP MATERIALIZED VIEW IF EXISTS mv_monthly_summary CASCADE;") + + +def downgrade() -> None: + # destructive-ok: same rebuild-at-boot contract in reverse — older application code + # recreates the previous 2-level definition via createMaterializedViews on next start. + op.execute("DROP MATERIALIZED VIEW IF EXISTS mv_monthly_summary CASCADE;") diff --git a/apps/frontend/src/lib/api/imports.ts b/apps/frontend/src/lib/api/imports.ts index 235b375c..27dfbf12 100644 --- a/apps/frontend/src/lib/api/imports.ts +++ b/apps/frontend/src/lib/api/imports.ts @@ -50,7 +50,7 @@ const IMPORT_STREAM_SCHEMAS: Record = { export function importCSV( file: File, bankName: string, -): Promise<{ batch_id: string; imported: number; duplicates: number; total_processed: number; message: string }> { +): Promise<{ batch_id: number; imported: number; duplicates: number; total_processed: number; message: string }> { const queryParams = new URLSearchParams(); queryParams.append('bank_name', bankName); return postMultipartImport('/api/import/csv', file, queryParams); @@ -155,7 +155,7 @@ export function importCSVCustom( separator: string = ',', encoding: string = 'utf-8', skipRows: number = 0, -): Promise<{ batch_id: string; imported: number; duplicates: number; total_processed: number; message: string }> { +): Promise<{ batch_id: number; imported: number; duplicates: number; total_processed: number; message: string }> { const queryParams = new URLSearchParams(); queryParams.append('bank_name', bankName); queryParams.append('date_format', dateFormat); diff --git a/apps/frontend/src/types/generated.ts b/apps/frontend/src/types/generated.ts index 625ebc8b..89445024 100644 --- a/apps/frontend/src/types/generated.ts +++ b/apps/frontend/src/types/generated.ts @@ -8498,7 +8498,7 @@ export interface operations { content: { "application/json": components["schemas"]["Envelope"] & { data?: { - batch_id?: string; + batch_id?: number; total?: number; imported?: number; duplicates?: number; @@ -8515,7 +8515,7 @@ export interface operations { content: { "application/json": components["schemas"]["Envelope"] & { data?: { - batch_id?: string; + batch_id?: number; requires_review?: boolean; match_source_counts?: { [key: string]: number; diff --git a/apps/node-backend/package.json b/apps/node-backend/package.json index 85cbc849..518d85b0 100644 --- a/apps/node-backend/package.json +++ b/apps/node-backend/package.json @@ -24,6 +24,7 @@ "archiver": "^8.0.0", "eslint": "^10.7.0", "globals": "^17.7.0", + "supertest": "^7.2.2", "typescript": "^6.0.3", "vitest": "^4.1.10", "yauzl": "^3.4.0" diff --git a/apps/node-backend/scripts/checkjs-ratchet.js b/apps/node-backend/scripts/checkjs-ratchet.js index abacb7a7..0be7dc69 100644 --- a/apps/node-backend/scripts/checkjs-ratchet.js +++ b/apps/node-backend/scripts/checkjs-ratchet.js @@ -45,19 +45,117 @@ const CONFIG_PATH = path.join(ROOT, 'tsconfig.check.strict.json'); * Seeded with the core data-layer row shapes (`src/types/rows.js`), then grown * file-by-file until every repository was annotated. The whole data layer is * now held as directory prefixes, so a NEW file under either directory is - * ratcheted from birth. Per the original plan, the next targets are further - * directories (services/, routes/, …) — grow them file-by-file, then collapse - * to the directory prefix once complete. + * ratcheted from birth. + * + * `src/services/` is being taken the same way, one whole subdirectory at a + * time: a subdirectory is annotated to zero errors and only then added here as + * a prefix, so new files under it are ratcheted from birth too. Held so far as + * prefixes: the two import pipelines, currency/FX, prices, tax, and info. + * + * The top level of `src/services/` (all 51 `*.js` files directly in that + * directory, not its subdirectories) is now fully held — but still as 51 + * individual file paths, not a `src/services/` prefix: `isRatcheted`'s prefix + * match is per-string-prefix, not per-directory-level, so a `src/services/` + * entry would ALSO match every still-dirty subdirectory below (`aiChat/` and + * most of `calculations/`/`research/`/`reports/`/`portfolio/`) and gate them + * sight-unseen — instant CI failure on this script's next run. Collapsing the + * 51 entries to a directory prefix is only safe once those subdirectories are + * themselves fully clean (at which point `src/services/` alone, without the + * now-redundant subdirectory prefixes, would cover the whole tree). + * + * `calculations/`, `research/`, `reports/`, and `portfolio/` are each + * individually-clean-file-only so far (not yet whole subdirectories) — most + * of those directories are still implicit-any-dirty. `aiChat/` is untouched. + * Beyond services: `routes/` and `lib/` are untouched. * * @type {string[]} */ const RATCHETED = [ 'src/types/', 'src/repositories/', + 'src/services/currency/', + 'src/services/importPipeline/', + 'src/services/info/', + 'src/services/portfolioImportPipeline/', + 'src/services/prices/', + 'src/services/tax/', + + // calculations/, research/, reports/, portfolio/ — individually-clean files + // only; the directories themselves are still mostly implicit-any-dirty. + 'src/services/calculations/aggregation/_envelope.js', + 'src/services/calculations/aggregation/_statisticsCache.js', + 'src/services/calculations/aggregation/averageVsCurrent.js', + 'src/services/calculations/aggregation/bankBalances.js', + 'src/services/calculations/aggregation/category.js', + 'src/services/calculations/forecast/methods/ensemble.js', + 'src/services/calculations/forecast/methods/simpleAverage.js', + 'src/services/calculations/normalization.js', + 'src/services/portfolio/portfolioIncomeService.js', + 'src/services/reports/dataFetcherPortfolio.js', + 'src/services/reports/sectionCatalog.js', + 'src/services/research/adapters/schemas.js', + 'src/services/research/providerRegistry.js', + + // Top level of src/services/ — all 51 files, alphabetical. + 'src/services/accountMergeService.js', + 'src/services/accountService.js', + 'src/services/aggregationRefresh.js', + 'src/services/aiChatService.js', + 'src/services/attachmentCleanup.js', + 'src/services/attachmentRecordService.js', + 'src/services/attachmentService.js', + 'src/services/bankAdapters.js', + 'src/services/belgianInflationService.js', + 'src/services/bulkSelection.js', + 'src/services/cashForecastInsightService.js', + 'src/services/categoryOutlierService.js', + 'src/services/categoryService.js', + 'src/services/crossWorkspaceAnalytics.js', + 'src/services/crossWorkspaceDataService.js', + 'src/services/customParserConfigService.js', + 'src/services/dataImportService.js', + 'src/services/dbEditor.js', + 'src/services/deduplication.js', + 'src/services/importBatchService.js', + 'src/services/infoService.js', + 'src/services/insightsDigestService.js', + 'src/services/marketLookupService.js', + 'src/services/materializedViewService.js', + 'src/services/openingBalanceService.js', + 'src/services/plannedExecutionService.js', + 'src/services/plannedMatchService.js', + 'src/services/plannedTransactionService.js', + 'src/services/portfolioImportBatchService.js', + 'src/services/portfolioPerformanceSnapshotService.js', + 'src/services/priceProviderService.js', + 'src/services/providerHealthService.js', + 'src/services/quoteBackfillService.js', + 'src/services/recipientBankAccountService.js', + 'src/services/recipientClusterService.js', + 'src/services/recipientMergeService.js', + 'src/services/recipientPatternService.js', + 'src/services/recipientService.js', + 'src/services/reconcileService.js', + 'src/services/recurringDetectionService.js', + 'src/services/routeManifest.js', + 'src/services/savedChartsService.js', + 'src/services/settingsService.js', + 'src/services/splitService.js', + 'src/services/subscriptionCreepService.js', + 'src/services/tagService.js', + 'src/services/transactionBulkService.js', + 'src/services/transactionExport.js', + 'src/services/transactionService.js', + 'src/services/transferReconciliationService.js', + 'src/services/watchlistService.js', ]; -/** Directory the "ready to ratchet" hint scans for already-clean files. */ -const HINT_SCOPE = 'src/repositories/'; +/** + * Directory the "ready to ratchet" hint scans for already-clean files. Points + * at the frontier: `src/repositories/` is complete, so the hint now surfaces + * service files that are already implicit-any-clean and could be listed. + */ +const HINT_SCOPE = 'src/services/'; /** * @param {string} absolutePath diff --git a/apps/node-backend/src/middleware/errorHandler.js b/apps/node-backend/src/middleware/errorHandler.js index 3b559862..7f834307 100644 --- a/apps/node-backend/src/middleware/errorHandler.js +++ b/apps/node-backend/src/middleware/errorHandler.js @@ -8,6 +8,10 @@ * * Untyped errors fall through to a 500 with the production-safe message used by * the previous inline handler in main.js. Production mode hides raw messages. + * + * One exception: an untyped error that carries its own 4xx status (body-parser's + * http-errors, raised before any route runs) keeps that status. See THE RULE + * above `forwardable4xx` for exactly when, and when its message is echoed. */ import { logger } from '../config/logger.js'; @@ -74,6 +78,106 @@ export class RateLimitedError extends AppError { } } +/* ── Non-AppError errors that carry their own HTTP status ───────────────── + * + * Some errors reaching this handler are not ours and never will be: body-parser + * (mounted by `express.json()` in main.js:130) rejects a request BEFORE any + * route runs and raises an `http-errors` instance carrying the correct status — + * 400 for truncated JSON, 413 for a body over the 1 MB cap. Collapsing those to + * 500 reports a client typo as a server fault, and in production the 5xx + * sanitizer then hides the one thing the client needed to know ("request entity + * too large"). + * + * THE RULE (two independent decisions — status, then message): + * + * 1. STATUS is forwarded when a non-AppError carries a numeric `status` or + * `statusCode` in the 400-499 range. 5xx and nonsense values (NaN, 0, 700, + * strings) are ignored and still take the sanitized 500 path, so an + * internal failure can never downgrade itself into a client error. + * + * 2. MESSAGE is echoed verbatim ONLY when the error also carries a `type` from + * `TRUSTED_ERROR_TYPES` below — body-parser's fixed, non-sensitive strings. + * Every other forwarded 4xx gets the generic reason phrase for its status. + * Reason: `.status`/`.statusCode` is a convention any library may adopt, + * and its message is not vetted. In this codebase the only other non-AppError + * with a status is `OllamaError` (integrations/ollama/client.js:25), which + * stores the UPSTREAM provider's HTTP status and a message naming our + * internal call ("Ollama POST /api/chat failed with 404"); it is normally + * wrapped into an AppError (aiChatService.js:325), but if one ever escapes, + * this rule keeps the wording out of the response. Same for the plain + * `err.statusCode = 400` throws in services/calculations/loanSchedule.js:70,102 + * — they get a truthful 400 without this handler vouching for their text. + * + * `details` is still AppError-only: nothing here fabricates one. + */ + +/** body-parser `type` values whose message is a fixed library string, safe to echo. */ +const TRUSTED_ERROR_TYPES = new Set([ + 'entity.parse.failed', // 400 — malformed/truncated JSON body + 'entity.too.large', // 413 — body over the express.json({ limit }) cap + 'parameters.too.many', // 413 — urlencoded parameter-count cap + 'request.aborted', // 400 — client hung up mid-body + 'request.size.invalid', // 400 — Content-Length disagreed with the body read + 'charset.unsupported', // 415 + 'encoding.unsupported', // 415 + 'entity.verify.failed', // 403 — an express.json({ verify }) hook rejected it +]); + +/** Generic reason phrases for a forwarded 4xx whose own message is not trusted. */ +const GENERIC_4XX_MESSAGES = { + 400: 'Bad Request', + 401: 'Unauthorized', + 403: 'Forbidden', + 404: 'Not Found', + 405: 'Method Not Allowed', + 409: 'Conflict', + 413: 'Payload Too Large', + 415: 'Unsupported Media Type', + 422: 'Unprocessable Content', + 429: 'Too Many Requests', +}; + +/** + * Map a forwarded 4xx status onto the stable client-visible code vocabulary + * (ADR-026). The list is deliberately the existing one — no new codes are + * minted for these, since every UI already branches on VALIDATION_ERROR for a + * "your request was wrong" outcome. + * + * @param {number} status + * @returns {string} + */ +function codeForForwardedStatus(status) { + switch (status) { + case 401: return ApiErrorCode.UNAUTHORIZED; + case 403: return ApiErrorCode.FORBIDDEN; + case 404: return ApiErrorCode.NOT_FOUND; + case 409: return ApiErrorCode.CONFLICT; + case 429: return ApiErrorCode.RATE_LIMITED; + // 400 / 405 / 413 / 415 / 422 / any other 4xx: the request itself was + // rejected — VALIDATION_ERROR is the code clients already handle for that. + default: return ApiErrorCode.VALIDATION_ERROR; + } +} + +/** + * Decide whether a non-AppError may keep its own status, per THE RULE above. + * + * @param {any} err + * @returns {{ status: number, code: string, message: string }|null} null when the + * error must take the ordinary 500 path. + */ +function forwardable4xx(err) { + const raw = typeof err.status === 'number' ? err.status : err.statusCode; + if (typeof raw !== 'number' || !Number.isInteger(raw) || raw < 400 || raw > 499) return null; + + const trusted = typeof err.type === 'string' && TRUSTED_ERROR_TYPES.has(err.type); + return { + status: raw, + code: codeForForwardedStatus(raw), + message: trusted ? err.message : (GENERIC_4XX_MESSAGES[raw] || 'Request rejected'), + }; +} + /** * Factory: returns Express error-handling middleware bound to the provided * `isProduction` predicate. Injecting the predicate keeps this module free of @@ -96,12 +200,18 @@ export function createErrorHandler(isProduction) { } const isApp = err instanceof AppError; - const status = isApp ? err.status : 500; - const code = isApp ? err.code : ApiErrorCode.INTERNAL_SERVER_ERROR; + // Errors raised before any route ran (body-parser) carry a correct 4xx of + // their own — see THE RULE above for when it is honoured. + const forwarded = isApp ? null : forwardable4xx(err); + const status = isApp ? err.status : (forwarded ? forwarded.status : 500); + const code = isApp ? err.code : (forwarded ? forwarded.code : ApiErrorCode.INTERNAL_SERVER_ERROR); // Typed 4xx errors are expected business outcomes — log at warn, not error. const logFn = status >= 500 ? logger.error : logger.warn; - logFn.call(logger, isApp ? 'Handled application error' : 'Unhandled exception', { + const logLabel = isApp + ? 'Handled application error' + : (forwarded ? 'Rejected request' : 'Unhandled exception'); + logFn.call(logger, logLabel, { error: err.message, code, status, @@ -112,7 +222,13 @@ export function createErrorHandler(isProduction) { }); let message; - if (status < 500) { + if (forwarded) { + // Not ours: body-parser's fixed strings pass through, everything else + // gets the generic reason phrase. Unlike the 5xx branch this is NOT + // environment-dependent — the whole point is that a client hitting the + // body-size cap in production can still read why. + message = forwarded.message; + } else if (status < 500) { // 4xx messages are authored by us — safe to expose. message = err.message; } else if (isProduction()) { diff --git a/apps/node-backend/src/repositories/accountBalanceSql.js b/apps/node-backend/src/repositories/accountBalanceSql.js index 0c418208..5bd38599 100644 --- a/apps/node-backend/src/repositories/accountBalanceSql.js +++ b/apps/node-backend/src/repositories/accountBalanceSql.js @@ -1,5 +1,10 @@ /** - * Shared SQL for an account's *current* computed balance (ADR-094). + * Shared SQL for an account's computed balance (ADR-094). + * + * `COMPUTED_BALANCE_LATERAL` is the canonical *current*-balance definition; + * `computedBalanceByCurrencyLateral` and `computedBalanceSeriesCtes` below are + * the two derived forms (per-currency, and the same figure as a daily series) + * that the balance-history charts need in order to agree with it. * * The naive "latest active transaction with a non-null balance" lateral froze * at the last *imported* statement balance: `transactions.balance` is only @@ -58,4 +63,273 @@ export const COMPUTED_BALANCE_LATERAL = ` ) lb ON true `; -export default { COMPUTED_BALANCE_LATERAL }; +/** + * Same anchor+delta definition as {@link COMPUTED_BALANCE_LATERAL}, but emitted + * **per currency**: one row per currency the account holds. + * + * Why it exists: `SUM(t2.amount)` in the unpartitioned lateral adds a EUR + * amount to a USD amount as bare numbers, and the caller then converts that + * total at the ONE rate belonging to the most recent row's currency — 100 EUR + + * 100 USD at rate 0.5 came out as 100 EUR instead of 150. Partitioning the + * whole anchor+delta computation by currency and converting each partition + * separately is the only correct reading. + * + * Partition semantics (the composition question): a stamped + * `transactions.balance` is the bank's statement figure **for the currency of + * the row that carries it** — `transactions.currency` is per row, so there is + * no other consistent reading. An anchor therefore anchors *only its own + * currency's* partition; amounts in any other currency are summed from scratch + * in their own partition (they were never embedded in that statement figure). + * A statement-anchored EUR balance plus later USD deltas yields + * `(EUR: anchor + later EUR rows)` and `(USD: Σ USD rows)` — the anchor's + * partition carries the anchor, and nothing else does. + * + * For a single-currency account (the overwhelmingly common case) there is + * exactly one partition, its anchor is the account's latest stamped row and its + * delta is every active row after that anchor — i.e. byte-identical to + * {@link COMPUTED_BALANCE_LATERAL}. + * + * Emits ZERO rows for an account with no active rows at all (rather than a + * synthetic 0); every caller already excludes those accounts. + * + * Columns exposed under `alias`: + * - `currency` — partition currency (`COALESCE(t.currency, 'EUR')`). + * - `balance` — anchored running balance for that currency. It is a + * *current* balance, so callers convert it at today's rate, + * not at the partition's last-activity date. + * + * @param {{ account: string, alias?: string }} opts + * `account` is interpolated raw: pass a LITERAL SQL expression from the call + * site (`a.id`), never user input. + * @returns {string} + */ +export function computedBalanceByCurrencyLateral({ account, alias = 'bal' }) { + return ` + JOIN LATERAL ( + SELECT ccy.currency, + COALESCE(anch.balance, 0) + dlt.amount AS balance + FROM ( + SELECT COALESCE(t.currency, 'EUR') AS currency + FROM transactions t + WHERE t.account_id = ${account} AND t.is_active = true + GROUP BY COALESCE(t.currency, 'EUR') + ) ccy + LEFT JOIN LATERAL ( + SELECT t.balance, t.date, t.id + FROM transactions t + WHERE t.account_id = ${account} AND t.is_active = true + AND t.balance IS NOT NULL + AND COALESCE(t.currency, 'EUR') = ccy.currency + ORDER BY t.date DESC, t.id DESC + LIMIT 1 + ) anch ON true + JOIN LATERAL ( + SELECT COALESCE(SUM(t2.amount), 0) AS amount + FROM transactions t2 + WHERE t2.account_id = ${account} AND t2.is_active = true + AND COALESCE(t2.currency, 'EUR') = ccy.currency + AND (anch.date IS NULL OR (t2.date, t2.id) > (anch.date, anch.id)) + ) dlt ON true + ) ${alias} ON true +`; +} + +/** + * The same anchor+delta balance as a **daily series**: one row per + * (account[, currency], day) over a caller-supplied day grid. + * + * Evaluating the lateral above once per (account, day) is the obvious + * formulation and is O(days × rows): every day re-sums the whole post-anchor + * window, and for a never-stamped account that window is the account's entire + * history. Measured on a 15k-row / 5-year ledger that form ran ~2.3s against + * the ~50ms of the stamped-only probe both history queries used before this + * change (the shipped baseline these numbers are compared against). This + * computes the identical figures in one pass over the rows plus one over the + * spans they cover, via the identity + * + * balance(day) = cum(last row ≤ day) + adj(last stamped row ≤ day) + * where cum(r) = Σ amounts up to and including r + * adj(r) = r.balance − cum(r) (0 when nothing is stamped yet) + * + * which is just `anchor.balance + Σ(amounts after the anchor)` rearranged: + * `adj` is the constant that the bank's statement figure adds on top of our own + * running sum, and it only changes when a newer stamp arrives. + * + * Emits CTE definitions (comma-terminated, to splice into a WITH chain AFTER + * the caller's own) ending in `balance_series(account_id, currency, day, + * balance, row_currency)`, where `row_currency` is the currency of the latest + * active row on or before that day — the FX currency for a cross-currency + * (`byCurrency: false`) series, and redundant with `currency` otherwise. The + * caller must already define: + * - `account_list(account_id, …)` — the accounts in scope. + * - `days(day)` — the day grid. Only its MIN/MAX are read; + * the emitted series is dense between them, + * which for the dense grids both callers + * generate is the same set of days. + * + * Series edges: a day before the account's first active row yields NO row (its + * first known balance is never carried backwards). Activity that predates the + * grid is folded onto the grid's first day, so a series that started earlier + * opens at its true running balance rather than at zero. Rows after the grid's + * last day are excluded, so a future-dated transaction does not leak backwards. + * + * @param {{ byCurrency?: boolean, accountList?: string, days?: string }} [opts] + * `byCurrency` partitions the whole computation by `transactions.currency` + * (each partition anchored by its own stamps — see + * {@link computedBalanceByCurrencyLateral} for why that is the only + * consistent reading). With it false the sum is cross-currency, matching + * {@link COMPUTED_BALANCE_LATERAL}, and `currency` is a constant the caller + * should ignore. Names are interpolated raw: pass LITERAL CTE names. + * @returns {string} + */ +export function computedBalanceSeriesCtes({ + byCurrency = false, + accountList = 'account_list', + days = 'days', +} = {}) { + const currencyExpr = byCurrency ? `COALESCE(t.currency, 'EUR')` : `''::text`; + // Only the per-currency form confines a partition to one currency; the + // cross-currency form has a single partition per account. + const samePartition = byCurrency ? `AND COALESCE(t.currency, 'EUR') = p.currency` : ''; + // The window partition drops the currency column entirely in cross-currency + // mode: carrying a constant in the PARTITION BY only lengthens every sort key. + const partitionBy = byCurrency ? 'account_id, currency' : 'account_id'; + return ` + bs_span AS ( + SELECT MIN(day) AS first_day, MAX(day) AS last_day FROM ${days} + ), + bs_opening AS ( + -- Everything that happened BEFORE the grid, collapsed into one synthetic row + -- per (account[, currency]): a 12-month chart over a 5-year ledger otherwise + -- drags all five years through the window pass below just to know where the + -- window opens. The row is dated the day before the grid and carries the + -- anchor+delta balance as of then, in the balance column — i.e. it enters + -- the machinery below as a *stamp*, which is exactly what a carried-in + -- balance is (it embeds all prior activity, so its own amount is 0). + -- Empty when the grid starts at or before the account's first row, which is + -- the net-worth case. + SELECT p.account_id, + p.currency, + ${byCurrency ? 'p.currency' : 'rc.row_currency'} AS row_currency, + (SELECT first_day FROM bs_span) - 1 AS date, + -1 AS id, + COALESCE(anch.balance, 0) + dlt.amount AS balance, + 0::numeric AS amount + FROM ( + SELECT DISTINCT t.account_id, ${currencyExpr} AS currency + FROM transactions t + JOIN ${accountList} bs_al ON bs_al.account_id = t.account_id + WHERE t.is_active = true AND t.date < (SELECT first_day FROM bs_span) + ) p + LEFT JOIN LATERAL ( + SELECT t.balance, t.date, t.id + FROM transactions t + WHERE t.account_id = p.account_id AND t.is_active = true + AND t.balance IS NOT NULL + AND t.date < (SELECT first_day FROM bs_span) + ${samePartition} + ORDER BY t.date DESC, t.id DESC + LIMIT 1 + ) anch ON true + JOIN LATERAL ( + SELECT COALESCE(SUM(t.amount), 0) AS amount + FROM transactions t + WHERE t.account_id = p.account_id AND t.is_active = true + AND t.date < (SELECT first_day FROM bs_span) + ${samePartition} + AND (anch.date IS NULL OR (t.date, t.id) > (anch.date, anch.id)) + ) dlt ON true + ${byCurrency ? '' : `LEFT JOIN LATERAL ( + SELECT COALESCE(t.currency, 'EUR') AS row_currency + FROM transactions t + WHERE t.account_id = p.account_id AND t.is_active = true + AND t.date < (SELECT first_day FROM bs_span) + ORDER BY t.date DESC, t.id DESC + LIMIT 1 + ) rc ON true`} + ), + bs_src AS ( + SELECT account_id, currency, row_currency, date, id, balance, amount + FROM bs_opening + UNION ALL + SELECT t.account_id, + ${currencyExpr}, + COALESCE(t.currency, 'EUR'), + t.date, t.id, t.balance, t.amount + FROM transactions t + JOIN ${accountList} bs_al ON bs_al.account_id = t.account_id + WHERE t.is_active = true + AND t.date >= (SELECT first_day FROM bs_span) + AND t.date <= (SELECT last_day FROM bs_span) + ), + bs_rows AS ( + -- Everything that can share ONE ordering of the rows is computed here, in a + -- single window pass: the running sum, and (via LEAD) the flag marking the + -- last row of each day. Splitting these across CTEs with different window + -- partitions costs a full re-sort of the ledger apiece. + -- The day column clamps the opening row onto the grid's first day; because + -- it is monotonic along the window order, "last row of the day" is simply + -- "the next row belongs to another day". + SELECT account_id, + currency, + row_currency, + date, + id, + balance, + GREATEST(date, (SELECT first_day FROM bs_span)) AS day, + LEAD(GREATEST(date, (SELECT first_day FROM bs_span))) OVER bs_w AS next_day, + SUM(amount) OVER bs_w AS cum + FROM bs_src + WINDOW bs_w AS (PARTITION BY ${partitionBy} ORDER BY date, id) + ), + bs_carry AS ( + -- Carry the latest stamp's adj (balance − cum) forward. MAX over the packed + -- [date, id, adj] array IS that carry-forward: the array compares + -- element-wise, (date, id) is unique, and the frame is everything up to the + -- current row — so the maximum is the newest stamp at or before it, and + -- element 3 is its adj. Written this way to reuse bs_rows' ordering; the + -- readable alternative (FIRST_VALUE over a COUNT(balance) grouping) needs a + -- different partition and therefore a second sort of the whole ledger. + SELECT account_id, currency, row_currency, day, next_day, cum, + MAX(CASE WHEN balance IS NOT NULL + THEN ARRAY[(date - DATE '2000-01-01')::numeric, id::numeric, balance - cum] + END) OVER (PARTITION BY ${partitionBy} ORDER BY date, id) AS stamp + FROM bs_rows + ), + bs_day_end AS ( + -- End-of-day balance per (account[, currency], day). No stamp yet → the + -- carry is NULL and the balance is the plain running sum. + SELECT account_id, currency, row_currency, day, + cum + COALESCE(stamp[3], 0) AS balance + FROM bs_carry + WHERE next_day IS DISTINCT FROM day + ), + bs_spans AS ( + -- Each day-end balance holds until the day before the next one (or the end + -- of the grid). Expanding these spans is what fills the quiet days. + SELECT account_id, currency, balance, row_currency, + day AS from_day, + COALESCE( + LEAD(day) OVER (PARTITION BY ${partitionBy} ORDER BY day) - 1, + (SELECT last_day FROM bs_span) + ) AS thru_day + FROM bs_day_end + ), + balance_series AS ( + -- Expand rather than join: forward-filling by LEFT JOINing a dense grid + -- against a statistics-less CTE let the planner demote the day column from + -- the merge key to a join filter, making the fill O(days²) per account — + -- measured 27× slower end-to-end than the stamped probe this replaced. + -- generate_series emits each span's days directly, so the fill costs one + -- row per output row and no join at all. + SELECT s.account_id, s.currency, g.day::date AS day, s.balance, s.row_currency + FROM bs_spans s + CROSS JOIN LATERAL generate_series(s.from_day, s.thru_day, interval '1 day') AS g(day) + )`; +} + +export default { + COMPUTED_BALANCE_LATERAL, + computedBalanceByCurrencyLateral, + computedBalanceSeriesCtes, +}; diff --git a/apps/node-backend/src/repositories/infoRepository.js b/apps/node-backend/src/repositories/infoRepository.js index d07b214a..d31d9330 100644 --- a/apps/node-backend/src/repositories/infoRepository.js +++ b/apps/node-backend/src/repositories/infoRepository.js @@ -19,6 +19,7 @@ export { clearMvCache } from './infoRepositoryHelpers.js'; +import { getIncludeTransfers } from './infoRepositoryHelpers.js'; import { statisticsRepository } from './infoRepositoryStatistics.js'; import { getMonthlyFinancialSummary } from './infoRepositoryMonthly.js'; import { getAverageVsCurrentSpending } from './infoRepositoryAverageVsCurrent.js'; @@ -35,6 +36,10 @@ import { recipientInsightsRepository } from './infoRepositoryRecipients.js'; export const infoRepository = { ...statisticsRepository, + // ADR-083 toggle read. Exposed on the facade because the forecast cache key + // (services/calculations/forecast/index.js) needs it before it decides + // whether to call any of the repositories below. + getIncludeTransfers, getMonthlyFinancialSummary, getAverageVsCurrentSpending, getCashflowComparison, diff --git a/apps/node-backend/src/repositories/infoRepositoryAverageVsCurrent.js b/apps/node-backend/src/repositories/infoRepositoryAverageVsCurrent.js index 10b738ed..ae5badd4 100644 --- a/apps/node-backend/src/repositories/infoRepositoryAverageVsCurrent.js +++ b/apps/node-backend/src/repositories/infoRepositoryAverageVsCurrent.js @@ -14,6 +14,67 @@ import { } from './infoRepositoryHelpers.js'; import { toAppTz } from '../lib/timezone.js'; +/** Lookback length: N complete, already-elapsed calendar months. */ +const WINDOW_MONTHS = 6; + +/** + * 'YYYY-MM' month key for a pg DATE column (a JS Date via node-postgres, or a + * string on some paths), or null when the column was NULL/absent. + * @param {unknown} value + * @returns {string|null} + */ +function monthKeyFromDbDate(value) { + if (value == null) return null; + const ymd = value instanceof Date ? formatDateToYmd(value) : String(value).slice(0, 10); + return /^\d{4}-\d{2}/.test(ymd) ? ymd.slice(0, 7) : null; +} + +/** + * Denominator for the historical averages — the SAME counting rule + * `infoRepositoryForecast.countObservedMonths` uses (elapsed months from the + * ledger's first in-window transaction through the last complete month), + * applied to this card's 6-month window. Note the forecast card's window is + * 24 months, so the two cards' divisors legitimately differ for ledgers older + * than 6 months — what is shared is the rule, not the number. Two failure + * modes bracket the right answer: + * + * - Dividing by "months that happen to carry rows" (the old behaviour here) + * reported a single busy month at FULL weight — one 240-spend month became a + * 240/month "6-month average" — and let a month whose only rows are INCOME + * enter the divisor as a *populated* month, halving the figure with zero + * extra spend. An elapsed month is a real observation either way: the user + * spent nothing, so it must count as a zero, not be skipped and not be + * counted only when some unrelated row lands in it. + * - Dividing by the whole window unconditionally deflates a short ledger: a + * user who installed last month has no data for month -5 because the app did + * not exist for them, not because they spent nothing. + * + * So the divisor is the span from the month the ledger started through the last + * complete month, inclusive, capped at the window. Empty months inside that + * span count as zero; months before the ledger's first entry are not counted. + * + * `ledgerStartMonth` MUST come from an unfiltered probe of `transactions` + * (sqlLedgerStart below): deriving it from the filtered result set would let the + * ADR-083 transfer predicate empty the oldest months and silently re-base the + * divisor. + * + * @param {string|null} ledgerStartMonth 'YYYY-MM' of the ledger's first + * in-window transaction, or null when it has none. + * @param {number} lastCompleteMonthIdx Absolute month index (year*12 + month-1) + * of the last complete month. + * @returns {number} Months to divide by; always in [1, WINDOW_MONTHS]. + */ +function countObservedMonths(ledgerStartMonth, lastCompleteMonthIdx) { + if (!ledgerStartMonth) return 1; + const startIdx = Number(ledgerStartMonth.slice(0, 4)) * 12 + (Number(ledgerStartMonth.slice(5, 7)) - 1); + const span = lastCompleteMonthIdx - startIdx + 1; + // Clamp only guarantees the divisor lands in [1, WINDOW_MONTHS]; it does not + // reconcile the Postgres CURRENT_DATE clock feeding `ledgerStartMonth` with + // the app-timezone clock feeding `lastCompleteMonthIdx` (same drift noted in + // infoRepositoryForecast.js). It is here so the span can never be 0. + return Math.min(WINDOW_MONTHS, Math.max(1, span)); +} + export async function getAverageVsCurrentSpending(targetCurrency = 'EUR') { // Exclude internal transfers (ADR-083) from spending aggregates unless the // user has explicitly opted in. Without this, a checking->savings transfer's @@ -33,7 +94,7 @@ export async function getAverageVsCurrentSpending(targetCurrency = 'EUR') { FROM transactions t WHERE t.is_active = true ${transferFilter} - AND t.date >= date_trunc('month', CURRENT_DATE) - interval '6 months' + AND t.date >= date_trunc('month', CURRENT_DATE) - interval '${WINDOW_MONTHS} months' AND t.date < date_trunc('month', CURRENT_DATE) GROUP BY t.date, t.currency, (t.amount < 0) `; @@ -47,9 +108,21 @@ export async function getAverageVsCurrentSpending(targetCurrency = 'EUR') { GROUP BY t.date, t.currency, (t.amount < 0) `; - const [past6Result, currentResult] = await Promise.all([ + // Ledger start for the average denominators (countObservedMonths above). + // Deliberately UNFILTERED — no transfer predicate — and kept LAST in the + // Promise.all so the two data queries keep their call order. "When did this + // ledger start having history" is a property of the ledger, not of this view. + const sqlLedgerStart = ` + SELECT MIN(t.date) AS first_date + FROM transactions t + WHERE t.is_active = true + AND t.date >= date_trunc('month', CURRENT_DATE) - interval '${WINDOW_MONTHS} months' + `; + + const [past6Result, currentResult, ledgerStartResult] = await Promise.all([ query(sql6m), query(sqlCurrent), + query(sqlLedgerStart), ]); const past6Converted = await convertRowsToEur( @@ -63,29 +136,46 @@ export async function getAverageVsCurrentSpending(targetCurrency = 'EUR') { const dateStr = row.date instanceof Date ? formatDateToYmd(row.date) : row.date; const eur = row.amount_eur; const monthKey = extractYearMonth(dateStr); + // Income rows contribute nothing at all — not even a month key. The + // denominator is the observed calendar span (below), never "months that + // carry a row", so an income-only month must not enter it as a populated + // month; it is already counted as an elapsed zero-spend month. + if (eur >= 0) continue; if (!monthlySpending[monthKey]) monthlySpending[monthKey] = 0; // Decimal accumulation (money-hygiene): native `+=` over many converted // rows drifts sub-cent before the final round. - if (eur < 0) monthlySpending[monthKey] = toNumber(toDecimal(monthlySpending[monthKey]).plus(toDecimal(Math.abs(eur)))); + monthlySpending[monthKey] = toNumber(toDecimal(monthlySpending[monthKey]).plus(toDecimal(Math.abs(eur)))); } const monthKeys = Object.keys(monthlySpending); - const monthsCount = monthKeys.length || 1; const totalMonthlySpending = toNumber(addAll(monthKeys.map((k) => monthlySpending[k]))); - const avgMonthlySpending = totalMonthlySpending / monthsCount; - // Calendar-day denominators, NOT counts of days that happened to have a - // transaction. The 6-month window is 6 complete prior months; dividing by - // transaction-day counts overstated the per-day rate (and the projection - // below multiplied a per-active-day rate by full calendar days). + // Calendar denominators, NOT counts of months/days that happened to have a + // transaction. Both figures divide the SAME numerator by the SAME window + // expressed in two units, so `avg_monthly_spending` and `avg_daily_spending` + // agree by construction: monthly / daily == days-per-month of that window. + // + // The window is the observed one (see countObservedMonths): the last + // `monthsCount` complete months, where `monthsCount` runs from the ledger's + // first in-window transaction through the last complete month, capped at + // WINDOW_MONTHS. Empty months inside it are real zeros; months before the + // ledger existed are not charged as zeros. + // // "Today" is resolved in APP_TIMEZONE (ADR-009), not the server process's // local time, so this agrees with the CURRENT_DATE-based SQL above near // midnight regardless of host TZ. const { year: nowYear, month: nowMonth, day: nowDay } = toAppTz(new Date()); - const sixMonthStart = new Date(Date.UTC(nowYear, nowMonth - 1 - 6, 1)); + const lastCompleteMonthIdx = nowYear * 12 + (nowMonth - 1) - 1; + const monthsCount = countObservedMonths( + monthKeyFromDbDate(ledgerStartResult.rows[0]?.first_date), + lastCompleteMonthIdx, + ); + const avgMonthlySpending = totalMonthlySpending / monthsCount; + + const observedStart = new Date(Date.UTC(nowYear, nowMonth - 1 - monthsCount, 1)); const currentMonthStart = new Date(Date.UTC(nowYear, nowMonth - 1, 1)); - const calendarDays6m = Math.max(1, Math.round((currentMonthStart.getTime() - sixMonthStart.getTime()) / 86400000)); - const avgDailySpending = totalMonthlySpending / calendarDays6m; + const calendarDaysObserved = Math.max(1, Math.round((currentMonthStart.getTime() - observedStart.getTime()) / 86400000)); + const avgDailySpending = totalMonthlySpending / calendarDaysObserved; const currentConverted = await convertRowsToEur( mapRowsForAmountConversion(currentResult.rows, 'amount', false), diff --git a/apps/node-backend/src/repositories/infoRepositoryBanks.js b/apps/node-backend/src/repositories/infoRepositoryBanks.js index 4f6ef917..90f73992 100644 --- a/apps/node-backend/src/repositories/infoRepositoryBanks.js +++ b/apps/node-backend/src/repositories/infoRepositoryBanks.js @@ -5,9 +5,14 @@ import { query } from '../database/connection.js'; import { toDecimal, toNumber } from '../lib/money.js'; import { toYmd } from '../utils/portfolioMath.js'; -import { COMPUTED_BALANCE_LATERAL } from './accountBalanceSql.js'; +import { + COMPUTED_BALANCE_LATERAL, + computedBalanceByCurrencyLateral, + computedBalanceSeriesCtes, +} from './accountBalanceSql.js'; import { roundToCents, + toWireDate, batchConvertGroupsWithHistoricalRateFallback, } from './infoRepositoryHelpers.js'; @@ -35,10 +40,27 @@ export const banksRepository = { * - `anchor_date` / `post_anchor_count` — balance provenance from the * shared lateral ("as of {date} statement + {n} entries since" vs * "sum of {n} entries" when anchor_date is absent). + * + * Multi-currency accounts: the anchor+delta computation is partitioned by + * `transactions.currency` (`computedBalanceByCurrencyLateral`) and each + * partition is converted at its own rate before the per-account total is + * summed. The previous single-partition form added a EUR amount to a USD + * amount as bare numbers and converted the sum at the most recent row's rate + * (100 EUR + 100 USD at rate 0.5 → 100 instead of 150). Single-currency accounts + * have exactly one partition and are unaffected. + * + * The 12-month history uses that SAME definition evaluated as of each day, + * and both sides convert at the rate of the day they represent (the history + * at each day, the headline at today — `CURRENT_DATE`, where the series + * ends). So the last history point equals `total_net_position`, in every + * currency, by construction. The series is no longer gated on stamped rows + * (which hid manual-only accounts from the chart while they counted in the + * headline, and froze stamped accounts at their last statement figure). An + * account contributes no point at all before its first active row (not a + * zero, and not its first known balance carried backwards). */ async getBankBalances(targetCurrency = 'EUR') { const accounts = []; - let totalNetPosition = 0; // Both queries are independent — run in parallel, then batch-convert // with one historical-rate lookup instead of two. @@ -46,27 +68,39 @@ export const banksRepository = { query(` SELECT a.name AS bank_account, COALESCE(a.display_name, a.name) AS display_name, - tx.currency, - COALESCE(lb.balance, 0) AS balance, + bal.currency, + bal.balance, CASE WHEN a.statement_balance IS NOT NULL THEN a.statement_balance - COALESCE(lb.balance, 0) ELSE NULL END AS drift, lb.anchor_date, lb.post_anchor_count, - tx.last_transaction AS date, + -- FX anchor for the conversion below: TODAY, the day this + -- balance represents — the same day the history series ends on, + -- so the headline and the last chart point convert at one rate. + -- Keying it on the account's last activity instead (as this did) + -- revalued a stamped foreign-currency account at the rate of its + -- last statement while the chart moved with the daily rate: a + -- USD account last imported 30 days ago showed 500 over a chart + -- ending at 900. Native-currency figures (drift) are unaffected. + to_char(CURRENT_DATE, 'YYYY-MM-DD') AS date, tx.transaction_count, tx.first_transaction, tx.last_transaction FROM accounts a ${COMPUTED_BALANCE_LATERAL} + -- One row per currency the account holds, each with its own anchored + -- running balance, converted independently below (the per-account total + -- is summed after conversion). lb above stays the account-level + -- (cross-currency) figure ONLY for drift and the provenance fields, so + -- the drift badge keeps matching the accounts hub, which reads the same + -- lateral. + ${computedBalanceByCurrencyLateral({ account: 'a.id' })} JOIN LATERAL ( - -- Per-account activity metadata over active rows. The currency is the - -- most recent active row's (multi-currency partitioning is D2); the - -- date anchors the FX conversion below to the latest activity. + -- Per-account activity metadata over active rows. SELECT COUNT(*) AS transaction_count, MIN(t.date) AS first_transaction, - MAX(t.date) AS last_transaction, - (ARRAY_AGG(COALESCE(t.currency, 'EUR') ORDER BY t.date DESC, t.id DESC))[1] AS currency + MAX(t.date) AS last_transaction FROM transactions t WHERE t.account_id = a.id AND t.is_active = true ) tx ON true @@ -76,7 +110,7 @@ export const banksRepository = { -- leaves this widget the moment it is closed, matching net worth. AND a.in_net_worth = true AND tx.transaction_count > 0 - ORDER BY a.name + ORDER BY a.name, bal.currency `), query(` WITH days AS ( @@ -97,37 +131,37 @@ export const banksRepository = { SELECT t.account_id FROM transactions t WHERE t.is_active = true AND t.account_id IS NOT NULL ) - ) - -- One index-backed LATERAL probe per (account, day) for the latest - -- balance ≤ day, instead of a ROW_NUMBER over a series×transactions - -- CROSS JOIN that materialized many copies of the table. Mirrors the - -- "latest balance ≤ day" pattern in infoRepositoryNetWorth.js; same - -- tie-break (date DESC, id DESC). Uses idx_transactions_account_date_active. + ), + -- The series carries the SAME per-currency anchor+delta definition the + -- current balance above uses, evaluated at every day. The old "latest + -- STAMPED balance ≤ day" probe (plus a WHERE lb.balance IS NOT NULL + -- gate) dropped never-stamped accounts from the chart while they + -- counted in total_net_position, and froze stamped accounts at their + -- last statement, so today's chart point disagreed with the headline + -- sitting above it. A day before the account's first active row still + -- yields no point, so a series starts at first activity rather than + -- back-filling zeroes. + -- -- Daily (not month-end) points: the dashboard Balance History chart's -- time-scale auto-ticks outnumbered monthly datapoints, duplicating - -- month labels; ~365 probes per account stay cheap index scans. + -- month labels. + -- + -- Note the series is bounded at each day while the headline (like the + -- accounts hub) is unbounded, so a FUTURE-dated row counts in the + -- headline before it reaches the chart. That is pre-existing and + -- deliberate: bounding the headline instead would diverge it from the + -- hub. + ${computedBalanceSeriesCtes({ byCurrency: true })} -- to_char keeps the day a plain string (pg DATE → local-midnight JS Date -- otherwise — the recurring day-shift hazard). SELECT a.bank_account, - to_char(d.day, 'YYYY-MM-DD') AS day, - COALESCE(lb.currency, 'EUR') AS currency, - lb.balance, - lb.date - FROM days d - CROSS JOIN account_list a - LEFT JOIN LATERAL ( - SELECT t.currency, t.balance, t.date - FROM transactions t - WHERE t.is_active = true - AND t.account_id = a.account_id - AND t.balance IS NOT NULL - AND t.date <= d.day - ORDER BY t.date DESC, t.id DESC - LIMIT 1 - ) lb ON true - WHERE lb.balance IS NOT NULL - ORDER BY a.account_id, d.day + to_char(s.day, 'YYYY-MM-DD') AS day, + s.currency, + s.balance + FROM balance_series s + JOIN account_list a ON a.account_id = s.account_id + ORDER BY s.account_id, s.day, s.currency `), ]); @@ -145,14 +179,18 @@ export const banksRepository = { amount: toNumber(toDecimal(r.balance)), currency: r.currency || 'EUR', })), + // History rows carry no `date`: the conversion helper falls back to + // `day` (see resolveDateFromRow), which is the right FX anchor for an + // as-of-that-day balance — and the convention net worth's history + // already follows. historyResult.rows .filter((/** @type {{ bank_account: string, day: string, currency: string, - balance: string, date: Date, + balance: string, }} */ r) => r.bank_account) .map((/** @type {{ bank_account: string, day: string, currency: string, - balance: string, date: Date, + balance: string, }} */ r) => ({ ...r, amount: toNumber(toDecimal(r.balance)), @@ -163,41 +201,69 @@ export const banksRepository = { 'date' ); + // One row per (account, currency partition): sum the CONVERTED partitions + // back into a single per-account balance, keeping the account metadata from + // whichever partition arrived first (it is identical across them). + /** @type {Map, balance: import('decimal.js').Decimal }>} */ + const accountsByName = new Map(); for (const row of currentBalancesConverted) { - const balance = roundToCents(row.amount_eur); - accounts.push({ - bank_account: row.bank_account, - display_name: row.display_name || row.bank_account, - balance, - // Native-currency drift, matching the hub's badge (accountRepository). - // SQL NULL (no statement balance) → undefined, never null (convention). - drift: row.drift == null ? undefined : roundToCents(toNumber(toDecimal(row.drift))), - // Provenance (WP-A1): anchor_date is already a YYYY-MM-DD string via - // to_char in the lateral; NULL (no stamp) → undefined. - anchor_date: row.anchor_date == null ? undefined : row.anchor_date, - post_anchor_count: row.post_anchor_count == null - ? undefined - : parseInt(row.post_anchor_count, 10), - transaction_count: parseInt(row.transaction_count, 10), - first_transaction: row.first_transaction, - last_transaction: row.last_transaction, - }); - totalNetPosition += balance; + let entry = accountsByName.get(row.bank_account); + if (!entry) { + entry = { + account: { + bank_account: row.bank_account, + display_name: row.display_name || row.bank_account, + balance: 0, + // Native-currency drift, matching the hub's badge (accountRepository). + // SQL NULL (no statement balance) → undefined, never null (convention). + drift: row.drift == null ? undefined : roundToCents(toNumber(toDecimal(row.drift))), + // Provenance (WP-A1): anchor_date is already a YYYY-MM-DD string via + // to_char in the lateral; NULL (no stamp) → undefined. + anchor_date: row.anchor_date == null ? undefined : row.anchor_date, + post_anchor_count: row.post_anchor_count == null + ? undefined + : parseInt(row.post_anchor_count, 10), + transaction_count: parseInt(row.transaction_count, 10), + // DATE columns cross the wire as calendar-day strings: pg reads DATE + // as a local-midnight Date, which JSON-serializes to the PREVIOUS + // day east of UTC (lib/dateFormat.js). + first_transaction: toWireDate(row.first_transaction), + last_transaction: toWireDate(row.last_transaction), + }, + balance: toDecimal(0), + }; + accountsByName.set(row.bank_account, entry); + accounts.push(entry.account); + } + entry.balance = entry.balance.plus(toDecimal(row.amount_eur)); } + let totalNetPositionDec = toDecimal(0); + for (const entry of accountsByName.values()) { + entry.account.balance = roundToCents(entry.balance); + totalNetPositionDec = totalNetPositionDec.plus(toDecimal(entry.account.balance)); + } + const totalNetPosition = toNumber(totalNetPositionDec); - /** @type {Record>} */ - const historyMap = {}; + // Same per-(account, day) fold: a multi-currency account contributes one + // converted partition per currency to the same chart point. + /** @type {Record>} */ + const historyByDay = {}; for (const row of historyConverted) { const key = row.bank_account; - if (!historyMap[key]) historyMap[key] = []; - + if (!historyByDay[key]) historyByDay[key] = new Map(); // toYmd uses local getters for the defensive Date branch (the SQL emits // day via to_char, so this normally passes the string straight through). - historyMap[key].push({ date: toYmd(row.day), balance: roundToCents(row.amount_eur) }); + const day = toYmd(row.day); + const dayTotals = historyByDay[key]; + dayTotals.set(day, (dayTotals.get(day) ?? toDecimal(0)).plus(toDecimal(row.amount_eur))); } - for (const key of Object.keys(historyMap)) { - historyMap[key].sort((a, b) => a.date.localeCompare(b.date)); + /** @type {Record>} */ + const historyMap = {}; + for (const [key, dayTotals] of Object.entries(historyByDay)) { + historyMap[key] = [...dayTotals.entries()] + .sort((a, b) => a[0].localeCompare(b[0])) + .map(([date, balance]) => ({ date, balance: roundToCents(balance) })); } const totalsByDate = new Map(); diff --git a/apps/node-backend/src/repositories/infoRepositoryForecast.js b/apps/node-backend/src/repositories/infoRepositoryForecast.js index 0af3f37e..9e805245 100644 --- a/apps/node-backend/src/repositories/infoRepositoryForecast.js +++ b/apps/node-backend/src/repositories/infoRepositoryForecast.js @@ -11,6 +11,7 @@ import { formatDateToYmd, mapRowsForAmountConversion, batchConvertGroupsWithHistoricalRateFallback, + getIncludeTransfers, } from './infoRepositoryHelpers.js'; import { todayAppDateString } from '../lib/timezone.js'; import { ValidationError } from '../middleware/errorHandler.js'; @@ -31,15 +32,83 @@ function aggregateByDate(rows) { return Array.from(map, ([date, net]) => ({ date, net })).sort((a, b) => a.date.localeCompare(b.date)); } +// Absolute month index for a 'YYYY-MM' key, so month spans are plain integer +// subtraction (year * 12 + zero-based month). +/** + * @param {string} monthKey 'YYYY-MM' + * @returns {number} + */ +function monthIndex(monthKey) { + return Number(monthKey.slice(0, 4)) * 12 + (Number(monthKey.slice(5, 7)) - 1); +} + +/** + * Denominator for the historical monthly average. + * + * The lookback is N *complete, already-elapsed* calendar months ending with the + * month before the current one. Two failure modes bracket the right answer: + * + * - Dividing by "months that happen to carry rows" (the old behaviour) reports + * a single busy month at FULL weight: 240 in one month of a 24-month window + * became an "average" of 240. An elapsed month with no rows is a real + * observation — the user spent nothing — and must count as a zero. + * - Dividing by the whole window unconditionally deflates a short ledger: a + * user who installed three months ago has no data for month -20 because the + * app did not exist for them, not because they spent nothing. Charging them + * 21 phantom zeros would shrink the average line ~8x. + * + * So the divisor is the span from the month the ledger started through the last + * complete month, inclusive — "elapsed months since this ledger has history, + * capped at the lookback window". Empty months inside that span count as zero; + * months before the ledger's first entry are not counted at all. + * + * `ledgerStartMonth` MUST come from an unfiltered probe of `transactions` + * (sqlLedgerStart below), never from the month keys of the filtered result set + * — see the comment on that query for why, and for why a planned row cannot + * establish it. + * + * @param {string|null} ledgerStartMonth 'YYYY-MM' of the ledger's first + * in-window transaction, or null when it has none. + * @param {number} lastCompleteMonthIdx {@link monthIndex} of the last complete month. + * @param {number} windowMonths Lookback length in months. + * @returns {number} Months to divide by; always >= 1. + */ +function countObservedMonths(ledgerStartMonth, lastCompleteMonthIdx, windowMonths) { + if (!ledgerStartMonth) return 1; + const span = lastCompleteMonthIdx - monthIndex(ledgerStartMonth) + 1; + // The clamp guarantees only that the divisor lands in [1, windowMonths] — it + // does NOT reconcile the two clocks feeding it. `ledgerStartMonth` comes from + // Postgres (CURRENT_DATE-anchored window) while `lastCompleteMonthIdx` comes + // from the app timezone (ADR-009); for the couple of hours a month where the + // two disagree on the calendar date across a month boundary, a mid-range span + // can still be one month off. That drift is tracked separately; the clamp is + // here so it can never produce a 0 (divide-by-zero) or an over-window span. + return Math.min(windowMonths, Math.max(1, span)); +} + +/** + * 'YYYY-MM' month key for a pg DATE column (a JS Date via node-postgres, or a + * string on some paths), or null when the column was NULL/absent. + * @param {unknown} value + * @returns {string|null} + */ +function monthKeyFromDbDate(value) { + if (value == null) return null; + const ymd = value instanceof Date ? formatDateToYmd(value) : String(value).slice(0, 10); + return /^\d{4}-\d{2}/.test(ymd) ? ymd.slice(0, 7) : null; +} + // Average, across months, of the running cumulative day-of-month net (SIMP-50). -// `monthDayNet` is { monthKey: { dayOfMonth: net } }. +// `monthDayNet` is { monthKey: { dayOfMonth: net } }. `monthCount` is the +// divisor from countObservedMonths — NOT Object.keys(monthDayNet).length, see +// that function for why. /** * @param {Record>} monthDayNet + * @param {number} monthCount Months to divide by (>= 1). * @returns {Record} day-of-month → average cumulative net */ -function computeAvgCumulativeByDay(monthDayNet) { +function computeAvgCumulativeByDay(monthDayNet, monthCount) { const monthKeys = Object.keys(monthDayNet); - const monthCount = monthKeys.length || 1; /** @type {Record} */ const out = {}; for (const mk of monthKeys) { @@ -84,6 +153,16 @@ export async function getCashflowComparison( const categoryExclusionWhere = excl.whereSql ? `AND ${excl.whereSql}` : ''; const excludeParams = excl.params; + // ADR-083: internal transfers must not inflate cash-flow aggregates unless + // the user opts in via the runtime `includeTransfers` setting. Identical + // predicate and identical setting read as the sibling surfaces rendered on + // the same dashboard (infoRepositoryAverageVsCurrent.js:22-23, + // infoRepositoryMonthly.js:38/194, infoRepositoryStatistics.js:19/129) — + // without it a checking->savings transfer's outflow leg was counted here and + // excluded there, so two cards on one dashboard disagreed on one fixture. + const includeTransfers = await getIncludeTransfers(); + const transferFilter = includeTransfers ? '' : 'AND t.is_transfer = false'; + // Aggregate in SQL per (date, currency) rather than streaming every row to // Node. batchConvertGroupsWithHistoricalRateFallback converts by (currency, // date), and every consumer below re-buckets by date — and rows sharing a @@ -97,6 +176,7 @@ export async function getCashflowComparison( FROM transactions t ${categoryExclusionJoin} WHERE t.is_active = true + ${transferFilter} AND t.date >= date_trunc('month', CURRENT_DATE) - interval '${HISTORY_MONTHS} months' AND t.date < date_trunc('month', CURRENT_DATE) ${categoryExclusionWhere} @@ -109,12 +189,19 @@ export async function getCashflowComparison( FROM transactions t ${categoryExclusionJoin} WHERE t.is_active = true + ${transferFilter} AND t.date >= date_trunc('month', CURRENT_DATE) AND t.date <= CURRENT_DATE ${categoryExclusionWhere} GROUP BY t.date, t.currency `; + // The two planned_transactions overlays below carry NO transfer predicate: + // ADR-083 added `is_transfer` / `transfer_peer_id` to `transactions` only, + // and planned_transactions has no such column. Planned rows are user-authored + // future intents, not reconciled bank legs, so there is no detected pair to + // net out; a user who plans an internal transfer excludes it the pre-ADR-083 + // way, via category/recipient exclusions. const sqlPlannedCurrent = ` SELECT SUM(pt.amount) AS amount, pt.currency, pt.planned_date, EXTRACT(DAY FROM pt.planned_date)::int AS day_of_month @@ -138,12 +225,43 @@ export async function getCashflowComparison( GROUP BY pt.planned_date, pt.currency `; - const [pastResult, currentResult, plannedCurrentResult, plannedHistResult] = await Promise.all([ - query(sqlPast, excludeParams), - query(sqlCurrent, excludeParams), - query(sqlPlannedCurrent), - query(sqlPlannedHist), - ]); + // Ledger start for the historical-average divisor (countObservedMonths). + // Deliberately UNFILTERED — no exclusions, no transfer predicate — and kept + // LAST in the Promise.all so the four data queries keep their call order. + // + // "When did this ledger start having history" is a property of the ledger, + // not of the current view. Deriving it from the month keys of the filtered + // result set let a category/recipient exclusion — or the ADR-083 transfer + // filter itself — empty the oldest months and silently re-base the divisor, + // so toggling an exclusion moved the average line by a factor that had + // nothing to do with the excluded rows. + // + // It also reads `transactions` ONLY. A planned row must never establish the + // start: a recurring plan the auto-linker never matched keeps its original + // past `planned_date` forever, and letting one un-executed row dated 24 + // months back set the divisor deflated a one-month-old ledger's average 24x + // — precisely the "short history" failure this divisor exists to prevent. A + // planned row inside the window still contributes its numerator; it just + // cannot extend the timeline backwards. Both historical series then share + // this one divisor, so the overlay stays commensurable with the base line. + // + // `is_active` applies because a soft-deleted row is not history; the window + // floor applies because a row older than the lookback cannot shorten it. + const sqlLedgerStart = ` + SELECT MIN(t.date) AS first_date + FROM transactions t + WHERE t.is_active = true + AND t.date >= date_trunc('month', CURRENT_DATE) - interval '${HISTORY_MONTHS} months' + `; + + const [pastResult, currentResult, plannedCurrentResult, plannedHistResult, ledgerStartResult] = + await Promise.all([ + query(sqlPast, excludeParams), + query(sqlCurrent, excludeParams), + query(sqlPlannedCurrent), + query(sqlPlannedHist), + query(sqlLedgerStart), + ]); const [pastConverted, currentCashflowConverted, plannedCurrentConverted, plannedHistConverted] = await batchConvertGroupsWithHistoricalRateFallback( @@ -166,8 +284,6 @@ export async function getCashflowComparison( monthDayNet[mk][row.day_of_month] = (monthDayNet[mk][row.day_of_month] || 0) + eur; } - const avgCumulativeByDay = computeAvgCumulativeByDay(monthDayNet); - /** @type {Record} */ const currentDayNet = {}; for (const row of currentCashflowConverted) { @@ -196,7 +312,20 @@ export async function getCashflowComparison( plannedHistMonthDay[mk][row.day_of_month] = (plannedHistMonthDay[mk][row.day_of_month] || 0) + row.amount_eur; } - const avgPlannedCumByDay = computeAvgCumulativeByDay(plannedHistMonthDay); + // ONE divisor for both historical series, taken from the unfiltered ledger + // probe above: a month with transactions but no *planned* rows is a month in + // which the user planned nothing (a real zero), so the overlay must not be + // re-based onto its own shorter span and averaged up. + const lastCompleteMonthIdx = + Number(todayYmd.slice(0, 4)) * 12 + (Number(todayYmd.slice(5, 7)) - 1) - 1; + const observedMonths = countObservedMonths( + monthKeyFromDbDate(ledgerStartResult.rows[0]?.first_date), + lastCompleteMonthIdx, + HISTORY_MONTHS, + ); + + const avgCumulativeByDay = computeAvgCumulativeByDay(monthDayNet, observedMonths); + const avgPlannedCumByDay = computeAvgCumulativeByDay(plannedHistMonthDay, observedMonths); const withoutPlanned = []; const withPlanned = []; @@ -255,6 +384,10 @@ export async function getCashflowForecastData( const categoryExclusionWhere = excl.whereSql ? `AND ${excl.whereSql}` : ''; const excludeParams = excl.params; + // ADR-083 transfer exclusion — see getCashflowComparison for the rationale. + const includeTransfers = await getIncludeTransfers(); + const transferFilter = includeTransfers ? '' : 'AND t.is_transfer = false'; + // GROUP BY (date, currency) in SQL — aggregateByDate re-buckets by date and // conversion is per (currency, date), so this is identical to the old per-row // stream (see getCashflowComparison for the full rationale). @@ -263,6 +396,7 @@ export async function getCashflowForecastData( FROM transactions t ${categoryExclusionJoin} WHERE t.is_active = true + ${transferFilter} AND t.date >= date_trunc('month', CURRENT_DATE) - interval '${historyMonths} months' AND t.date < date_trunc('month', CURRENT_DATE) ${categoryExclusionWhere} @@ -273,11 +407,14 @@ export async function getCashflowForecastData( FROM transactions t ${categoryExclusionJoin} WHERE t.is_active = true + ${transferFilter} AND t.date >= date_trunc('month', CURRENT_DATE) AND t.date <= CURRENT_DATE ${categoryExclusionWhere} GROUP BY t.date, t.currency `; + // No transfer predicate on the planned overlays: planned_transactions has no + // `is_transfer` column (ADR-083 flagged `transactions` only). const sqlPlannedCurrent = ` SELECT SUM(pt.amount) AS amount, pt.currency, pt.planned_date AS date FROM planned_transactions pt @@ -357,6 +494,10 @@ export async function getCashflowForecastDataRolling( const categoryExclusionWhere = excl.whereSql ? `AND ${excl.whereSql}` : ''; const excludeParams = excl.params; + // ADR-083 transfer exclusion — see getCashflowComparison for the rationale. + const includeTransfers = await getIncludeTransfers(); + const transferFilter = includeTransfers ? '' : 'AND t.is_transfer = false'; + // History ends at `today - daysBack` (exclusive) so it never overlaps with currentActual. // GROUP BY (date, currency) — identical to the old per-row stream (aggregateByDate re-buckets by date). const sqlHistory = ` @@ -364,6 +505,7 @@ export async function getCashflowForecastDataRolling( FROM transactions t ${categoryExclusionJoin} WHERE t.is_active = true + ${transferFilter} AND t.date >= (CURRENT_DATE - interval '${daysBack} days') - interval '${historyMonths} months' AND t.date < (CURRENT_DATE - interval '${daysBack} days') ${categoryExclusionWhere} @@ -374,11 +516,13 @@ export async function getCashflowForecastDataRolling( FROM transactions t ${categoryExclusionJoin} WHERE t.is_active = true + ${transferFilter} AND t.date >= (CURRENT_DATE - interval '${daysBack} days') AND t.date <= CURRENT_DATE ${categoryExclusionWhere} GROUP BY t.date, t.currency `; + // No transfer predicate — planned_transactions has no `is_transfer` column. const sqlPlannedFuture = ` SELECT SUM(pt.amount) AS amount, pt.currency, pt.planned_date AS date FROM planned_transactions pt @@ -436,6 +580,13 @@ export async function getCashflowForecastDataByCategory( const excludeParams = excl.params; const exclusionWhere = excl.whereSql ? `AND ${excl.whereSql}` : ''; + // ADR-083 transfer exclusion — see getCashflowComparison for the rationale. + // This matters doubly here: the category breakdown attributes a transfer leg + // to whatever category the account's recipient defaults to, inventing spend + // in a category the user never spent in. + const includeTransfers = await getIncludeTransfers(); + const transferFilter = includeTransfers ? '' : 'AND t.is_transfer = false'; + // Aggregate per (date, currency, effective category) in SQL — aggregateByDateAndCategory // re-buckets by (date, category) and conversion is per (currency, date), so SUM-then-convert // is identical to the old per-row stream. @@ -463,6 +614,7 @@ export async function getCashflowForecastDataByCategory( SELECT ${selectCols} FROM transactions t ${joins} WHERE t.is_active = true + ${transferFilter} AND t.date >= date_trunc('month', CURRENT_DATE) - interval '${historyMonths} months' AND t.date < date_trunc('month', CURRENT_DATE) ${exclusionWhere} @@ -472,6 +624,7 @@ export async function getCashflowForecastDataByCategory( SELECT ${selectCols} FROM transactions t ${joins} WHERE t.is_active = true + ${transferFilter} AND t.date >= date_trunc('month', CURRENT_DATE) AND t.date <= CURRENT_DATE ${exclusionWhere} diff --git a/apps/node-backend/src/repositories/infoRepositoryMonthly.js b/apps/node-backend/src/repositories/infoRepositoryMonthly.js index df2bfa91..dbb3733d 100644 --- a/apps/node-backend/src/repositories/infoRepositoryMonthly.js +++ b/apps/node-backend/src/repositories/infoRepositoryMonthly.js @@ -181,7 +181,12 @@ export async function getMonthlyFinancialSummary( t.amount, t.currency, t.date, - COALESCE(t.category_id, r.default_category_id) AS effective_category_id + -- Canonical 3-level effective category (own → recipient default → + -- PRIMARY recipient's default), matching transactionRepository and the + -- mv_monthly_summary definition. The exclusion clauses above already + -- resolve 3 levels via buildExclusionClauses, so this column keeps the + -- CTE's own resolution consistent with them. + COALESCE(t.category_id, r.default_category_id, pr.default_category_id) AS effective_category_id FROM transactions t LEFT JOIN recipients r ON t.recipient_id = r.id LEFT JOIN recipients pr ON r.primary_recipient_id = pr.id diff --git a/apps/node-backend/src/repositories/infoRepositoryNetWorth.js b/apps/node-backend/src/repositories/infoRepositoryNetWorth.js index 8f37b738..309736cb 100644 --- a/apps/node-backend/src/repositories/infoRepositoryNetWorth.js +++ b/apps/node-backend/src/repositories/infoRepositoryNetWorth.js @@ -4,7 +4,10 @@ import { query } from '../database/connection.js'; import { logger } from '../config/logger.js'; -import { COMPUTED_BALANCE_LATERAL } from './accountBalanceSql.js'; +import { + COMPUTED_BALANCE_LATERAL, + computedBalanceSeriesCtes, +} from './accountBalanceSql.js'; import { toNumber, toDecimal } from '../lib/money.js'; import { todayAppDateString } from '../lib/timezone.js'; import { @@ -25,15 +28,20 @@ export const netWorthRepository = { * Bank balances are still derived live from the transactions table. * No network calls — all data from the database. * - * The liquid/liability *history* series is stamp-based (latest stamped - * `transactions.balance` ≤ each day), but the **current** point — headline, - * last chart point, latest table row — is overridden with the unified - * anchor+delta computed balance (`COMPUTED_BALANCE_LATERAL`, ADR-094 / - * WP-A1), the same single definition the accounts hub and dashboard widget - * consume. The naive stamped walk it replaced silently - * dropped manual-only (never-stamped) in-net-worth accounts from the - * headline and froze stamped accounts at their last imported statement - * figure. + * The **current** point — headline, last chart point, latest table row — is + * the unified anchor+delta computed balance (`COMPUTED_BALANCE_LATERAL`, + * ADR-094 / WP-A1), the same single definition the accounts hub and dashboard + * widget consume. The naive stamped read it replaced silently dropped + * manual-only (never-stamped) in-net-worth accounts from the headline and + * froze stamped accounts at their last imported statement figure. + * + * The liquid/liability *history* series applies that same definition to every + * earlier day (`computedBalanceSeriesCtes`). It used to be stamp-based, so a + * manual-only account showed up in the last point only and the chart stepped + * up overnight — a step the monthly-change figure then reported as a real + * gain. A day before an account's first active row still yields no row for it + * (it contributes 0 to that day's total, and its first known balance is never + * carried backwards). * * @param {string} [targetCurrency] * @param {{ liveInvestments?: number }} [opts] @@ -87,9 +95,8 @@ export const netWorthRepository = { // Postgres CURRENT_DATE follows the server timezone, not the app's. const todayYmd = todayAppDateString(); - // History walk (stamp-based, per WP-A1 decision) and the unified - // current-point balances (anchor+delta lateral) are independent — run in - // parallel. + // History walk and the unified current-point balances (the same + // anchor+delta definition, unbounded) are independent — run in parallel. const [bankHistoryResult, currentBalancesResult] = await Promise.all([ query(` WITH bounds AS ( @@ -104,38 +111,37 @@ export const netWorthRepository = { -- tracking-only account (in_net_worth=false) does not contribute. -- is_liability splits negative debt balances (ADR-092) out of the -- "liquid assets" bucket so a mortgage is not counted as liquid cash. - -- The stamped-only probe below serves the HISTORY series only (WP-A1 - -- decision); the current point is overridden after the walk with the - -- unified computed-balance lateral. SELECT a.id AS account_id, a.name AS bank_account, - (a.type = 'liability') AS is_liability + (a.type = 'liability') AS is_liability, + a.currency AS account_currency FROM accounts a WHERE a.in_net_worth = true AND a.id IN ( SELECT t.account_id FROM transactions t WHERE t.is_active = true AND t.account_id IS NOT NULL ) - ) + ), + -- The walk resolves each day with the SAME unstamped-tolerant + -- anchor+delta definition the current point uses, bounded at that day. + -- The stamped-only probe it replaces (plus a WHERE lb.balance IS NOT + -- NULL gate) hid never-stamped accounts from EVERY point except the last + -- one — the current-point override below then added them back in one go, + -- so the chart stepped up overnight and reported it as a monthly gain. + -- Cross-currency Σ (not per-currency) on purpose: it must mirror the + -- current-point lateral below exactly, or the step returns for + -- multi-currency accounts. + ${computedBalanceSeriesCtes()} + -- The currency mirrors the current-point query: the latest active row's + -- (as of the day), falling back to the account's own. SELECT - to_char(d.day, 'YYYY-MM-DD') AS day, + to_char(s.day, 'YYYY-MM-DD') AS day, a.bank_account, a.is_liability, - COALESCE(lb.currency, 'EUR') AS currency, - lb.balance - FROM days d - CROSS JOIN account_list a - LEFT JOIN LATERAL ( - SELECT t.currency, t.balance - FROM transactions t - WHERE t.is_active = true - AND t.account_id = a.account_id - AND t.balance IS NOT NULL - AND t.date <= d.day - ORDER BY t.date DESC, t.id DESC - LIMIT 1 - ) lb ON true - WHERE lb.balance IS NOT NULL - ORDER BY d.day, a.account_id + COALESCE(s.row_currency, a.account_currency, 'EUR') AS currency, + s.balance + FROM balance_series s + JOIN account_list a ON a.account_id = s.account_id + ORDER BY s.day, s.account_id `, [firstDataDateYmd, todayYmd]), // Unified current balance per in-net-worth account (WP-A1): the shared // anchor+delta lateral, with NO `balance IS NOT NULL` population gate — @@ -184,6 +190,34 @@ export const netWorthRepository = { ]); let bankHistoryConverted = bankHistoryConvertedInitial; + // Reached whenever the walk produced no rows at all: no in-net-worth + // account owns an active row. That covers the un-migrated ledger this was + // written for (transactions still carrying a NULL account_id) but ALSO the + // case where every account with activity is in_net_worth=false. The walk + // itself no longer needs rescuing when nothing is stamped, since an + // unstamped account resolves to its running Σ(amount) day by day, which is + // exactly what this fallback computes. + // + // `NOT_TRACKING_ONLY` below is what keeps the two populations apart. The + // fallback used to sum EVERY active transaction with no account / + // in_net_worth predicate at all, so a ledger whose only active accounts are + // in_net_worth=false reported THEIR running total as net worth (measured: + // liquid −143.25 where 0 is correct). It cannot simply inner-join + // `accounts` either — the un-migrated ledger it exists for has rows with a + // `bank_account` string (or nothing) and NO accounts row behind them, and + // an inner join would drop exactly the rows the fallback is here to count. + // + // So the predicate excludes only rows POSITIVELY attributed to an + // in_net_worth=false account, mirroring the walk's own resolution, which + // reads `transactions.account_id` and nothing else (`account_list` above). + // Rows with a NULL account_id stay counted: they are unattributed, and + // nothing says they belong to a tracking-only account. An all-tracking + // ledger therefore yields no rows at all → every day is 0. + const NOT_TRACKING_ONLY = ` + AND NOT EXISTS ( + SELECT 1 FROM accounts a + WHERE a.id = t.account_id AND a.in_net_worth = false + )`; if (bankHistoryConverted.length === 0) { logger.debug('Net worth account balance history empty; using transaction flow fallback', { targetCurrency, @@ -204,6 +238,7 @@ export const netWorthRepository = { WHERE t.is_active = true AND t.date >= (SELECT start_date FROM bounds) AND t.date <= (SELECT end_date FROM bounds) + ${NOT_TRACKING_ONLY} ), tx_daily AS ( SELECT @@ -214,6 +249,7 @@ export const netWorthRepository = { WHERE t.is_active = true AND t.date >= (SELECT start_date FROM bounds) AND t.date <= (SELECT end_date FROM bounds) + ${NOT_TRACKING_ONLY} GROUP BY t.date::date, COALESCE(t.currency, 'EUR') ), tx_series AS ( @@ -292,14 +328,16 @@ export const netWorthRepository = { const sanitizedSnapshots = sanitizeIsolatedDailyInvestmentSpikes(snapshots); - // WP-A1: override the *current* point's liquid/liability figures with the - // unified computed-balance definition (see the method doc). Only the last - // point moves — the history series deliberately stays stamp-based — so a - // manual-only account or post-anchor manual activity can introduce a step - // between the penultimate (stamped) and latest (computed) points. Skipped - // when the accounts query returned nothing (no in-net-worth accounts, e.g. - // an un-migrated ledger running on the transaction-flow fallback), keeping - // the walk/fallback-derived point instead. + // WP-A1: set the *current* point's liquid/liability figures from the + // unified computed-balance definition (see the method doc). Now that the + // walk resolves every earlier day with that same definition this is + // continuous with the point before it, so no step is introduced; it still + // matters because the walk is bounded at each day (a future-dated row + // counts here first) and because the population is every in-net-worth + // account, including ones with no active rows at all. Skipped when the + // accounts query returned nothing (no in-net-worth accounts, e.g. an + // un-migrated ledger running on the transaction-flow fallback), keeping the + // walk/fallback-derived point instead. if (currentBalancesConverted.length > 0 && sanitizedSnapshots.length > 0) { let liquidNow = toDecimal(0); let liabilitiesNow = toDecimal(0); diff --git a/apps/node-backend/src/repositories/infoRepositoryPlanned.js b/apps/node-backend/src/repositories/infoRepositoryPlanned.js index 008a0ff9..1c85932c 100644 --- a/apps/node-backend/src/repositories/infoRepositoryPlanned.js +++ b/apps/node-backend/src/repositories/infoRepositoryPlanned.js @@ -4,8 +4,7 @@ import { query } from '../database/connection.js'; import { convertRowsToEur } from '../services/currency/currencyConversionService.js'; -import { toAppTz, toAppDateString } from '../lib/timezone.js'; -import { calculateNextDate } from '../lib/calculations/recurrence.js'; +import { todayAppDateString, firstOfMonthYmd, addDaysYmd } from '../lib/timezone.js'; import { addAll, toDecimal, toNumber } from '../lib/money.js'; import { roundToCents, @@ -17,6 +16,22 @@ const MAX_OCCURRENCES = 120; // guard against infinite loops on tiny intervals const MS_PER_DAY = 24 * 60 * 60 * 1000; +const YMD_RE = /^\d{4}-\d{2}-\d{2}$/; + +/** + * Calendar day of a planned_date as a 'YYYY-MM-DD' string. + * + * pg reads a DATE column as a Date at *server-local* midnight, so the calendar + * day must be recovered with local getters (lib/dateFormat's rule); string + * values are already calendar days and are sliced. + * + * @param {Date|string} value + * @returns {string} + */ +function plannedDateToYmd(value) { + return value instanceof Date ? formatDateToYmd(value) : String(value).slice(0, 10); +} + /** * Fixed day-step for the day-based recurrence patterns, or null for the * month-based ones (monthly/quarterly/yearly), whose step length varies. @@ -37,20 +52,98 @@ function dayStepForPattern(pattern) { return null; } +/** + * Month-step for a month-based pattern, or null for anything else. + * + * @param {string|null|undefined} pattern + * @returns {number|null} + */ +function monthStepForPattern(pattern) { + const p = String(pattern || '').toLowerCase().trim(); + if (p === 'monthly') return 1; + if (p === 'quarterly') return 3; + if (p === 'yearly') return 12; + return null; +} + +/** + * Whole days from `fromYmd` to `toYmd` (negative if `toYmd` is earlier). Pure + * calendar math on the parsed components — host-timezone independent. + * + * @param {string} fromYmd 'YYYY-MM-DD' + * @param {string} toYmd 'YYYY-MM-DD' + * @returns {number} + */ +function diffDaysYmd(fromYmd, toYmd) { + const [fy, fm, fd] = fromYmd.split('-').map(Number); + const [ty, tm, td] = toYmd.split('-').map(Number); + return Math.round((Date.UTC(ty, tm - 1, td) - Date.UTC(fy, fm - 1, fd)) / MS_PER_DAY); +} + +/** + * Add `months` to a 'YYYY-MM-DD' string, clamping the day to the last day of + * the target month when it doesn't exist there (Jan 31 +1 month → Feb 28). + * + * String twin of recurrence.js's addMonthsClampedInAppTz, which clamps on the + * APP_TIMEZONE wall-clock day — for a start-of-day occurrence that is exactly + * this string's day, so the landing day is identical without the Date round + * trip. Applied one step at a time by the walk below, so the clamp compounds + * the same way sequential calculateNextDate hops do (Jan 31 → Feb 28 → Mar 28, + * NOT back to Mar 31). + * + * @param {string} ymd 'YYYY-MM-DD' + * @param {number} months + * @returns {string} + */ +function addMonthsClampedYmd(ymd, months) { + const day = parseInt(ymd.slice(8, 10), 10); + const firstOfTarget = firstOfMonthYmd(ymd, months); + const lastDayOfTarget = parseInt(addDaysYmd(firstOfMonthYmd(ymd, months + 1), -1).slice(8, 10), 10); + return `${firstOfTarget.slice(0, 8)}${String(Math.min(day, lastDayOfTarget)).padStart(2, '0')}`; +} + +/** + * The occurrence after `ymd` for `pattern`, or null for a pattern that can't be + * advanced. Calendar-string twin of calculateNextDate (same pattern grammar: + * daily/weekly/biweekly/"every N days" + monthly/quarterly/yearly). + * + * @param {string} ymd 'YYYY-MM-DD' + * @param {string} pattern + * @returns {string|null} + */ +function nextOccurrenceYmd(ymd, pattern) { + const stepDays = dayStepForPattern(pattern); + if (stepDays) return addDaysYmd(ymd, stepDays); + const stepMonths = monthStepForPattern(pattern); + if (stepMonths) return addMonthsClampedYmd(ymd, stepMonths); + return null; +} + /** * Walk a recurring planned transaction forward from its stored date, emitting * each occurrence (as a YYYY-MM-DD string in APP_TIMEZONE) that falls within - * [startYmd, endYmd). Returns [] for a pattern calculateNextDate can't advance. + * [startYmd, endYmd). Returns [] for a pattern that can't be advanced. + * + * Pure calendar-string arithmetic (ADR-009 helpers), so an occurrence lands on + * the same day on every host. It used to walk `Date`s instead: the base was the + * pg DATE read at SERVER-LOCAL midnight, each occurrence was rendered with + * toAppDateString (APP_TIMEZONE) and the fast-forward compared that instant + * against `Date.UTC(...)` of the window start — three different calendars. On a + * host east of APP_TIMEZONE (e.g. TZ=Asia/Tokyo with the default + * Europe/Brussels) local midnight is still the previous day in the app zone, so + * every occurrence shifted a day back; a weekly row anchored on the 1st then + * missed the window's first day and landed 6 days off for the whole month, and + * a monthly row shifted onto a month-end triggered a spurious clamp cascade. * * Day-stepped patterns fast-forward to just before the window in one jump: a * stale fast-cadence row (e.g. daily, last advanced >120 days ago) used to * exhaust MAX_OCCURRENCES before reaching next month and silently vanish from - * the forecast. The jump is whole-step ms arithmetic — exactly equivalent to N - * sequential calculateNextDate hops for these patterns — landing at least one - * step before the window so the boundary occurrence is never skipped. - * Month-based patterns keep the plain walk (120 monthly hops = 10 years, far - * beyond any realistic staleness, and bulk month jumps would change the - * sequential month-end clamping semantics). + * the forecast. The jump is a whole number of steps — exactly equivalent to N + * sequential hops for these patterns — landing at least one step before the + * window so the boundary occurrence is never skipped. Month-based patterns keep + * the plain walk (120 monthly hops = 10 years, far beyond any realistic + * staleness, and bulk month jumps would change the sequential month-end + * clamping semantics). * * @param {Date|string} plannedDate DATE column — a `Date` from pg. * @param {string} pattern @@ -62,27 +155,23 @@ function expandRecurringOccurrences(plannedDate, pattern, startYmd, endYmd) { /** @type {string[]} */ const ymds = []; if (!pattern) return ymds; - let current = plannedDate instanceof Date ? new Date(plannedDate.getTime()) : new Date(plannedDate); - if (Number.isNaN(current.getTime())) return ymds; + let current = plannedDateToYmd(plannedDate); + if (!YMD_RE.test(current)) return ymds; const stepDays = dayStepForPattern(pattern); if (stepDays) { - const [y, m, d] = startYmd.split('-').map((s) => parseInt(s, 10)); - const windowStartMs = Date.UTC(y, m - 1, d); - const stepMs = stepDays * MS_PER_DAY; - const deficitMs = windowStartMs - current.getTime(); - if (deficitMs > stepMs) { - const hops = Math.floor(deficitMs / stepMs) - 1; - if (hops > 0) current = new Date(current.getTime() + hops * stepMs); + const deficitDays = diffDaysYmd(current, startYmd); + if (deficitDays > stepDays) { + const hops = Math.floor(deficitDays / stepDays) - 1; + if (hops > 0) current = addDaysYmd(current, hops * stepDays); } } for (let i = 0; i < MAX_OCCURRENCES; i++) { - const ymd = toAppDateString(current); - if (ymd >= endYmd) break; - if (ymd >= startYmd) ymds.push(ymd); - const next = calculateNextDate(current, pattern); - if (!next || next.getTime() <= current.getTime()) break; + if (current >= endYmd) break; + if (current >= startYmd) ymds.push(current); + const next = nextOccurrenceYmd(current, pattern); + if (!next || next <= current) break; current = next; } return ymds; @@ -90,25 +179,44 @@ function expandRecurringOccurrences(plannedDate, pattern, startYmd, endYmd) { export const plannedRepository = { async getPlannedExpensesNextMonth(targetCurrency = 'EUR') { - // Anchor the month window to today's calendar month in APP_TIMEZONE. - // Constructed LOCALLY to pair with formatDateToYmd's local extraction — - // a UTC-constructed boundary would render as the last day of the previous - // month on any server west of UTC. (today.month is 1-based, so passing it - // as the 0-based month argument yields the first of the NEXT month.) - const today = toAppTz(new Date()); - const nextMonth = new Date(today.year, today.month, 1); - const monthAfter = new Date(today.year, today.month + 1, 1); - const lastDay = new Date(monthAfter.getTime() - 1); + // Anchor the month window to today's calendar month in APP_TIMEZONE and + // keep it as YYYY-MM-DD strings throughout (ADR-009 helpers: pure calendar + // math, host-timezone independent). + // + // The boundaries used to be `Date`s built with LOCAL getters + // (`new Date(today.year, today.month, 1)`) but read back with UTC ones + // (`getUTCMonth()`/`getUTCFullYear()`) for the `month`/`year` fields. East + // of UTC — including the default APP_TIMEZONE=Europe/Brussels — local + // midnight of the 1st is still the PREVIOUS month in UTC, so the response + // named one month while `period_start` (formatted with local getters) + // named the next: month=7 alongside period_start='2026-08-01'. + const todayYmd = todayAppDateString(); + const startYmd = firstOfMonthYmd(todayYmd, 1); // first of next month + const endYmd = firstOfMonthYmd(todayYmd, 2); // first of the month after (exclusive) + const lastDayYmd = addDaysYmd(endYmd, -1); + const [nextMonthYear, nextMonthMonth] = startYmd.split('-').map((s) => parseInt(s, 10)); const sql = ` SELECT pt.*, r.name AS recipient_name, + -- Same 3-level resolution as plannedTransactionRepository (and as + -- transactionRepository's CATEGORY_NAME_SQL): own (c) → recipient + -- default (rc) → PRIMARY recipient default (pc). This site + -- resolved c ONLY, joining neither rc nor pc, so a planned row + -- that inherits its category from the recipient reported + -- category_name: null here while its sibling repository — reading + -- the same table for the same row — categorised it. CASE WHEN c.id IS NOT NULL THEN c.general || ':' || c.detail + WHEN rc.id IS NOT NULL THEN rc.general || ':' || rc.detail + WHEN pc.id IS NOT NULL THEN pc.general || ':' || pc.detail ELSE NULL END AS category_name FROM planned_transactions pt LEFT JOIN recipients r ON pt.recipient_id = r.id + LEFT JOIN recipients pr ON r.primary_recipient_id = pr.id LEFT JOIN categories c ON pt.category_id = c.id + LEFT JOIN categories rc ON r.default_category_id = rc.id + LEFT JOIN categories pc ON pr.default_category_id = pc.id WHERE pt.is_active = true AND pt.is_executed = false AND ( @@ -118,19 +226,13 @@ export const plannedRepository = { ORDER BY pt.planned_date ASC `; - const result = await query(sql, [ - formatDateToYmd(nextMonth), - formatDateToYmd(monthAfter), - ]); + const result = await query(sql, [startYmd, endYmd]); const plannedConverted = await convertRowsToEur( mapRowsForAmountConversion(result.rows, 'amount', false), targetCurrency ); - const startYmd = formatDateToYmd(nextMonth); - const endYmd = formatDateToYmd(monthAfter); - /** * `total_income` / `total_expenses` are deliberately `any`: they hold * Decimals during the accumulation loop and are collapsed IN PLACE to @@ -186,9 +288,7 @@ export const plannedRepository = { pushOccurrence(ymd, row, eur); } } else { - const dateStr = row.planned_date instanceof Date - ? formatDateToYmd(row.planned_date) - : String(row.planned_date).slice(0, 10); + const dateStr = plannedDateToYmd(row.planned_date); // Non-recurring (or pattern-less) rows only count inside the window. if (dateStr >= startYmd && dateStr < endYmd) pushOccurrence(dateStr, row, eur); } @@ -211,10 +311,10 @@ export const plannedRepository = { } return { - month: nextMonth.getUTCMonth() + 1, - year: nextMonth.getUTCFullYear(), - period_start: formatDateToYmd(nextMonth), - period_end: formatDateToYmd(lastDay), + month: nextMonthMonth, + year: nextMonthYear, + period_start: startYmd, + period_end: lastDayYmd, daily_data: dailyData, summary: { total_income: roundToCents(totalIncome), diff --git a/apps/node-backend/src/repositories/infoRepositoryRecipients.js b/apps/node-backend/src/repositories/infoRepositoryRecipients.js index 108ccbf3..ea2d0919 100644 --- a/apps/node-backend/src/repositories/infoRepositoryRecipients.js +++ b/apps/node-backend/src/repositories/infoRepositoryRecipients.js @@ -42,10 +42,20 @@ export const recipientInsightsRepository = { const params = excl.params; const exclusionWhere = excl.whereSql ? `AND ${excl.whereSql}` : ''; + // Grouped per (recipient, DATE, currency) — the extra `t.date` key exists so + // the conversion below can use each row's OWN date rate, exactly as + // getRecipientByYear (:212) and getRecipientPivot (:324) do. Aggregating + // per recipient first and converting the SUM afterwards would have to pick a + // single rate for a multi-date total, which is what made Top merchants + // report a different EUR figure than by-year/pivot for the same recipient. + // amount < 0 is pinned, so ABS distributes over the same-sign SUM and + // SUM-then-convert per date is identical to converting each row; tx_count + // and the MIN/MAX date bounds are re-reduced per recipient in JS below. const topRawResult = await query(` SELECT COALESCE(pr.name, r.name) AS recipient_name, COALESCE(pr.id, r.id) AS recipient_id, + t.date, t.currency, SUM(ABS(t.amount)) AS total_abs_amount, COUNT(*) AS tx_count, @@ -58,12 +68,16 @@ export const recipientInsightsRepository = { AND t.is_active = true AND t.is_transfer = false ${exclusionWhere} - GROUP BY COALESCE(pr.id, r.id), COALESCE(pr.name, r.name), t.currency + GROUP BY COALESCE(pr.id, r.id), COALESCE(pr.name, r.name), t.date, t.currency `, params); + // Historical per-date rates, matching getRecipientByYear / getRecipientPivot. + // Converting at the LATEST rate here made one 2024 USD purchase read 90 in + // Top merchants and 25 in the by-year / pivot views of the same recipient. const topConverted = await convertRowsToEur( mapRowsForAmountConversion(topRawResult.rows, 'total_abs_amount', false), - targetCurrency + targetCurrency, + { useHistoricalRatesByDate: true, dateField: 'date' } ); /** @@ -109,11 +123,23 @@ export const recipientInsightsRepository = { avgAmount: roundToCents(r.totalSpend / r.transactionCount), })); + // Grouped per (recipient, period, DATE, currency) for the same reason top + // merchants above is: the conversion below needs each row's OWN date rate. + // Aggregating a whole month first and converting the SUM afterwards has to + // pick ONE rate for the month, and passing no options at all picked the + // CURRENT `is_latest` rate for BOTH months — so a rate move between the two + // compared months was invisible (the change percent measured spend only) + // and MoM's EUR figures disagreed with the now-historical top-merchants / + // by-year / pivot surfaces on the same page. amount < 0 is pinned, so ABS + // distributes over the same-sign SUM and SUM-then-convert per date is + // identical to converting each row; the per-(recipient, period) totals are + // re-reduced in JS below. const momRawResult = await query(` SELECT COALESCE(pr.id, r.id) AS recipient_id, COALESCE(pr.name, r.name) AS recipient_name, TO_CHAR(t.date, 'YYYY-MM') AS period, + t.date, t.currency, SUM(ABS(t.amount)) AS abs_amount FROM transactions t @@ -132,12 +158,14 @@ export const recipientInsightsRepository = { + (CURRENT_DATE - DATE_TRUNC('month', CURRENT_DATE)::date) ) ${exclusionWhere} - GROUP BY COALESCE(pr.id, r.id), COALESCE(pr.name, r.name), TO_CHAR(t.date, 'YYYY-MM'), t.currency + GROUP BY COALESCE(pr.id, r.id), COALESCE(pr.name, r.name), TO_CHAR(t.date, 'YYYY-MM'), t.date, t.currency `, params); + // Historical per-date rates, matching top merchants / by-year / pivot. const momConverted = await convertRowsToEur( mapRowsForAmountConversion(momRawResult.rows, 'abs_amount', false), - targetCurrency + targetCurrency, + { useHistoricalRatesByDate: true, dateField: 'date' } ); // Derive the current / previous month keys in the database so they match @@ -157,8 +185,14 @@ export const recipientInsightsRepository = { const rid = row.recipient_id; const eur = row.amount_eur; if (!momAgg[rid]) momAgg[rid] = { name: row.recipient_name, current: 0, previous: 0 }; - if (row.period === currentPeriod) momAgg[rid].current += eur; - else if (row.period === prevPeriod) momAgg[rid].previous += eur; + // Decimal accumulation (money-hygiene), as in the top-merchants reduction + // above: per-date conversion means a month is now many converted rows, + // and native `+=` over them drifts sub-cent before the final round. + if (row.period === currentPeriod) { + momAgg[rid].current = toNumber(toDecimal(momAgg[rid].current).plus(toDecimal(eur))); + } else if (row.period === prevPeriod) { + momAgg[rid].previous = toNumber(toDecimal(momAgg[rid].previous).plus(toDecimal(eur))); + } } const monthOverMonth = Object.entries(momAgg) diff --git a/apps/node-backend/src/repositories/plannedTransactionRepository.js b/apps/node-backend/src/repositories/plannedTransactionRepository.js index 0ac63c1c..3272719c 100644 --- a/apps/node-backend/src/repositories/plannedTransactionRepository.js +++ b/apps/node-backend/src/repositories/plannedTransactionRepository.js @@ -36,17 +36,32 @@ import { buildSetClauses } from '../lib/sqlClauses.js'; // getDueSoon, getForForecast and the update() RETURNING wrapper all read the // same recipient_name + resolved category_name shape over the same joins; // keeping the block in one place avoids five-way drift. -const PLANNED_SELECT_FIELDS = `pt.*, - r.name AS recipient_name, - CASE +// planned_transactions carries its own `recipient_id` + `category_id`, so the +// same 3-level resolution the transactions list uses applies verbatim here: +// own (c) → recipient default (rc) → PRIMARY recipient default (pc), mirroring +// COALESCE(pt.category_id, r.default_category_id, pr.default_category_id). +// This used to stop at `rc` (no `pc` branch, no `pr` join), so a planned row +// booked against an ALIAS recipient that inherits from its primary showed no +// category at all while the equivalent transaction showed one. +// NB: `category_id` in the projection is deliberately still pt's OWN stored +// column (that is the editable field the PATCH round-trips); `category_name` is +// the resolved DISPLAY name. They only coincide when pt.category_id is set. +const PLANNED_CATEGORY_NAME_SQL = `CASE WHEN c.id IS NOT NULL THEN c.general || ':' || c.detail WHEN rc.id IS NOT NULL THEN rc.general || ':' || rc.detail + WHEN pc.id IS NOT NULL THEN pc.general || ':' || pc.detail ELSE NULL - END AS category_name`; + END`; + +const PLANNED_SELECT_FIELDS = `pt.*, + r.name AS recipient_name, + ${PLANNED_CATEGORY_NAME_SQL} AS category_name`; const PLANNED_JOINS = `LEFT JOIN recipients r ON pt.recipient_id = r.id + LEFT JOIN recipients pr ON r.primary_recipient_id = pr.id LEFT JOIN categories c ON pt.category_id = c.id - LEFT JOIN categories rc ON r.default_category_id = rc.id`; + LEFT JOIN categories rc ON r.default_category_id = rc.id + LEFT JOIN categories pc ON pr.default_category_id = pc.id`; /** * Attach the executions, loan_schedule and tags sub-collections to a hydrated @@ -126,10 +141,11 @@ function buildPlannedTransactionWhereClause({ pt.comment ILIKE $${paramIdx} OR pt.bank_account ILIKE $${paramIdx} OR r.name ILIKE $${paramIdx} OR - c.general ILIKE $${paramIdx} OR - c.detail ILIKE $${paramIdx} OR - rc.general ILIKE $${paramIdx} OR - rc.detail ILIKE $${paramIdx} + -- Match the RESOLVED label the row displays, not each candidate level in + -- turn. ORing c/rc separately both missed rows categorised through the + -- primary recipient (no pc term at all) and matched rows whose own + -- category_id overrode the recipient default the term hit. + ${PLANNED_CATEGORY_NAME_SQL} ILIKE $${paramIdx} )`; params.push(sp); } @@ -614,11 +630,7 @@ export const plannedTransactionRepository = { SELECT pt.id, pt.planned_date, pt.amount, pt.currency, pt.memo, pt.is_recurring, pt.recurrence_pattern, r.name AS recipient_name, - CASE - WHEN c.id IS NOT NULL THEN c.general || ':' || c.detail - WHEN rc.id IS NOT NULL THEN rc.general || ':' || rc.detail - ELSE NULL - END AS category_name + ${PLANNED_CATEGORY_NAME_SQL} AS category_name FROM planned_transactions pt ${PLANNED_JOINS} WHERE pt.is_active = true diff --git a/apps/node-backend/src/repositories/splitRepository.js b/apps/node-backend/src/repositories/splitRepository.js index 1c681081..24e57f6e 100644 --- a/apps/node-backend/src/repositories/splitRepository.js +++ b/apps/node-backend/src/repositories/splitRepository.js @@ -371,10 +371,17 @@ export const splitRepository = { (ts.amount - COALESCE(sp_agg.paid, 0)) AS amount, t.currency, t.balance, + -- Same branch order as transactionRepository's CATEGORY_NAME_SQL: + -- own (c) → transaction recipient default (rc) → primary-recipient + -- default (pc), mirroring COALESCE(t.category_id, + -- tr.default_category_id, pr.default_category_id). It used to test pc + -- before rc, so an ALIAS recipient with its own default under a + -- differently-defaulted PRIMARY exported the primary's category name + -- while the transactions list showed the alias's. CASE WHEN c.id IS NOT NULL THEN c.general || ':' || c.detail - WHEN pc.id IS NOT NULL THEN pc.general || ':' || pc.detail WHEN rc.id IS NOT NULL THEN rc.general || ':' || rc.detail + WHEN pc.id IS NOT NULL THEN pc.general || ':' || pc.detail ELSE '' END AS category_name, t.comment diff --git a/apps/node-backend/src/repositories/transactionRepository.js b/apps/node-backend/src/repositories/transactionRepository.js index 2c0aa099..fdb5938d 100644 --- a/apps/node-backend/src/repositories/transactionRepository.js +++ b/apps/node-backend/src/repositories/transactionRepository.js @@ -64,10 +64,19 @@ const TRANSACTION_JOINS = ` // getById/create paths resolve categories identically to the list paths — an // alias recipient must not show categorized in lists but uncategorized on GET. const EFFECTIVE_CATEGORY_ID_SQL = 'COALESCE(t.category_id, r.default_category_id, pr.default_category_id)'; +// Displayed name for exactly the category EFFECTIVE_CATEGORY_ID_SQL resolves, +// so the two can never denote different categories. The branch order therefore +// mirrors that COALESCE: own (c) → recipient default (rc) → primary-recipient +// default (pc). It used to test `pc` before `rc`, so an ALIAS recipient with its +// own default whose PRIMARY carried a different one reported the alias's +// category id next to the primary's category name — and the aggregation +// surfaces, which follow the id, then disagreed with this list's label. +// (The same CASE is inlined at the sort-column map and in getAll / +// getAllWithCount below; all four copies share this order.) const CATEGORY_NAME_SQL = `CASE WHEN c.id IS NOT NULL THEN c.general || ':' || c.detail - WHEN pc.id IS NOT NULL THEN pc.general || ':' || pc.detail WHEN rc.id IS NOT NULL THEN rc.general || ':' || rc.detail + WHEN pc.id IS NOT NULL THEN pc.general || ':' || pc.detail ELSE NULL END`; const RECIPIENT_NAME_SQL = 'COALESCE(pr.name, r.name)'; @@ -102,8 +111,8 @@ const TRANSACTION_SORT_COLUMNS = { recipient: 'COALESCE(pr.name, r.name)', category: `CASE WHEN c.id IS NOT NULL THEN c.general || ':' || c.detail - WHEN pc.id IS NOT NULL THEN pc.general || ':' || pc.detail WHEN rc.id IS NOT NULL THEN rc.general || ':' || rc.detail + WHEN pc.id IS NOT NULL THEN pc.general || ':' || pc.detail ELSE NULL END`, bank: 't.bank_account', @@ -218,8 +227,8 @@ export const transactionRepository = { COALESCE(t.category_id, r.default_category_id, pr.default_category_id) AS effective_category_id, CASE WHEN c.id IS NOT NULL THEN c.general || ':' || c.detail - WHEN pc.id IS NOT NULL THEN pc.general || ':' || pc.detail WHEN rc.id IS NOT NULL THEN rc.general || ':' || rc.detail + WHEN pc.id IS NOT NULL THEN pc.general || ':' || pc.detail ELSE NULL END AS category_name${runningBalanceCol} FROM transactions t @@ -572,8 +581,8 @@ export const transactionRepository = { COALESCE(t.category_id, r.default_category_id, pr.default_category_id) AS effective_category_id, CASE WHEN c.id IS NOT NULL THEN c.general || ':' || c.detail - WHEN pc.id IS NOT NULL THEN pc.general || ':' || pc.detail WHEN rc.id IS NOT NULL THEN rc.general || ':' || rc.detail + WHEN pc.id IS NOT NULL THEN pc.general || ':' || pc.detail ELSE NULL END AS category_name${runningBalanceCol} FROM transactions t diff --git a/apps/node-backend/src/services/accountMergeService.js b/apps/node-backend/src/services/accountMergeService.js index 7f8f03c4..42264419 100644 --- a/apps/node-backend/src/services/accountMergeService.js +++ b/apps/node-backend/src/services/accountMergeService.js @@ -155,7 +155,9 @@ export async function previewMerge(sourceId, targetId) { 'SELECT id, currency FROM accounts WHERE id = ANY($1::int[])', [[sourceId, targetId]], ); - const byId = new Map(accounts.rows.map((r) => [r.id, r])); + const byId = new Map(accounts.rows.map( + (/** @type {{ id: number, currency: string }} */ r) => [r.id, r], + )); if (!byId.has(targetId)) throw new NotFoundError(`Account ${targetId} not found`); if (!byId.has(sourceId)) throw new NotFoundError(`Account ${sourceId} not found`); diff --git a/apps/node-backend/src/services/accountService.js b/apps/node-backend/src/services/accountService.js index 62a759a7..79f35316 100644 --- a/apps/node-backend/src/services/accountService.js +++ b/apps/node-backend/src/services/accountService.js @@ -33,6 +33,7 @@ const nameField = z.string({ error: 'name is required and must be a non-empty st .transform((s) => s.trim()); // Clearable free-text: null clears, strings are trimmed, anything else rejects. +/** @param {string} key */ const clearableStringField = (key) => z.string({ error: `${key} must be a string` }) .nullable() .transform((value) => (value === null ? null : value.trim())) @@ -55,9 +56,14 @@ const currencyField = z.unknown().transform((value, ctx) => { return code; }).optional(); +/** + * @param {string} key + * @param {string[]} allowed + */ const enumField = (key, allowed) => z.enum(allowed, { error: `${key} must be one of: ${allowed.join(', ')}` }).optional(); +/** @param {string} key */ const boolField = (key) => z.boolean({ error: `${key} must be a boolean` }).optional(); // FK reference: null clears; otherwise Number() coercion + positive-integer @@ -125,6 +131,8 @@ const accountCreateSchema = accountUpdateSchema.extend({ name: nameField }); /** * Validate + normalize an account payload. On create, name is required; on * update every field is optional. Returns only the fields that were provided. + * @param {any} body unvalidated wire payload — zod does the actual validation. + * @param {{ requireName: boolean }} opts */ function sanitize(body, { requireName }) { const schema = requireName ? accountCreateSchema : accountUpdateSchema; @@ -142,6 +150,14 @@ function sanitize(body, { requireName }) { * A statement balance is only meaningful with its as-of date (ADR-094 drift * anchors on it). Enforced here for a friendly 4xx; migration 0065's CHECK * (ck_accounts_statement_balance_has_date) backstops at the DB. + * + * `balance` is `number|string` because callers pass either side of a "provided + * vs. stored" ternary: a zod-sanitized incoming value (coerced to `number`) or + * the value already on the row (`AccountRow.statement_balance`, pg NUMERIC — + * a `string`). Only presence is checked here, so the numeric-vs-string + * distinction doesn't matter to this function. + * @param {number|string|null|undefined} balance + * @param {string|null|undefined} date */ function assertStatementBalanceHasDate(balance, date) { if (balance != null && date == null) { @@ -184,12 +200,14 @@ export const accountService = { return { items, total }; }, + /** @param {number} id */ async get(id) { const account = await accountRepository.getById(id); if (!account) throw new NotFoundError(`Account ${id} not found`); return account; }, + /** @param {any} body unvalidated wire payload — see `sanitize`. */ async create(body) { const fields = sanitize(body, { requireName: true }); assertStatementBalanceHasDate(fields.statement_balance, fields.statement_balance_date); @@ -204,6 +222,10 @@ export const accountService = { } }, + /** + * @param {number} id + * @param {any} body unvalidated wire payload — see `sanitize`. + */ async update(id, body) { // Widened for closed_at: server-stamped below by the lifecycle logic (D5), // never part of the parsed payload. @@ -258,6 +280,7 @@ export const accountService = { * with zero referencing rows (FK ON DELETE RESTRICT); otherwise a 409 routes * the caller to the close flow (lifecycle D5: active → closed → deleted). */ + /** @param {number} id */ async remove(id) { let removed; try { diff --git a/apps/node-backend/src/services/aggregationRefresh.js b/apps/node-backend/src/services/aggregationRefresh.js index 10064cdd..8b4ec312 100644 --- a/apps/node-backend/src/services/aggregationRefresh.js +++ b/apps/node-backend/src/services/aggregationRefresh.js @@ -50,6 +50,7 @@ function clearForecastMcCaches() { // Mirrors the legacy service's 1s coalescing window for rapid single-row edits. const MC_CLEAR_DEBOUNCE_MS = 1000; +/** @type {ReturnType|null} */ let mcClearTimer = null; /** diff --git a/apps/node-backend/src/services/aiChatService.js b/apps/node-backend/src/services/aiChatService.js index 7a7bd116..39afa4c3 100644 --- a/apps/node-backend/src/services/aiChatService.js +++ b/apps/node-backend/src/services/aiChatService.js @@ -24,6 +24,17 @@ import { getOllamaClient, OllamaError } from '../integrations/ollama/client.js'; import { buildChatMessages } from '../integrations/ollama/prompts.js'; import { dispatchTool, getToolSchemas, getToolNames } from './aiChat/tools/index.js'; +/** @typedef {import('../types/rows.js').AiConversationRow} AiConversationRow */ +/** @typedef {import('../types/rows.js').AiMessageRow} AiMessageRow */ + +/** + * A message in the array sent to/received from the Ollama `/api/chat` + * endpoint (see integrations/ollama/prompts.js `toOllamaMessage`). `tool_calls` + * only appears on an assistant message that invoked a tool; `name` only + * appears on a `role: 'tool'` result message. + * @typedef {{ role: string, content: string, tool_calls?: any[], name?: string }} OllamaMessage + */ + const MAX_TOOL_ITERATIONS = 6; const DEFAULT_CONVERSATION_TITLE = 'New conversation'; @@ -41,6 +52,11 @@ export class AiChatServiceError extends AppError { } } +/** + * @param {any} rawArgs raw `function.arguments` from an Ollama tool call — + * either an already-parsed object, a JSON string, or absent. + * @returns {any} + */ function parseToolCallArguments(rawArgs) { if (rawArgs == null) return {}; if (typeof rawArgs === 'object') return rawArgs; @@ -56,6 +72,10 @@ function parseToolCallArguments(rawArgs) { return rawArgs; } +/** + * @param {any} toolCall raw Ollama `tool_calls[]` entry. + * @returns {{ name: string|null, args: any }} + */ function normalizeToolCall(toolCall) { const fn = toolCall?.function || toolCall || {}; const name = fn.name || toolCall?.name || null; @@ -63,6 +83,11 @@ function normalizeToolCall(toolCall) { return { name, args }; } +/** + * @param {string|null|undefined} text + * @param {number} [maxLen] + * @returns {string} + */ function truncateTitle(text, maxLen = 60) { const trimmed = (text || '').trim().replace(/\s+/g, ' '); if (!trimmed) return DEFAULT_CONVERSATION_TITLE; @@ -73,6 +98,9 @@ function truncateTitle(text, maxLen = 60) { /** * Ensure a conversation exists. If `conversationId` is provided, validate it; * otherwise create a new one titled from the first user message. + * + * @param {{ conversationId?: string|null, model?: string|null, firstUserMessage?: string }} args + * @returns {Promise} */ async function ensureConversation({ conversationId, model, firstUserMessage }) { if (conversationId) { @@ -221,6 +249,7 @@ async function runChatTurnInner({ const toolSchemas = useTools ? getToolSchemas() : []; const toolNames = useTools ? getToolNames() : []; + /** @type {OllamaMessage[]} */ const baseMessages = buildChatMessages({ toolNames, history, @@ -228,8 +257,10 @@ async function runChatTurnInner({ maxHistoryMessages: settings.aiChat.maxHistoryMessages, }); + /** @type {AiMessageRow[]} */ const toolMessages = []; let iterations = 0; + /** @type {{ evalCount: number|null, promptEvalCount: number|null, totalDurationMs: number|null }} */ let lastUsage = { evalCount: null, promptEvalCount: null, totalDurationMs: null }; // Request-scoped cache shared across every tool call in this chat turn so @@ -295,7 +326,7 @@ async function runChatTurnInner({ messages: baseMessages, tools: toolSchemas.length > 0 ? toolSchemas : undefined, signal, - onToken: async (delta) => { + onToken: async (/** @type {string} */ delta) => { if (delta) await onEvent?.({ type: 'token', data: delta }); }, }); @@ -419,6 +450,10 @@ export async function listConversations() { return aiChatRepository.listConversations(); } +/** + * @param {string} id UUID. + * @returns {Promise<{ conversation: AiConversationRow, messages: AiMessageRow[] }|null>} + */ export async function getConversationWithMessages(id) { const conversation = await aiChatRepository.getConversation(id); if (!conversation) return null; @@ -426,6 +461,10 @@ export async function getConversationWithMessages(id) { return { conversation, messages }; } +/** + * @param {{ title?: string|null, model?: string|null }} args + * @returns {Promise<{ conversation: AiConversationRow, messages: AiMessageRow[] }>} + */ export async function createEmptyConversation({ title, model }) { const conversation = await aiChatRepository.createConversation({ title: truncateTitle(title) || DEFAULT_CONVERSATION_TITLE, @@ -434,10 +473,19 @@ export async function createEmptyConversation({ title, model }) { return { conversation, messages: [] }; } +/** + * @param {string} id UUID. + * @param {string} title + * @returns {Promise} + */ export async function renameConversation(id, title) { return aiChatRepository.renameConversation(id, truncateTitle(title)); } +/** + * @param {string} id UUID. + * @returns {Promise} + */ export async function deleteConversation(id) { return aiChatRepository.deleteConversation(id); } diff --git a/apps/node-backend/src/services/attachmentCleanup.js b/apps/node-backend/src/services/attachmentCleanup.js index 40cebded..32b46b31 100644 --- a/apps/node-backend/src/services/attachmentCleanup.js +++ b/apps/node-backend/src/services/attachmentCleanup.js @@ -15,6 +15,10 @@ import { removeAttachmentFile } from './attachmentService.js'; import { logger } from '../config/logger.js'; +/** + * @param {string[]} storedPaths + * @returns {Promise} + */ export async function removeAttachmentFilesBestEffort(storedPaths) { for (const storedPath of storedPaths) { try { diff --git a/apps/node-backend/src/services/attachmentService.js b/apps/node-backend/src/services/attachmentService.js index fa36268b..8a22678b 100644 --- a/apps/node-backend/src/services/attachmentService.js +++ b/apps/node-backend/src/services/attachmentService.js @@ -10,6 +10,7 @@ * Multer uses memoryStorage so the service controls the final path. */ +/// import { mkdirSync, promises as fsPromises } from 'fs'; import { join, extname, resolve, sep } from 'path'; import { randomUUID } from 'crypto'; @@ -17,6 +18,11 @@ import multer from 'multer'; import { env } from '../config/env.js'; import { sniffMime, extensionMime } from '../lib/fileSniff.js'; +/** + * The slice of a multer memoryStorage upload this service reads. + * @typedef {{ originalname: string, buffer: Buffer, mimetype: string }} MulterFile + */ + const ALLOWED_MIME_PREFIXES = ['image/', 'application/pdf']; const MAX_SIZE_BYTES = env.ATTACHMENT_MAX_SIZE_MB * 1024 * 1024; @@ -25,11 +31,15 @@ export function getAttachmentsRoot() { return resolve(env.ATTACHMENTS_DIR); } -/** Absolute path to the per-transaction directory. */ +/** + * Absolute path to the per-transaction directory. + * @param {number|string} transactionId + */ export function getTransactionDir(transactionId) { return join(getAttachmentsRoot(), String(transactionId)); } +/** @param {string} mimeType */ function isAllowedMime(mimeType) { return ALLOWED_MIME_PREFIXES.some((prefix) => mimeType.startsWith(prefix)); } @@ -40,6 +50,8 @@ function isAllowedMime(mimeType) { * Returns the canonical sniffed MIME type when valid; throws otherwise. * The sniffed value should be persisted to the DB instead of the * client-supplied req.file.mimetype. + * @param {MulterFile} file + * @returns {string} canonical sniffed MIME type */ export function verifyAttachmentContent(file) { const sniffed = sniffMime(file.buffer); @@ -67,7 +79,11 @@ export function verifyAttachmentContent(file) { export const attachmentUpload = multer({ storage: multer.memoryStorage(), limits: { fileSize: MAX_SIZE_BYTES }, - fileFilter: (_req, file, cb) => { + fileFilter: ( + /** @type {unknown} */ _req, + /** @type {MulterFile} */ file, + /** @type {(error: Error|null, acceptFile?: boolean) => void} */ cb, + ) => { if (!isAllowedMime(file.mimetype)) { cb(new Error(`Unsupported file type: ${file.mimetype}. Allowed: images and PDF.`)); } else { @@ -82,6 +98,9 @@ export const attachmentUpload = multer({ * Returns the stored_path (relative to ATTACHMENTS_DIR root, forward * slashes) for persistence in the DB. Callers store this path and pass * it back to resolveAbsolutePath() to serve the file later. + * @param {number|string} transactionId + * @param {MulterFile} file + * @returns {Promise} stored_path relative to the attachments root */ export async function storeAttachment(transactionId, file) { const ext = extname(file.originalname).toLowerCase() || ''; @@ -102,6 +121,8 @@ export async function storeAttachment(transactionId, file) { * Resolve a DB-stored relative path back to an absolute filesystem path. * Throws if the resolved path escapes the attachments root (path traversal guard). * Uses realpath to resolve symlinks so a symlinked path cannot escape the root. + * @param {string} storedPath + * @returns {Promise} */ export async function resolveAbsolutePath(storedPath) { const root = getAttachmentsRoot(); @@ -123,6 +144,7 @@ export async function resolveAbsolutePath(storedPath) { /** * Delete a file from disk. Silently ignores ENOENT (already gone). + * @param {string} storedPath */ export async function removeAttachmentFile(storedPath) { const absPath = await resolveAbsolutePath(storedPath); diff --git a/apps/node-backend/src/services/belgianInflationService.js b/apps/node-backend/src/services/belgianInflationService.js index 5fc4b71d..45ddec65 100644 --- a/apps/node-backend/src/services/belgianInflationService.js +++ b/apps/node-backend/src/services/belgianInflationService.js @@ -3,6 +3,8 @@ import { logger } from '../config/logger.js'; import { recordSuccess as recordProviderSuccess, recordError as recordProviderError } from './providerHealthService.js'; import { roundMoney } from '../lib/money.js'; +/** @typedef {import('../types/rows.js').BelgianInflationRate} BelgianInflationRate */ + // belgian_inflation_rates.monthly_rate is NUMERIC(10,8): keep the full 8 dp of // stored scale. Rounding to 6 dp here threw away precision the column could // hold, and the truncation compounds multiplicatively in snapshotBuilder. @@ -22,10 +24,16 @@ const STATBEL_CANDIDATE_URLS = [ const EUROSTAT_HICP_INDEX_URL = 'https://ec.europa.eu/eurostat/api/dissemination/statistics/1.0/data/prc_hicp_midx?geo=BE&coicop=CP00&unit=I15'; -let memoryCache = null; // { rates: Array<{ month: string, monthly_rate: number }>, timestamp: number } +/** @type {{ rates: BelgianInflationRate[], timestamp: number }|null} */ +let memoryCache = null; let statbelFailureLogState = { lastWarnAt: 0, suppressed: 0 }; +/** @type {Promise|null} */ let backgroundRefreshPromise = null; +/** + * @param {any} value + * @returns {boolean} + */ function isTruthy(value) { if (value === true || value === 1) return true; if (typeof value === 'string') { @@ -38,10 +46,17 @@ function isTruthy(value) { // Deliberately global-setTimeout-based (not node:timers/promises): the retry // throttle test drives this with vi.useFakeTimers(), which patches the global // but cannot fake timers/promises. +/** + * @param {number} ms + * @returns {Promise} + */ function sleep(ms) { return new Promise((resolve) => setTimeout(resolve, ms)); } +/** + * @param {any} error + */ function logStatbelFallbackWarning(error) { const now = Date.now(); if (now - statbelFailureLogState.lastWarnAt >= STATBEL_WARN_THROTTLE_MS) { @@ -60,6 +75,10 @@ function logStatbelFallbackWarning(error) { }); } +/** + * @param {any} value + * @returns {string|undefined} 'YYYY-MM' + */ function normalizeMonthInput(value) { if (!value) return undefined; const text = String(value).trim(); @@ -75,6 +94,10 @@ function normalizeMonthInput(value) { return undefined; } +/** + * @param {any} value + * @returns {string|undefined} 'YYYY-MM' + */ function monthKeyFromDatabaseValue(value) { if (value === null || value === undefined) return undefined; @@ -96,6 +119,10 @@ function monthKeyFromDatabaseValue(value) { return undefined; } +/** + * @param {any} value + * @returns {number|undefined} + */ function parseNumeric(value) { if (typeof value === 'number' && Number.isFinite(value)) return value; if (typeof value !== 'string') return undefined; @@ -105,6 +132,7 @@ function parseNumeric(value) { return Number.isFinite(parsed) ? parsed : undefined; } +/** @type {Record} */ const MONTH_NAME_TO_NUMBER = { january: 1, jan: 1, januari: 1, janvier: 1, february: 2, feb: 2, februari: 2, fevrier: 2, février: 2, @@ -120,12 +148,20 @@ const MONTH_NAME_TO_NUMBER = { december: 12, dec: 12, decembre: 12, décembre: 12, }; +/** + * @param {any} value + * @returns {number|undefined} + */ function parseMonthName(value) { if (!value) return undefined; const normalized = String(value).trim().toLowerCase(); return MONTH_NAME_TO_NUMBER[normalized]; } +/** + * @param {any} row raw provider payload row — key set varies by upstream (EN/NL/FR). + * @returns {string|undefined} 'YYYY-MM' + */ function parseMonthFromRow(row) { const directMonth = normalizeMonthInput( row.month @@ -149,6 +185,10 @@ function parseMonthFromRow(row) { return `${Math.trunc(year)}-${String(Math.trunc(monthNumber)).padStart(2, '0')}`; } +/** + * @param {any} row raw provider payload row — key set varies by upstream (EN/NL/FR). + * @returns {number|undefined} + */ function parseMonthlyRateFromRow(row) { const candidateKeys = [ 'monthly_rate', @@ -189,6 +229,15 @@ function parseMonthlyRateFromRow(row) { return parsed; } +/** + * Recursively flatten a nested provider JSON payload (structure varies by + * upstream) down to the leaf "row-like" objects — those with at least one + * string/number property, which is `parseMonthFromRow`/`parseMonthlyRateFromRow`'s + * signal that an object is a data row rather than a container. + * + * @param {any} payload + * @returns {any[]} + */ function extractObjectRows(payload) { if (Array.isArray(payload)) { return payload.flatMap((item) => extractObjectRows(item)); @@ -200,11 +249,16 @@ function extractObjectRows(payload) { const objectValues = values.filter((value) => value && typeof value === 'object'); const looksLikeRow = Object.values(payload).some((value) => ['string', 'number'].includes(typeof value)); + /** @type {any[]} */ const nestedRows = objectValues.flatMap((value) => extractObjectRows(value)); return looksLikeRow ? [payload, ...nestedRows] : nestedRows; } +/** + * @param {any} payload + * @returns {BelgianInflationRate[]} + */ function normalizeRatesFromPayload(payload) { const rows = extractObjectRows(payload); const byMonth = new Map(); @@ -222,6 +276,10 @@ function normalizeRatesFromPayload(payload) { return [...byMonth.values()].sort((a, b) => a.month.localeCompare(b.month)); } +/** + * @param {any} value e.g. 'January 2024'. + * @returns {string|undefined} 'YYYY-MM' + */ function parseStatbelMonthString(value) { if (!value) return undefined; const parts = String(value).trim().split(/\s+/); @@ -232,6 +290,10 @@ function parseStatbelMonthString(value) { return `${Math.trunc(year)}-${String(monthNum).padStart(2, '0')}`; } +/** + * @param {any} payload + * @returns {BelgianInflationRate[]} + */ function normalizeRatesFromStatbelPayload(payload) { const facts = payload?.facts; if (!Array.isArray(facts) || facts.length === 0) return []; @@ -260,6 +322,10 @@ function normalizeRatesFromStatbelPayload(payload) { return rates; } +/** + * @param {any} payload + * @returns {BelgianInflationRate[]} + */ function normalizeRatesFromEurostatIndexPayload(payload) { const timeIndex = payload?.dimension?.time?.category?.index; const values = payload?.value; @@ -304,6 +370,7 @@ function normalizeRatesFromEurostatIndexPayload(payload) { } async function fetchFromStatbel() { + /** @type {(url: string) => Promise} */ const fetchWithRetries = async (url) => { let lastError; @@ -429,6 +496,11 @@ function scheduleBackgroundInflationRefresh() { })(); } +/** + * @param {string} [startMonth] 'YYYY-MM' + * @param {string} [endMonth] 'YYYY-MM' + * @returns {Promise} + */ async function loadFromDatabase(startMonth, endMonth) { const result = await query( `SELECT month_date, monthly_rate @@ -439,24 +511,33 @@ async function loadFromDatabase(startMonth, endMonth) { [startMonth ? `${startMonth}-01` : null, endMonth ? `${endMonth}-01` : null] ); - return result.rows + return /** @type {Pick[]} */ (result.rows) .map((row) => ({ month: monthKeyFromDatabaseValue(row.month_date), monthly_rate: Number(row.monthly_rate), })) - .filter((row) => row.month); + .filter( + /** @returns {row is BelgianInflationRate} */ + (row) => Boolean(row.month), + ); } // Postgres caps each query at 65535 bind parameters. With 3 params per row // (month_date, monthly_rate, source), 1000 rows per chunk leaves headroom. const INFLATION_INSERT_CHUNK = 1000; +/** + * @param {BelgianInflationRate[]} rates + * @param {string} [source] + * @returns {Promise} + */ async function saveToDatabase(rates, source = 'statbel') { if (!Array.isArray(rates) || rates.length === 0) return; await withTransaction(async (client) => { for (let i = 0; i < rates.length; i += INFLATION_INSERT_CHUNK) { const chunk = rates.slice(i, i + INFLATION_INSERT_CHUNK); + /** @type {any[]} */ const params = []; const valueRows = chunk.map((rate, idx) => { const offset = idx * 3; @@ -479,6 +560,12 @@ async function saveToDatabase(rates, source = 'statbel') { }); } +/** + * @param {BelgianInflationRate[]} rates + * @param {string|undefined} startMonth 'YYYY-MM' + * @param {string|undefined} endMonth 'YYYY-MM' + * @returns {BelgianInflationRate[]} + */ function filterRates(rates, startMonth, endMonth) { return rates.filter((rate) => { if (startMonth && rate.month < startMonth) return false; diff --git a/apps/node-backend/src/services/bulkSelection.js b/apps/node-backend/src/services/bulkSelection.js index 10fb2c0c..c78cd76b 100644 --- a/apps/node-backend/src/services/bulkSelection.js +++ b/apps/node-backend/src/services/bulkSelection.js @@ -19,14 +19,30 @@ import { EXPORT_JOINS_SQL } from './transactionExport.js'; const DEFAULT_ID_CAP = 500; const DEFAULT_FILTER_CAP = 5000; +/** + * Wire filter body accepted by the bulk-action endpoints. Deliberately a + * dynamic property bag (`Record`), not a fixed interface: it + * accepts BOTH the camelCase shape used internally and the snake_case + * query-string shape the frontend sends for the same field (see `pick` + * below), so any given key may or may not be present under either name. + * @typedef {Record} BulkFilterInput + */ + /** * Coerce a wire filter object into the shape `buildTransactionWhere` expects. * Accepts both the camelCase shape used internally and the snake_case query-string * shape used by the frontend, so callers can forward request bodies as-is. + * @param {BulkFilterInput} filter + * @returns {object} */ export function normalizeBulkFilter(filter) { if (!filter || typeof filter !== 'object') return {}; + /** + * @param {string} camel + * @param {string} snake + * @returns {any} + */ const pick = (camel, snake) => filter[camel] ?? filter[snake] ?? null; const tagSlugs = Array.isArray(filter.tagSlugs) @@ -135,7 +151,7 @@ export async function resolveBulkSelection(selector = {}, opts = {}) { `SELECT t.id ${EXPORT_JOINS_SQL} WHERE ${whereSql} ORDER BY t.id`, params, ); - return idsResult.rows.map((row) => row.id); + return idsResult.rows.map((/** @type {{ id: number }} */ row) => row.id); } export const BULK_SELECTION_DEFAULTS = Object.freeze({ diff --git a/apps/node-backend/src/services/calculations/aggregation/sankey.js b/apps/node-backend/src/services/calculations/aggregation/sankey.js index 5d872861..7260595e 100644 --- a/apps/node-backend/src/services/calculations/aggregation/sankey.js +++ b/apps/node-backend/src/services/calculations/aggregation/sankey.js @@ -19,6 +19,8 @@ */ import { query } from '../../../database/connection.js'; +import { buildExclusionClauses } from '../../../lib/filterBuilder.js'; +import { getIncludeTransfers } from '../../../repositories/infoRepositoryHelpers.js'; import { convertRowsToEur } from '../../currency/currencyConversionService.js'; import { buildEnvelope } from './_envelope.js'; import { assertNoNaN } from './_invariants.js'; @@ -44,17 +46,38 @@ export async function computeSankeyFlow({ /** @type {any[]} */ const params = [yearStart, yearEnd]; - const clauses = []; - if (excludedCategoryIds.length > 0) { - params.push(excludedCategoryIds); - clauses.push(`AND COALESCE(t.category_id, r.default_category_id) != ALL($${params.length})`); - } - - if (excludedRecipientIds.length > 0) { - params.push(excludedRecipientIds); - clauses.push(`AND t.recipient_id != ALL($${params.length})`); - } + // Canonical exclusion clauses (lib/filterBuilder.buildExclusionClauses), + // shared with every other money surface: the 3-level effective-category + // COALESCE (own → recipient default → PRIMARY recipient's default, matching + // transactionRepository and the category JOIN below) and the ALIAS-AWARE + // recipient form, both carrying the `-1` NULL sentinel. + // + // This file used to hand-roll both, and drifted from the canonical pair in + // two ways. `!= ALL($n)` without the sentinel evaluates to NULL — not true — + // for a NULL effective category, so *every uncategorised row* was silently + // dropped as soon as any exclusion was applied: excluding one spending + // category erased an unrelated €3000 uncategorised income row and rendered + // "Income 0" with money still flowing out of it. And the bare + // `t.recipient_id != ALL(...)` was not alias-aware, so excluding a PRIMARY + // recipient left its aliases' rows in the graph. Routing through the shared + // builder keeps sankey out of the exclusion-drift business for good. + // + // The date filter owns $1/$2, so the exclusion placeholders start at $3. + const excl = buildExclusionClauses({ + excludedCategoryIds, + excludedRecipientIds, + startParamIdx: params.length + 1, + }); + params.push(...excl.params); + const exclusionWhere = excl.whereSql ? `AND ${excl.whereSql}` : ''; + + // ADR-083: internal transfers are excluded from income/spending by default — + // a savings transfer's two legs would otherwise inflate BOTH sides of the + // flow graph (fake income in, fake spending out). Governed by the same + // runtime `includeTransfers` setting every other aggregation honours, so a + // user who opted in to seeing transfers sees them here too. + const includeTransfers = await getIncludeTransfers(); // Aggregate in SQL: this endpoint converts with latest rates only (one rate // per currency below), so SUM(ABS(amount)) per (category, currency, is_income) @@ -70,10 +93,12 @@ export async function computeSankeyFlow({ SUM(ABS(t.amount)) AS amount FROM transactions t LEFT JOIN recipients r ON t.recipient_id = r.id - LEFT JOIN categories c ON COALESCE(t.category_id, r.default_category_id) = c.id + LEFT JOIN recipients pr ON r.primary_recipient_id = pr.id + LEFT JOIN categories c ON COALESCE(t.category_id, r.default_category_id, pr.default_category_id) = c.id WHERE t.is_active = true + ${includeTransfers ? '' : 'AND t.is_transfer = false'} AND t.date BETWEEN $1 AND $2 - ${clauses.join('\n ')} + ${exclusionWhere} GROUP BY 1, 2, 3 `, params, diff --git a/apps/node-backend/src/services/calculations/forecast/index.js b/apps/node-backend/src/services/calculations/forecast/index.js index 29ea271c..fb4af385 100644 --- a/apps/node-backend/src/services/calculations/forecast/index.js +++ b/apps/node-backend/src/services/calculations/forecast/index.js @@ -243,10 +243,22 @@ function buildSeed({ yyyymm, filterHash }) { // mcPercentiles, but nothing else guarded historyMonths — a request with // history_months=120 happily got a cached 36-month forecast served back, and // a non-default history cached under the same key, colliding both directions. -function filterHash({ excludedCategoryIds, excludedRecipientIds, currency, includePlanned, historyMonths }) { +// +// includeTransfers (ADR-083) is part of the key for the same reason: the +// forecast repositories now apply the transfer predicate, so the setting +// changes the numbers. Without it in the hash, toggling the setting kept +// serving the pre-toggle forecast for the cache's 6h TTL — and the two answers +// differ in polarity, not just magnitude, when transfers dominate the ledger. +/** + * @param {{ excludedCategoryIds?: number[], excludedRecipientIds?: number[], + * currency: string, includePlanned: boolean, historyMonths: number, + * includeTransfers: boolean }} input + * @returns {string} + */ +function filterHash({ excludedCategoryIds, excludedRecipientIds, currency, includePlanned, historyMonths, includeTransfers }) { const cats = [...(excludedCategoryIds ?? [])].sort((a, b) => a - b).join(','); const recs = [...(excludedRecipientIds ?? [])].sort((a, b) => a - b).join(','); - return `${currency}|${cats}|${recs}|${includePlanned ? 1 : 0}|h${historyMonths}`; + return `${currency}|${cats}|${recs}|${includePlanned ? 1 : 0}|h${historyMonths}|t${includeTransfers ? 1 : 0}`; } function isDefaultMcParams(mcPaths, mcPercentiles) { @@ -276,7 +288,9 @@ export async function computeCashflowForecast({ _forceCache = false, } = {}) { const { all, future, todayDay, daysInMonth, yyyymm } = currentMonthDates(); - const hash = filterHash({ excludedCategoryIds, excludedRecipientIds, currency: targetCurrency, includePlanned, historyMonths }); + // Read before the cache probe: the setting is a cache-key input (ADR-083). + const includeTransfers = await infoRepository.getIncludeTransfers(); + const hash = filterHash({ excludedCategoryIds, excludedRecipientIds, currency: targetCurrency, includePlanned, historyMonths, includeTransfers }); // Try cache when not forcing a refresh and using default MC params. if (!_forceCache && isDefaultMcParams(mcPaths, mcPercentiles)) { @@ -432,7 +446,9 @@ export async function computeCashflowForecastRolling({ userId = 'anonymous', } = {}) { const { all, future, todayIndex, todayIso } = rollingWindowDates(daysBack, daysForward); - const hash = filterHash({ excludedCategoryIds, excludedRecipientIds, currency: targetCurrency, includePlanned, historyMonths }); + // Read before the cache probe: the setting is a cache-key input (ADR-083). + const includeTransfers = await infoRepository.getIncludeTransfers(); + const hash = filterHash({ excludedCategoryIds, excludedRecipientIds, currency: targetCurrency, includePlanned, historyMonths, includeTransfers }); // Cache check: skip for non-default rolling MC params or when backtest is requested. if (isDefaultRollingMcParams(mcPaths, mcPercentiles) && !includeBacktest) { diff --git a/apps/node-backend/src/services/cashForecastInsightService.js b/apps/node-backend/src/services/cashForecastInsightService.js index af25932f..c1e13c8d 100644 --- a/apps/node-backend/src/services/cashForecastInsightService.js +++ b/apps/node-backend/src/services/cashForecastInsightService.js @@ -33,14 +33,25 @@ const MOVE_PCT = 0.15; // €20 projection is noise, not a signal. const MOVE_ABS_FLOOR = 100; +/** + * The subset of a computeCashflowForecast method result this module reads. + * The envelope's `methods` field is `any[]` (calculations/forecast/ is + * outside this ratchet slice) — narrowed locally to what this file touches. + * @typedef {object} ForecastMethod + * @property {string} id + * @property {string|null} [error] + * @property {Array<{ date: string, value: number }>} [cumulative] actuals-to-date folded with the projection. + * @property {{ p10?: Array<{ date: string, value: number }>, p90?: Array<{ date: string, value: number }> }|null} [bands] daily (non-cumulative) percentile series, MC methods only. + */ + /** * Pick the primary forecast method from the payload's `methods` array: * 1. first Monte-Carlo method (non-null `bands`, no `error`), * 2. else the ensemble method (no `error`), * 3. else the first method with no `error`. * - * @param {Array<{ id: string, bands: object|null, error: string|null }>} methods - * @returns {object|null} the chosen method, or null when none is usable + * @param {ForecastMethod[]} methods + * @returns {ForecastMethod|null} the chosen method, or null when none is usable */ function pickPrimaryMethod(methods) { if (!Array.isArray(methods)) return null; @@ -78,7 +89,7 @@ function foldDailyBandToMonthEnd(bandSeries, futureDays, anchor) { /** * Pure builder: distill a forecast payload into the month-end cash finding. * - * @param {{ month: string, currency: string, current_day: number, methods: any[] }} payload + * @param {{ month: string, currency: string, current_day: number, methods: ForecastMethod[] }} payload * The `data` payload of the computeCashflowForecast envelope. * @param {number|null} [previousMonthEndProjected] Month-end P50 from a prior * run, used to detect a significant move; null disables the comparison. diff --git a/apps/node-backend/src/services/crossWorkspaceAnalytics.js b/apps/node-backend/src/services/crossWorkspaceAnalytics.js index 18f9f053..c9bb9a47 100644 --- a/apps/node-backend/src/services/crossWorkspaceAnalytics.js +++ b/apps/node-backend/src/services/crossWorkspaceAnalytics.js @@ -73,6 +73,7 @@ export function rebalanceDeployment({ actualValues, targetWeights, availableCash const totalAfter = actualTotal.plus(cash); // Shortfall per sleeve = max(0, desired − actual) where desired = totalAfter·target. + /** @type {Record>} */ const shortfalls = {}; let shortfallSum = toDecimal(0); for (const [sleeve, weight] of Object.entries(targetWeights ?? {})) { diff --git a/apps/node-backend/src/services/crossWorkspaceDataService.js b/apps/node-backend/src/services/crossWorkspaceDataService.js index 8142afe1..17dff374 100644 --- a/apps/node-backend/src/services/crossWorkspaceDataService.js +++ b/apps/node-backend/src/services/crossWorkspaceDataService.js @@ -42,9 +42,13 @@ export async function assembleRebalanceInputs({ currency = 'EUR' } = {}) { const { summaries } = await getPortfolioSummary(target); const actualValues = /** @type {Record} */ ({}); - for (const s of summaries) { + // getPortfolioSummary's JSDoc return type is deliberately loose + // (`summaries: object[]`) since portfolioSummaryService.js is outside this + // ratchet slice — narrow locally to the two fields this loop actually reads. + const typedSummaries = /** @type {Array<{ asset_class?: string, currentValue?: number }>} */ (summaries); + for (const s of typedSummaries) { const assetClass = s.asset_class || 'other'; - const key = SLEEVE_ROLLUP[assetClass] ?? assetClass; + const key = /** @type {string} */ (SLEEVE_ROLLUP[/** @type {keyof typeof SLEEVE_ROLLUP} */ (assetClass)] ?? assetClass); actualValues[key] = toNumber(roundToCents(toDecimal(actualValues[key] ?? 0).plus(toDecimal(s.currentValue ?? 0)))); } diff --git a/apps/node-backend/src/services/currency/currencyConversionService.js b/apps/node-backend/src/services/currency/currencyConversionService.js index a5baaae4..00c50f8b 100644 --- a/apps/node-backend/src/services/currency/currencyConversionService.js +++ b/apps/node-backend/src/services/currency/currencyConversionService.js @@ -32,7 +32,14 @@ import { } from './rateFetcher.js'; import { settingsRepository } from '../../repositories/settingsRepository.js'; +/** + * @typedef {import('../../types/rows.js').ExchangeRateRow} ExchangeRateRow + * @typedef {import('../../types/rows.js').HistoricalRateIndex} HistoricalRateIndex + * @typedef {import('../../types/rows.js').RateTable} RateTable + */ + // In-memory cache: { rates: {...}, timestamp: number } | null +/** @type {{ rates: RateTable, timestamp: number } | null} */ let memoryCache = null; // Process-level cache of the built historical-rate index. Historical-FX row @@ -43,7 +50,7 @@ let memoryCache = null; // refreshed (warmCache), the memory cache is cleared, or a backfill runs, and // expires after CACHE_LIFETIME_MS as a backstop. Cache misses still resolve // correctly via the per-date fallback paths, so conversion results are unchanged. -/** @type {{ index: Map>, currencies: string[], builtAt: number } | null} */ +/** @type {{ index: HistoricalRateIndex, currencies: string[], builtAt: number } | null} */ let historicalIndexCache = null; // Static hardcoded fallback rates. Never mutated. @@ -105,6 +112,8 @@ let liveFallbackRates = FALLBACK_RATES; * 1. In-memory cache (24-hour TTL) * 2. Database (latest rows) * 3. Hardcoded fallback + * + * @returns {Promise} */ async function getRates() { if (memoryCache && Date.now() - memoryCache.timestamp < CACHE_LIFETIME_MS) { @@ -245,6 +254,10 @@ export async function warmCache() { /** * Convert an amount from `fromCurrency` to EUR. + * + * @param {number} amount + * @param {string|null|undefined} fromCurrency falsy is treated as "already EUR" + * @returns {Promise} */ export async function convertToEur(amount, fromCurrency) { return convertToCurrency(amount, fromCurrency, 'EUR'); @@ -267,13 +280,34 @@ export async function convertRowsToEur(rows, targetCurrency = 'EUR', options = { const toCur = (targetCurrency || 'EUR').toUpperCase().trim(); const rates = await getRates(); + /** @type {Map} */ const historicalRateCache = new Map(); + // Per-request memo for currencies the historical index knows NOTHING about. + // The index is empty for a currency only when `exchange_rates` holds no row + // for it at all, and then the per-date point lookup below misses on EVERY + // date and goes to the network. A daily series (balance history: up to 366 + // distinct days per request) turned that into hundreds of sequential + // round-trips, each of which is a multi-second timeout when offline. One + // attempt per currency per request is enough: it saves what it fetches, so + // the index has a row to interpolate from next time. + /** @type {Map} */ + const unindexedCurrencyRate = new Map(); + + /** + * @param {Record} row + * @returns {string|null} 'YYYY-MM-DD', or null when the row carries no usable date + */ function resolveDateFromRow(row) { if (dateField && row[dateField]) return normalizeDateInput(row[dateField]); return normalizeDateInput(row.date || row.day || row.transaction_date || row.planned_date || row.rate_date); } + /** + * @param {string} currencyCode + * @param {string|null} rowDate + * @returns {Promise} + */ async function getRate(currencyCode, rowDate) { if (!useHistoricalRatesByDate || !rowDate) return rates[currencyCode]; const key = `${currencyCode}:${rowDate}`; @@ -287,6 +321,11 @@ export async function convertRowsToEur(rows, targetCurrency = 'EUR', options = { // boolean flagging whether we fell back to current rates because the // requested historical rate was missing. Callers surface this so the // frontend can label affected rows. + /** + * @param {string} code + * @param {string|null} rowDate + * @returns {Promise<{ rate: number|undefined, fellBack: boolean }>} + */ async function resolveRateWithFallback(code, rowDate) { if (code === 'EUR') return { rate: 1, fellBack: false }; @@ -299,7 +338,16 @@ export async function convertRowsToEur(rows, targetCurrency = 'EUR', options = { : undefined; if (historical !== undefined) return { rate: historical, fellBack: false }; - const fetched = await getRate(code, rowDate); + // Index built but empty for this currency (findNearestRateInIndex only + // misses when the currency has no entries at all): one network attempt per + // request, not one per date. See unindexedCurrencyRate above. + let fetched; + if (historicalIndex && unindexedCurrencyRate.has(code)) { + fetched = unindexedCurrencyRate.get(code); + } else { + fetched = await getRate(code, rowDate); + if (historicalIndex) unindexedCurrencyRate.set(code, fetched); + } if (fetched !== undefined) return { rate: fetched, fellBack: false }; const fallback = rates[code]; @@ -311,6 +359,7 @@ export async function convertRowsToEur(rows, targetCurrency = 'EUR', options = { return { rate: undefined, fellBack: true }; } + /** @type {HistoricalRateIndex|null} */ let historicalIndex = null; if (useHistoricalRatesByDate) { const relevantCurrencies = [...new Set([ @@ -362,6 +411,11 @@ export async function convertRowsToEur(rows, targetCurrency = 'EUR', options = { /** * Generic converter from any currency to any currency. + * + * @param {number} amount + * @param {string|null|undefined} fromCurrency falsy short-circuits to `amount` + * @param {string|null|undefined} toCurrency defaults to EUR when falsy + * @returns {Promise} */ export async function convertToCurrency(amount, fromCurrency, toCurrency) { if (!fromCurrency || fromCurrency.toUpperCase().trim() === (toCurrency || 'EUR').toUpperCase().trim()) { @@ -375,6 +429,8 @@ export async function convertToCurrency(amount, fromCurrency, toCurrency) { * Fetch the current rate table once. Callers that convert many rows in a loop * should call this once and pass the result to {@link convertWithRates}, * avoiding a per-row `await` on the (already memory-cached) rate lookup. + * + * @returns {Promise} */ export async function loadCurrentRates() { return getRates(); @@ -383,6 +439,12 @@ export async function loadCurrentRates() { /** * Synchronous conversion against a pre-fetched rate table. Mirrors the logic of * {@link convertToCurrency} exactly — only the rate acquisition is hoisted out. + * + * @param {number} amount + * @param {string|null|undefined} fromCurrency falsy short-circuits to `amount` + * @param {string|null|undefined} toCurrency defaults to EUR when falsy + * @param {RateTable} rates + * @returns {number} */ export function convertWithRates(amount, fromCurrency, toCurrency, rates) { if (!fromCurrency || fromCurrency.toUpperCase().trim() === (toCurrency || 'EUR').toUpperCase().trim()) { @@ -448,7 +510,8 @@ async function repairHistoricalRatesFromFullHistory(pairs) { [currencies] ); const storedByKey = new Map( - storedResult.rows.map((r) => [`${r.currency_code}:${r.rate_date}`, toNumber(toDecimal(r.rate_to_eur))]) + /** @type {Array & { rate_date: string }>} */ + (storedResult.rows).map((r) => [`${r.currency_code}:${r.rate_date}`, toNumber(toDecimal(r.rate_to_eur))]) ); let repaired = 0; @@ -563,7 +626,8 @@ export async function backfillPortfolioHistoricalRates() { [codes, dates] ); const present = new Set( - existsResult.rows.map((r) => `${r.currency_code}|${String(r.rate_date).slice(0, 10)}`) + /** @type {Array & { rate_date: string }>} */ + (existsResult.rows).map((r) => `${r.currency_code}|${String(r.rate_date).slice(0, 10)}`) ); for (const p of resolvedPairs) { if (present.has(`${p.currencyCode}|${p.rateDate}`)) inserted += 1; diff --git a/apps/node-backend/src/services/currency/rateFetcher.js b/apps/node-backend/src/services/currency/rateFetcher.js index 4e7e9e28..9f1d63ad 100644 --- a/apps/node-backend/src/services/currency/rateFetcher.js +++ b/apps/node-backend/src/services/currency/rateFetcher.js @@ -11,6 +11,19 @@ import { toDecimal, toNumber } from '../../lib/money.js'; import { todayAppDateString } from '../../lib/timezone.js'; import { formatDateToYmd, epochMsToUtcYmd } from '../../lib/dateFormat.js'; +/** + * @typedef {import('../../types/rows.js').ExchangeRateRow} ExchangeRateRow + * @typedef {import('../../types/rows.js').HistoricalRatePoint} HistoricalRatePoint + * @typedef {import('../../types/rows.js').HistoricalRateIndex} HistoricalRateIndex + * @typedef {import('../../types/rows.js').RateTable} RateTable + */ + +/** + * A parsed ECB history feed: 'YYYY-MM-DD' → that day's rate table. + * + * @typedef {Map} RatesByDate + */ + const ECB_LATEST_URL = 'https://www.ecb.europa.eu/stats/eurofxref/eurofxref-daily.xml'; const ERAR_LATEST_URL = 'https://open.er-api.com/v6/latest/EUR'; const ECB_HISTORY_90D_URL = 'https://www.ecb.europa.eu/stats/eurofxref/eurofxref-hist-90d.xml'; @@ -26,7 +39,9 @@ export const CACHE_LIFETIME_MS = 24 * 60 * 60 * 1000; // 24 hours export const HISTORICAL_FULL_CACHE_IDLE_MS = 60 * 60 * 1000; // 1 hour // { byDate: Map, timestamp } +/** @type {{ byDate: RatesByDate, timestamp: number } | null} */ let historicalEcb90dCache = null; +/** @type {{ byDate: RatesByDate, timestamp: number } | null} */ let historicalEcbFullCache = null; /** @type {ReturnType | null} */ let historicalEcbFullEvictTimer = null; @@ -54,6 +69,17 @@ function scheduleHistoricalFullEviction() { // ─── Date helpers ───────────────────────────────────────────────────────────── +/** + * Coerce whatever a caller has for a date into a 'YYYY-MM-DD' calendar day. + * + * Accepts the two shapes that actually reach it: a pg DATE/TIMESTAMPTZ value + * (a `Date`) and an already-stringy day (`to_char` projections, request query + * params). Anything else is stringified and matched against the leading + * YYYY-MM-DD, so unparseable input yields `null` rather than throwing. + * + * @param {string|Date|null|undefined} dateValue + * @returns {string|null} 'YYYY-MM-DD', or null when the input is empty/unparseable + */ export function normalizeDateInput(dateValue) { if (!dateValue) return null; // pg returns DATE columns as a local-midnight JS Date, whose String() form is @@ -75,8 +101,12 @@ export function normalizeDateInput(dateValue) { * Parse ECB daily/historical XML into a { EUR: 1, USD: x, ... } rates map. * ECB publishes EUR->X rates; we store X->EUR (1 / eurToX). * Handles both single-quoted and double-quoted attributes. + * + * @param {string} xmlText + * @returns {RateTable|null} null when the document contained no usable rates */ function parseEcbXml(xmlText) { + /** @type {RateTable} */ const rates = { EUR: 1.0 }; const q = `['"]`; const currencyPattern = new RegExp( @@ -93,7 +123,14 @@ function parseEcbXml(xmlText) { return Object.keys(rates).length > 1 ? rates : null; } +/** + * Split an ECB history document into per-day rate tables. + * + * @param {string} xmlText + * @returns {RatesByDate} + */ function parseEcbHistoricalXml(xmlText) { + /** @type {RatesByDate} */ const byDate = new Map(); const dayBlocks = xmlText.match(//g) || []; for (const block of dayBlocks) { @@ -111,6 +148,8 @@ function parseEcbHistoricalXml(xmlText) { /** * Fetch the latest rates from the ECB daily feed. * Returns a { EUR: 1, USD: x, ... } map (X->EUR), or null on failure. + * + * @returns {Promise} */ export async function fetchFromEcb() { try { @@ -134,6 +173,8 @@ export async function fetchFromEcb() { /** * Fetch the latest rates from open.er-api.com (supplementary source). * Returns a { EUR: 1, USD: x, ... } map (X->EUR), or null on failure. + * + * @returns {Promise} */ export async function fetchFromErApi() { try { @@ -147,6 +188,7 @@ export async function fetchFromErApi() { logger.error('Unexpected response from open.er-api', { result: data.result }); return null; } + /** @type {RateTable} */ const rates = { EUR: 1.0 }; for (const [currency, eurToX] of Object.entries(data.rates)) { if ( @@ -166,6 +208,11 @@ export async function fetchFromErApi() { } } +/** + * The last ~90 days of ECB reference rates, memoised for CACHE_LIFETIME_MS. + * + * @returns {Promise} empty map when the feed is unreachable + */ export async function fetchHistoricalFromEcb90d() { if (historicalEcb90dCache && Date.now() - historicalEcb90dCache.timestamp < CACHE_LIFETIME_MS) { return historicalEcb90dCache.byDate; @@ -182,6 +229,12 @@ export async function fetchHistoricalFromEcb90d() { } } +/** + * The full ECB reference-rate history (daily, back to 1999), memoised for + * CACHE_LIFETIME_MS and idle-evicted by {@link scheduleHistoricalFullEviction}. + * + * @returns {Promise} empty map when the feed is unreachable + */ export async function fetchHistoricalFromEcbFull() { if (historicalEcbFullCache && Date.now() - historicalEcbFullCache.timestamp < CACHE_LIFETIME_MS) { scheduleHistoricalFullEviction(); // touch: keep a live cache from being evicted @@ -242,6 +295,8 @@ export function rateOnOrBeforeFromMap(byDate, currencyCode, dateStr, maxLookback /** * Load the stored latest rates from the database. * Returns null if no rows exist. + * + * @returns {Promise} null when nothing is stored or the query failed */ export async function loadFromDatabase() { try { @@ -250,8 +305,9 @@ export async function loadFromDatabase() { ); if (result.rows.length === 0) return null; + /** @type {RateTable} */ const rates = { EUR: 1.0 }; - for (const row of result.rows) { + for (const row of /** @type {Pick[]} */ (result.rows)) { rates[row.currency_code] = toNumber(toDecimal(row.rate_to_eur)); } logger.debug(`Loaded ${result.rows.length} exchange rates from database`); @@ -265,6 +321,9 @@ export async function loadFromDatabase() { /** * Replace stored rates with the freshly-fetched set. * Single transaction: clear latest markers, then upsert new rates. + * + * @param {RateTable} rates + * @returns {Promise} */ export async function saveToDatabase(rates) { try { @@ -302,6 +361,14 @@ export async function saveToDatabase(rates) { } } +/** + * Upsert one historical (non-latest) rate row. + * + * @param {string} currencyCode + * @param {string} dateStr 'YYYY-MM-DD' + * @param {number} rateToEur + * @returns {Promise} + */ export async function saveHistoricalRate(currencyCode, dateStr, rateToEur) { await query( `INSERT INTO exchange_rates (currency_code, rate_to_eur, rate_date, is_latest) @@ -314,6 +381,14 @@ export async function saveHistoricalRate(currencyCode, dateStr, rateToEur) { ); } +/** + * Nearest stored rate for a currency at ANY distance from the date — the last + * resort of {@link getRateToEurForDate}. + * + * @param {string} currencyCode + * @param {string} dateStr 'YYYY-MM-DD' + * @returns {Promise} undefined when nothing is stored for the currency + */ export async function getNearestRateFromDatabase(currencyCode, dateStr) { const result = await query( `SELECT rate_to_eur @@ -359,7 +434,22 @@ export async function getStoredRateToEurOnOrBefore(currencyCode, dateValue, maxL // ─── Historical rate index (in-memory binary search) ───────────────────────── +/** + * Group stored `exchange_rates` rows into a per-currency, date-ascending index + * for binary search. + * + * `rate_date` is typed loosely on purpose: the two call sites project it + * differently — `getHistoricalRateIndex` selects the raw DATE (so pg hands back + * a local-midnight `Date`) while `loadHistoricalRateIndex` in + * portfolioSummaryService projects `to_char(rate_date, 'YYYY-MM-DD')` (a + * string). `normalizeDateInput` accepts both, so the index is identical either + * way. + * + * @param {Array & { rate_date: Date|string }>} rows + * @returns {HistoricalRateIndex} + */ export function buildHistoricalRateIndex(rows) { + /** @type {HistoricalRateIndex} */ const byCurrency = new Map(); for (const row of rows) { const currency = String(row.currency_code || '').toUpperCase().trim(); @@ -406,6 +496,15 @@ function searchRateIndex(index, currencyCode, dateStr, resolve) { return resolve(hi >= 0 ? entries[hi] : null, lo < entries.length ? entries[lo] : null); } +/** + * Rate for a currency at the date, or the closest published date on EITHER + * side when the exact day is absent. + * + * @param {HistoricalRateIndex} index + * @param {string} currencyCode + * @param {string} dateStr 'YYYY-MM-DD' + * @returns {number|undefined} + */ export function findNearestRateInIndex(index, currencyCode, dateStr) { return searchRateIndex(index, currencyCode, dateStr, (prev, next) => { if (!prev) return next?.rate; @@ -421,6 +520,11 @@ export function findNearestRateInIndex(index, currencyCode, dateStr) { * Like {@link findNearestRateInIndex} but strictly ON-or-BEFORE the date — * the standard FX convention (a Saturday uses Friday's close, never Monday's). * Returns undefined when no rate exists on or before the date. + * + * @param {HistoricalRateIndex} index + * @param {string} currencyCode + * @param {string} dateStr 'YYYY-MM-DD' + * @returns {number|undefined} */ export function findRateOnOrBeforeInIndex(index, currencyCode, dateStr) { return searchRateIndex(index, currencyCode, dateStr, (prev) => (prev ? prev.rate : undefined)); @@ -428,6 +532,15 @@ export function findRateOnOrBeforeInIndex(index, currencyCode, dateStr) { // ─── Historical rate point lookup ───────────────────────────────────────────── +/** + * Point lookup of a currency's rate on a specific day, walking the tiers: + * exact stored row → 90-day ECB feed → full ECB history → nearest stored row. + * + * @param {string} currencyCode + * @param {string|Date|null|undefined} dateValue + * @param {{ saveFetchedHistoricalRate?: boolean }} [options] persist rates sourced from ECB (default true) + * @returns {Promise} undefined when the date is unparseable or nothing resolves + */ export async function getRateToEurForDate(currencyCode, dateValue, { saveFetchedHistoricalRate = true } = {}) { if (!currencyCode || currencyCode === 'EUR') return 1.0; const dateStr = normalizeDateInput(dateValue); diff --git a/apps/node-backend/src/services/dataImportService.js b/apps/node-backend/src/services/dataImportService.js index 149bccc4..b9ebd6e6 100644 --- a/apps/node-backend/src/services/dataImportService.js +++ b/apps/node-backend/src/services/dataImportService.js @@ -62,6 +62,10 @@ async function safeReadCsv(filePath, encoding) { export async function importRecipientsCSV(filePath, { separator = ',', encoding = 'utf-8' } = {}) { const content = await safeReadCsv(filePath, encoding); + // csv-parse's `columns: true` overload is generic (`T = unknown`) and infers + // `{}` with no column list supplied — annotate to the actual shape + // (header name -> cell string) so downstream property/index access typechecks. + /** @type {Record[]} */ let records; try { records = parse(content, { @@ -80,6 +84,7 @@ export async function importRecipientsCSV(filePath, { separator = ',', encoding for (const record of records) { const rowKeys = Object.keys(record); + /** @param {string} name */ const col = (name) => { const key = rowKeys.find(k => k.toLowerCase().trim() === name); return key ? (record[key] ?? '').trim() : ''; @@ -167,6 +172,7 @@ export async function importRecipientsCSV(filePath, { separator = ',', encoding export async function importCategoriesCSV(filePath, { separator = ',', encoding = 'utf-8' } = {}) { const content = await safeReadCsv(filePath, encoding); + /** @type {Record[]} */ let records; try { records = parse(content, { diff --git a/apps/node-backend/src/services/dbEditor.js b/apps/node-backend/src/services/dbEditor.js index c4ea9761..34ffee73 100644 --- a/apps/node-backend/src/services/dbEditor.js +++ b/apps/node-backend/src/services/dbEditor.js @@ -31,6 +31,73 @@ import { NotFoundError, } from '../middleware/errorHandler.js'; +/** @typedef {import('../types/rows.js').QueryRunner} QueryRunner */ + +/** + * Column metadata for one table column, as introspected from + * `information_schema.columns` + the primary-key query. Genuinely dynamic — + * this editor works against any public table, so there is no fixed row shape + * to type against; `ColumnMeta`/`TableMeta` describe the editor's own + * bookkeeping, not the tables it edits. + * @typedef {object} ColumnMeta + * @property {string} name + * @property {string} dataType + * @property {string} udtName + * @property {boolean} nullable + * @property {boolean} hasDefault + * @property {boolean} generated + * @property {boolean} writable + */ + +/** + * @typedef {object} TableMeta + * @property {string} table + * @property {ColumnMeta[]} columns + * @property {string[]} primaryKey + */ + +/** + * A structured filter clause from the browse UI (see the ADR-101 note on + * `readRows` — the raw-WHERE escape hatch was removed). + * @typedef {object} Filter + * @property {string} column + * @property {string} [op] one of FILTER_OPS; defaults to 'eq'. + * @property {unknown} [value] + */ + +/** + * A pending row edit from the DB editor UI. `values`/`set`/`pk` are raw + * column-name → value maps — genuinely dynamic (any editable table, any + * column set), typed as `Record` rather than a fixed shape. + * @typedef {object} Change + * @property {'insert'|'update'|'delete'} op + * @property {Record} [values] insert only. + * @property {Record} [set] update only — changed columns. + * @property {Record} [pk] update/delete — primary-key column values identifying the row. + * @property {unknown} [xmin] optimistic-concurrency token (the row's `xmin`) from the row the client loaded. + */ + +/** + * Per-change context threaded through the mutation builders. + * @typedef {object} MutationCtx + * @property {Map} colMeta + * @property {string[]} primaryKey + * @property {number} index + */ + +/** + * One row of the `db_editor_audit` sink (both the DB table and the + * structured-logger mirror) — mirrors whatever table/row was edited, so + * `pk`/`before`/`after` are the same dynamic row shape as `Change`. + * @typedef {object} AuditEntry + * @property {string} table + * @property {string} op + * @property {Record|undefined} pk + * @property {Record|undefined} before + * @property {Record|undefined} after + * @property {string} statement + */ + const DEFAULT_PAGE_SIZE = 100; const MAX_PAGE_SIZE = 500; const READ_TIMEOUT_MS = 15_000; @@ -46,12 +113,26 @@ const FILTER_OPS = new Set([ // ── Identifier safety ─────────────────────────────────────────────────────── +/** + * @param {unknown} name + * @returns {string} + */ function quoteIdent(name) { return `"${String(name).replace(/"/g, '""')}"`; } +/** + * @param {unknown} value + * @param {number} fallback + * @param {number} min + * @param {number} max + * @returns {number} + */ function clampInt(value, fallback, min, max) { - const n = Number.parseInt(value, 10); + // String(value): Number.parseInt already ToStrings a non-string argument + // internally (same algorithm), so this is a no-op for behavior — it only + // satisfies parseInt's string-typed signature. + const n = Number.parseInt(String(value), 10); if (!Number.isFinite(n)) return fallback; return Math.min(Math.max(n, min), max); } @@ -72,11 +153,15 @@ async function listUserTables() { `SELECT relname FROM pg_stat_user_tables WHERE schemaname = 'public'`, [], ); - const set = new Set(r.rows.map((row) => row.relname)); + const set = new Set(r.rows.map((/** @type {{ relname: string }} */ row) => row.relname)); userTablesCache = { value: set, expiresAt: now + META_TTL_MS }; return set; } +/** + * @param {unknown} table + * @returns {Promise} + */ async function assertEditableTable(table) { if (typeof table !== 'string' || table.length === 0) { throw new ValidationError('Table name is required'); @@ -92,6 +177,7 @@ async function assertEditableTable(table) { /** * Column + primary-key metadata for a public table. * @param {string} table + * @returns {Promise} */ export async function getTableMeta(table) { await assertEditableTable(table); @@ -121,7 +207,8 @@ export async function getTableMeta(table) { ), ]); - const columns = colsRes.rows.map((r) => { + /** @typedef {{ column_name: string, data_type: string, udt_name: string, is_nullable: string, column_default: string|null, is_generated: string, is_identity: string, ordinal_position: number }} RawColumnRow */ + const columns = (/** @type {RawColumnRow[]} */ (colsRes.rows)).map((r) => { const generatedAlways = r.is_generated === 'ALWAYS' || r.is_identity === 'YES'; return { name: r.column_name, @@ -135,13 +222,23 @@ export async function getTableMeta(table) { }; }); - const meta = { table, columns, primaryKey: pkRes.rows.map((r) => r.column_name) }; + const meta = { + table, + columns, + primaryKey: (/** @type {{ column_name: string }[]} */ (pkRes.rows)).map((r) => r.column_name), + }; tableMetaCache.set(table, { value: meta, expiresAt: now + META_TTL_MS }); return meta; } // ── Reads (browse / filter / sort / paginate) ─────────────────────────────── +/** + * @param {Filter} filter + * @param {unknown[]} params + * @param {Set} columnNames + * @returns {string} + */ function buildFilterFragment(filter, params, columnNames) { if (!columnNames.has(filter.column)) { throw new ValidationError(`Unknown filter column: ${filter.column}`); @@ -163,7 +260,9 @@ function buildFilterFragment(filter, params, columnNames) { params.push(`${filter.value}%`); return `${col}::text ILIKE $${params.length}`; default: { - const sqlOp = { eq: '=', ne: '<>', lt: '<', lte: '<=', gt: '>', gte: '>=' }[op]; + /** @type {Record} */ + const sqlOpByOp = { eq: '=', ne: '<>', lt: '<', lte: '<=', gt: '>', gte: '>=' }; + const sqlOp = sqlOpByOp[op]; params.push(filter.value); return `${col} ${sqlOp} $${params.length}`; } @@ -180,8 +279,8 @@ function buildFilterFragment(filter, params, columnNames) { * the rest of the statement past the `;` guard. * @param {string} table * @param {{limit?:number, offset?:number, orderBy?:string, dir?:string, - * filters?:Array<{column:string,op?:string,value?:unknown}>, - * where?:string}} opts `where` is accepted only to be rejected (400) — + * filters?:Filter[], + * where?:string}} [opts] `where` is accepted only to be rejected (400) — * the raw-WHERE escape hatch was removed. */ export async function readRows(table, opts = {}) { @@ -191,7 +290,9 @@ export async function readRows(table, opts = {}) { const limit = clampInt(opts.limit, DEFAULT_PAGE_SIZE, 1, MAX_PAGE_SIZE); const offset = clampInt(opts.offset, 0, 0, Number.MAX_SAFE_INTEGER); + /** @type {unknown[]} */ const params = []; + /** @type {string[]} */ const whereParts = []; for (const filter of opts.filters ?? []) { @@ -249,6 +350,10 @@ export async function readRows(table, opts = {}) { // ── Mutation building ─────────────────────────────────────────────────────── +/** + * @param {unknown} value + * @returns {string} + */ function literalForDisplay(value) { if (value === undefined || value === null) return 'NULL'; if (typeof value === 'number') return String(value); @@ -257,10 +362,19 @@ function literalForDisplay(value) { return `'${String(value).replace(/'/g, "''")}'`; } +/** + * @param {string} sql + * @param {unknown[]} params + * @returns {string} + */ function renderPreview(sql, params) { return sql.replace(/\$(\d+)/g, (_, n) => literalForDisplay(params[Number(n) - 1])); } +/** + * @param {unknown} value + * @returns {unknown} + */ function normalizeWrite(value) { return value === undefined ? null : value; } @@ -268,7 +382,12 @@ function normalizeWrite(value) { /** * Build the primary mutation statement (INSERT/UPDATE/DELETE) for a change. * Used both for dry-run previews and for execution, so the SQL shown to the - * user is exactly what runs. Returns { sql, params }. + * user is exactly what runs. + * + * @param {string} table + * @param {Change} change + * @param {MutationCtx} ctx + * @returns {{ sql: string, params: unknown[] }} */ function buildMutationSql(table, change, ctx) { const tbl = quoteIdent(table); @@ -310,6 +429,7 @@ function buildMutationSql(table, change, ctx) { throw new ValidationError(`Column "${c}" is generated and cannot be written`); } } + /** @type {unknown[]} */ const params = []; const assigns = cols.map((c) => { params.push(normalizeWrite(change.set[c])); @@ -324,6 +444,7 @@ function buildMutationSql(table, change, ctx) { } if (change.op === 'delete') { + /** @type {unknown[]} */ const params = []; const where = ctx.primaryKey.map((k) => { params.push(change.pk[k]); @@ -335,6 +456,11 @@ function buildMutationSql(table, change, ctx) { throw new ValidationError(`Unknown op: ${change.op}`); } +/** + * @param {Change} change + * @param {MutationCtx} ctx + * @returns {void} + */ function validateChange(change, ctx) { if (!change || typeof change !== 'object') { throw new ValidationError(`Change #${ctx.index} is malformed`); @@ -353,12 +479,25 @@ function validateChange(change, ctx) { // ── Mutation execution ────────────────────────────────────────────────────── +/** + * @param {Record|null|undefined} row + * @param {string[]} primaryKey + * @returns {Record} + */ function pickPk(row, primaryKey) { + /** @type {Record} */ const pk = {}; for (const k of primaryKey) pk[k] = row?.[k]; return pk; } +/** + * @param {QueryRunner} client + * @param {string} table + * @param {Change} change + * @param {MutationCtx} ctx + * @returns {Promise<{ op: string, after?: Record, audit: AuditEntry }>} + */ async function applyOne(client, table, change, ctx) { // INSERT: no row to lock; structural constraints enforced by Postgres. if (change.op === 'insert') { @@ -374,6 +513,7 @@ async function applyOne(client, table, change, ctx) { // UPDATE / DELETE: lock the row, verify its version, then mutate. const tbl = quoteIdent(table); + /** @type {unknown[]} */ const lockParams = []; const pkWhere = ctx.primaryKey.map((k) => { lockParams.push(change.pk[k]); @@ -408,6 +548,11 @@ async function applyOne(client, table, change, ctx) { return { op: 'update', after, audit: { table, op: 'update', pk: change.pk, before, after, statement: renderPreview(sql, params) } }; } +/** + * @param {QueryRunner} client + * @param {AuditEntry[]} audit + * @returns {Promise} + */ async function writeAuditRows(client, audit) { for (const a of audit) { await client.query( @@ -424,7 +569,7 @@ async function writeAuditRows(client, audit) { * transaction (all-or-nothing) and writes an audit row per change. * * @param {string} table - * @param {Array} changes + * @param {Change[]} changes * @param {{dryRun?:boolean}} [opts] */ export async function applyMutations(table, changes, { dryRun = false } = {}) { @@ -436,6 +581,7 @@ export async function applyMutations(table, changes, { dryRun = false } = {}) { throw new ValidationError('No changes provided'); } + /** @type {Map} */ const colMeta = new Map(columns.map((c) => [c.name, c])); const statements = changes.map((change, index) => { @@ -450,11 +596,13 @@ export async function applyMutations(table, changes, { dryRun = false } = {}) { } const client = await getClient(); + /** @type {AuditEntry[]} */ const audit = []; try { await client.query('BEGIN'); await client.query(`SET LOCAL statement_timeout = ${WRITE_TIMEOUT_MS}`); + /** @type {Array<{ op: string, after: Record|undefined }>} */ const results = []; for (let index = 0; index < changes.length; index++) { const ctx = { colMeta, primaryKey, index }; @@ -488,6 +636,9 @@ export async function applyMutations(table, changes, { dryRun = false } = {}) { /** * Translate raw Postgres error codes into friendly, typed API errors so the * UI can show "Column X cannot be null" instead of an opaque SQLSTATE. + * + * @param {any} err raw pg driver error — shape (code/detail/column/constraint) is upstream-defined. + * @returns {AppError} */ function mapDbError(err) { if (err instanceof AppError) return err; diff --git a/apps/node-backend/src/services/deduplication.js b/apps/node-backend/src/services/deduplication.js index 27be8e2b..0cabcd1e 100644 --- a/apps/node-backend/src/services/deduplication.js +++ b/apps/node-backend/src/services/deduplication.js @@ -6,11 +6,24 @@ import crypto from 'crypto'; import { query } from '../database/connection.js'; import { logger } from '../config/logger.js'; +/** + * @typedef {object} FieldHashInput + * @property {Date} date genuine UTC-instant Date (see contract note below). + * @property {number|string} amount + * @property {string} [recipient] + * @property {string} [memo] + * @property {string} [rawData] + */ + // `transactionData.date` must be a genuine UTC-instant Date (e.g. from the // import pipeline's parseDateFlexibleUtc) — `.toISOString()` extracts its UTC // calendar day. Do NOT pass a pg-read DATE column here: those parse as // local-midnight Date objects (see lib/dateFormat.js) and would day-shift the // hash on any host east of UTC. +/** + * @param {FieldHashInput} transactionData + * @returns {string} + */ export function createTransactionHash(transactionData) { let raw = transactionData.rawData; if (!raw) { @@ -19,14 +32,30 @@ export function createTransactionHash(transactionData) { return crypto.createHash('sha256').update(raw, 'utf-8').digest('hex'); } +/** + * @typedef {object} ManualHashInput + * @property {string|Date} date + * @property {number|string} amount + * @property {number|string|null} [recipientId] + * @property {string|null} [memo] + * @property {string|null} [bankAccount] + */ + /** * Create a hash for a manually added transaction. + * + * @param {ManualHashInput} input + * @returns {string} */ export function createManualTransactionHash({ date, amount, recipientId, memo, bankAccount }) { const raw = `manual|${date}|${amount}|${recipientId}|${(memo || '').toUpperCase()}|${(bankAccount || '').toUpperCase()}`; return crypto.createHash('sha256').update(raw, 'utf-8').digest('hex'); } +/** + * @param {FieldHashInput} transactionData + * @returns {Promise} + */ export async function isDuplicate(transactionData) { // Same UTC-instant contract as createTransactionHash above: // transactionData.date must be a genuine UTC-instant Date, not a pg-read @@ -58,6 +87,13 @@ export async function isDuplicate(transactionData) { return result.rows.length > 0; } +/** + * @param {string} date + * @param {number|string} amount + * @param {string} [recipientName] + * @param {string} [memo] + * @returns {Promise} + */ export async function isDuplicateByFields(date, amount, recipientName, memo) { const result = await query( `SELECT id FROM transactions t @@ -72,7 +108,9 @@ export async function isDuplicateByFields(date, amount, recipientName, memo) { /** * Check if a manually added transaction is a duplicate using the manual_raw_transactions table. - * Returns { isDuplicate: boolean, existingTransactionId: number|null } + * + * @param {ManualHashInput} input + * @returns {Promise<{ isDuplicate: boolean, existingTransactionId: number|null }>} */ export async function isManualDuplicate({ date, amount, recipientId, memo, bankAccount }) { const hash = createManualTransactionHash({ date, amount, recipientId, memo, bankAccount }); @@ -118,6 +156,9 @@ export async function isManualDuplicate({ date, amount, recipientId, memo, bankA /** * Record a manually added transaction in the raw table for future dedup. + * + * @param {ManualHashInput & { categoryId?: number|string|null, comment?: string|null, transactionId: number|string }} input + * @returns {Promise} */ export async function recordManualRawTransaction({ date, amount, recipientId, memo, bankAccount, categoryId, comment, transactionId }) { const hash = createManualTransactionHash({ date, amount, recipientId, memo, bankAccount }); diff --git a/apps/node-backend/src/services/importPipeline/adapters/_shared.js b/apps/node-backend/src/services/importPipeline/adapters/_shared.js index d8b40107..1c6f91f7 100644 --- a/apps/node-backend/src/services/importPipeline/adapters/_shared.js +++ b/apps/node-backend/src/services/importPipeline/adapters/_shared.js @@ -10,6 +10,35 @@ import fs from 'fs'; import { parse } from 'csv-parse/sync'; import { toDecimal } from '../../../lib/money.js'; +/** + * The row shape every bank CSV adapter emits and `stage.js` persists into + * `import_staging`. Adapters differ only in how they GET here; the shape itself + * is a contract shared by the whole pipeline. + * + * @typedef {object} ParsedBankTransaction + * @property {Date} date UTC-midnight (see parseDateFlexibleUtc) — never a local-midnight Date. + * @property {string} bankAccount canonical IBAN ({@link canonicalIban}) or the bank literal when the export carries no account. + * @property {string} recipient uppercased, cleaned counterparty name; 'UNKNOWN' when absent. + * @property {string} memo + * @property {number} amount signed — negative is an outflow. + * @property {string|null} currency ISO-4217 ({@link normalizeIsoCurrency}); null when the CSV had none, and commit defaults it to EUR. + * @property {number|null} balance the export's stamped running balance; null when the export carries none (ADR-094). + * @property {string|null} recipientAccount + * @property {string|null} recipientAddress + * @property {string|null} recipientBankName + * @property {string|null} comment + * @property {string} rawData the source line/record, kept for dedup + provenance. + * @property {[number, number]} [_seq] belfius only: (statement no, transaction no); consumed and stripped by its applyRunningBalances. + */ + +/** + * A parsed transaction list carrying the adapter's own count of rows it could + * not interpret. The counter rides on the array (rather than a wrapper object) + * because that is the shape every adapter's `parse` has always returned. + * + * @typedef {ParsedBankTransaction[] & { skipped?: number }} ParsedBankTransactions + */ + /** * Parse an already-cleaned numeric string into a number via the canonical * decimal path. Returns NaN for empty or non-numeric input. Use instead of @@ -63,6 +92,13 @@ export async function readTextWithEncodingFallback(filePath) { return utf8; } +/** + * Parse a `DD/MM/YYYY` cell into a UTC-midnight Date, rejecting out-of-range + * components rather than letting Date.UTC roll them over. + * + * @param {string} dateStr + * @returns {Date|null} + */ export function parseDayMonthYear(dateStr) { const dateParts = String(dateStr).split('/'); if (dateParts.length !== 3) return null; @@ -126,6 +162,12 @@ export function parseDateWithFormat(dateStr, fmt) { // (Date.UTC(2024, 24, 12) → 2026-01-12 for a MM/DD file parsed as %d/%m/%Y), // and a 2-digit year like "24" becomes 1924. Reject instead of importing a // wrong day — matches parseDayMonthYear's validation. + /** + * @param {number} y + * @param {number} m 1-based month + * @param {number} d + * @returns {Date|null} + */ const build = (y, m, d) => { if (!Number.isFinite(y) || !Number.isFinite(m) || !Number.isFinite(d)) return null; if (y < 100) return null; // 2-digit-year misparse (e.g. "24" → 1924) @@ -159,6 +201,13 @@ export function parseDateWithFormat(dateStr, fmt) { return parseDateFlexibleUtc(dateStr); } +/** + * Parse an EU-formatted decimal cell ("1.234,56"). A dot-decimal cell without a + * comma is left untouched. + * + * @param {unknown} value + * @returns {number} NaN when the cell isn't numeric + */ export function parseCommaDecimal(value) { const s = String(value).replace(/\s/g, ''); // EU format: comma is the decimal separator and dots are thousands separators. @@ -174,6 +223,9 @@ export function parseCommaDecimal(value) { /** * Robust amount parser that handles both EU (1.234,56) and US (1,234.56) * formats, currency symbols, parenthetical negatives, and leading sign. + * + * @param {unknown} raw + * @returns {number} NaN when the cell isn't numeric */ export function parseAmountField(raw) { let s = String(raw || '').trim(); @@ -214,6 +266,12 @@ export function parseAmountField(raw) { const UTF8_BOM_RE = /^\uFEFF/; +/** + * Split file content into physical lines, stripping a leading UTF-8 BOM. + * + * @param {unknown} content + * @returns {string[]} + */ export function splitCsvLines(content) { // Strip the UTF-8 BOM (U+FEFF) that Excel and several Windows tools // prepend to exported CSVs. Without this, the first header byte leaks @@ -247,6 +305,12 @@ export function splitDelimitedRecord(line, delimiter = ';') { } } +/** + * Join the adapter's collected comment fragments, or null when there are none. + * + * @param {string[]} commentParts + * @returns {string|null} + */ export function buildOptionalComment(commentParts) { return commentParts.length ? commentParts.join(' | ') : null; } @@ -266,15 +330,30 @@ export function canonicalIban(value) { } /** + * Read and parse a CSV file with csv-parse. + * + * The element type genuinely depends on `options`: with `columns: true` (what + * every record-based adapter passes) csv-parse yields `Record` + * objects, otherwise `string[]` tuples. csv-parse's own signature only models + * the tuple case, so this is typed `any[]` and each adapter states the shape it + * asked for on its own row handler. + * * @param {string} filePath - * @param {object} options + * @param {object} options csv-parse options * @param {BufferEncoding} [encoding] + * @returns {Promise} */ export async function parseCsvFile(filePath, options, encoding = 'utf-8') { const content = await fs.promises.readFile(filePath, encoding); return parse(content, options); } +/** + * Flatten a parsed CSV record into the `rawData` provenance string. + * + * @param {Record} row a `columns: true` csv-parse record + * @returns {string} + */ export function buildRawRowString(row) { return Object.values(row).join('|'); } diff --git a/apps/node-backend/src/services/importPipeline/adapters/belfius.js b/apps/node-backend/src/services/importPipeline/adapters/belfius.js index d5e0aadf..5fd6a5ad 100644 --- a/apps/node-backend/src/services/importPipeline/adapters/belfius.js +++ b/apps/node-backend/src/services/importPipeline/adapters/belfius.js @@ -8,12 +8,21 @@ import { logger } from '../../../config/logger.js'; import { parseDayMonthYear, parseCommaDecimal, buildOptionalComment, splitCsvLines, splitDelimitedRecord, canonicalIban, readTextWithEncodingFallback } from './_shared.js'; import { toDecimal, roundMoney } from '../../../lib/money.js'; +/** + * @typedef {import('./_shared.js').ParsedBankTransaction} ParsedBankTransaction + * @typedef {import('./_shared.js').ParsedBankTransactions} ParsedBankTransactions + */ + const NAME = 'belfius'; const BANK_LABEL = 'Belfius'; const HEADER_ROWS = 13; const BALANCE_LINE_INDEX = 9; const MIN_FIELDS = 12; +/** + * @param {string} line the statement's "Laatste saldo;" header line + * @returns {number|null} + */ function parseLastBalance(line) { if (!line.includes('Laatste saldo;')) return null; const parts = line.split(';'); @@ -25,6 +34,14 @@ function parseLastBalance(line) { return isNaN(val) ? null : val; } +/** + * Walk the statement's closing balance backwards over the rows, stamping each + * one's `balance`. Mutates `transactions` in place. + * + * @param {ParsedBankTransaction[]} transactions + * @param {number|null} lastBalance + * @returns {void} + */ function applyRunningBalances(transactions, lastBalance) { if (lastBalance === null || transactions.length === 0) return; @@ -55,6 +72,10 @@ function applyRunningBalances(transactions, lastBalance) { } } +/** + * @param {string} line one ';'-delimited statement record + * @returns {ParsedBankTransaction|null} null when the line is too short or unparseable + */ function parseTransactionLine(line) { const parts = splitDelimitedRecord(line); if (!parts || parts.length < MIN_FIELDS) return null; @@ -113,6 +134,10 @@ function parseTransactionLine(line) { }; } +/** + * @param {string|null|undefined} csvSample raw head of the uploaded file + * @returns {boolean} + */ export function detect(csvSample) { if (!csvSample) return false; const lines = splitCsvLines(csvSample).slice(0, 15); @@ -120,10 +145,14 @@ export function detect(csvSample) { || lines.some((line) => /^BE\d{2}/.test(line.trim()) && line.split(';').length >= MIN_FIELDS); } +/** + * @param {string} filePath + * @returns {Promise} + */ export async function parse(filePath) { const content = await readTextWithEncodingFallback(filePath); const lines = splitCsvLines(content); - const transactions = /** @type {any[] & { skipped?: number }} */ ([]); + const transactions = /** @type {ParsedBankTransactions} */ ([]); const lastBalance = lines.length > BALANCE_LINE_INDEX ? parseLastBalance(lines[BALANCE_LINE_INDEX].trim()) : null; diff --git a/apps/node-backend/src/services/importPipeline/adapters/bnp.js b/apps/node-backend/src/services/importPipeline/adapters/bnp.js index e285f658..d1e9b4bf 100644 --- a/apps/node-backend/src/services/importPipeline/adapters/bnp.js +++ b/apps/node-backend/src/services/importPipeline/adapters/bnp.js @@ -24,10 +24,19 @@ import { cleanRecipientName, normalizeToUppercase } from '../../../lib/textNorma import { logger } from '../../../config/logger.js'; import { parseDayMonthYear, parseAmountField, buildOptionalComment, splitCsvLines, splitDelimitedRecord, canonicalIban, readTextWithEncodingFallback } from './_shared.js'; +/** + * @typedef {import('./_shared.js').ParsedBankTransaction} ParsedBankTransaction + * @typedef {import('./_shared.js').ParsedBankTransactions} ParsedBankTransactions + */ + const NAME = 'bnp'; const BANK_LABEL = 'BNP Paribas Fortis'; const MIN_FIELDS = 9; +/** + * @param {string} line + * @returns {boolean} + */ function isHeaderLine(line) { return line.includes('Volgnummer') && line.includes('Uitvoeringsdatum'); } @@ -39,11 +48,20 @@ function isHeaderLine(line) { // silently drop a real transaction). NL/FR/EN refusal + cancellation stems. const NON_EXECUTED_STATUS_RE = /geweiger|geannuleer|annulering|refus|annul|reject|cancel/i; +/** + * @param {string} status the export's status column + * @param {string} rejectionReason non-empty means the payment was refused + * @returns {boolean} + */ function isNonExecutedRow(status, rejectionReason) { if (rejectionReason) return true; return NON_EXECUTED_STATUS_RE.test(status); } +/** + * @param {string} line one ';'-delimited statement record + * @returns {ParsedBankTransaction|null} null when too short, non-executed, or unparseable + */ function parseLine(line) { const parts = splitDelimitedRecord(line); if (!parts || parts.length < MIN_FIELDS) return null; @@ -96,6 +114,10 @@ function parseLine(line) { }; } +/** + * @param {string|null|undefined} csvSample raw head of the uploaded file + * @returns {boolean} + */ export function detect(csvSample) { if (!csvSample) return false; const lines = splitCsvLines(csvSample).slice(0, 3); @@ -106,10 +128,14 @@ export function detect(csvSample) { ); } +/** + * @param {string} filePath + * @returns {Promise} + */ export async function parse(filePath) { const content = await readTextWithEncodingFallback(filePath); const lines = splitCsvLines(content); - const transactions = /** @type {any[] & { skipped?: number }} */ ([]); + const transactions = /** @type {ParsedBankTransactions} */ ([]); let skipped = 0; let headerSeen = false; diff --git a/apps/node-backend/src/services/importPipeline/adapters/generic.js b/apps/node-backend/src/services/importPipeline/adapters/generic.js index a452c016..ddae3b50 100644 --- a/apps/node-backend/src/services/importPipeline/adapters/generic.js +++ b/apps/node-backend/src/services/importPipeline/adapters/generic.js @@ -7,12 +7,40 @@ import { logger } from '../../../config/logger.js'; import { normalizeToUppercase } from '../../../lib/textNormalization.js'; import { parseCsvFile, buildRawRowString, parseAmountField, SUPPORTED_DATE_FORMATS, parseDateWithFormat, normalizeIsoCurrency } from './_shared.js'; +/** + * @typedef {import('./_shared.js').ParsedBankTransaction} ParsedBankTransaction + * @typedef {import('./_shared.js').ParsedBankTransactions} ParsedBankTransactions + */ + +/** + * The custom-parser definition a `generic` import runs on. + * + * It arrives either from POST /api/import/csv (which builds it from form + * fields, so only bank_name/date_format and the date/recipient/amount column + * names are guaranteed) or from a saved `custom_parser_configs.config_json` + * row, which is free-form JSONB. Nothing re-validates it here, so every field + * beyond `column_mapping` is optional. + * + * @typedef {object} CustomTransactionParserConfig + * @property {string} [bank_name] defaults to 'CUSTOM' + * @property {string} [account_type] appended to the bank name to form the ADR-088 account label + * @property {string} [date_format] must be one of SUPPORTED_DATE_FORMATS + * @property {string} [separator] CSV delimiter; defaults to ',' + * @property {number} [skip_rows] leading rows to drop before the header + * @property {BufferEncoding} [encoding] defaults to 'utf-8' + * @property {{ date: string, recipient: string, amount: string, memo?: string, currency?: string, balance?: string }} column_mapping source column NAMES, not indices + */ + const NAME = 'generic'; const BANK_LABEL = 'Generic'; // Normalize to UPPER+trim so the custom/generic adapter matches every built-in adapter and the // manual-entry path (transactionRepository.create uppercases bank_account) — otherwise the same // bank reached two ways resolves to two different accounts (ADR-088 account identity). +/** + * @param {CustomTransactionParserConfig} config + * @returns {string} + */ function buildBankAccount(config) { const bankName = config.bank_name || 'CUSTOM'; const accountType = config.account_type; @@ -20,6 +48,11 @@ function buildBankAccount(config) { return normalizeToUppercase(label); } +/** + * @param {Record} row a `columns: true` csv-parse record + * @param {CustomTransactionParserConfig} config + * @returns {ParsedBankTransaction|null} null when the mapped date or amount is unusable + */ function rowToTransaction(row, config) { const colMap = config.column_mapping; const dateStr = String(row[colMap.date] || '').trim(); @@ -61,6 +94,12 @@ function rowToTransaction(row, config) { }; } +/** + * @param {string} filePath + * @param {CustomTransactionParserConfig} config + * @returns {Promise} + * @throws {Error} when `date_format` is not one of SUPPORTED_DATE_FORMATS + */ export async function parseWithConfig(filePath, config) { const dateFormat = config.date_format || ''; if (!SUPPORTED_DATE_FORMATS.includes(dateFormat)) { @@ -84,7 +123,7 @@ export async function parseWithConfig(filePath, config) { config.encoding || 'utf-8', ); - const transactions = /** @type {any[] & { skipped?: number }} */ ([]); + const transactions = /** @type {ParsedBankTransactions} */ ([]); let skipped = 0; for (const row of records) { try { @@ -103,11 +142,19 @@ export async function parseWithConfig(filePath, config) { return transactions; } +/** + * @returns {boolean} always false — the generic adapter is the explicit fallback + */ export function detect() { // Generic adapter is the fallback; never auto-detected. return false; } +/** + * @param {string} filePath + * @param {CustomTransactionParserConfig} [config] required — the generic adapter has no built-in mapping + * @returns {Promise} + */ export async function parse(filePath, config) { if (!config) { throw new Error('Generic adapter requires a customConfig'); diff --git a/apps/node-backend/src/services/importPipeline/adapters/index.js b/apps/node-backend/src/services/importPipeline/adapters/index.js index 1cc2d8af..def51c04 100644 --- a/apps/node-backend/src/services/importPipeline/adapters/index.js +++ b/apps/node-backend/src/services/importPipeline/adapters/index.js @@ -16,16 +16,46 @@ import sabb from './sabb.js'; import wise from './wise.js'; import generic from './generic.js'; +/** + * @typedef {import('./_shared.js').ParsedBankTransactions} ParsedBankTransactions + * @typedef {import('./generic.js').CustomTransactionParserConfig} CustomTransactionParserConfig + */ + +/** + * The interface every adapter module default-exports. + * + * `parseWithConfig` is optional because only `generic` has one — the pipeline + * feature-detects it (see stage.js) rather than branching on the adapter name. + * `parse`'s second parameter is likewise generic-only. + * + * @typedef {object} BankCsvAdapter + * @property {string} name internal key, e.g. 'bnp' + * @property {string} bankName display label, e.g. 'BNP Paribas Fortis' + * @property {(csvSample?: string|null) => boolean} detect + * @property {(filePath: string, config?: any) => Promise} parse + * @property {(filePath: string, config: any) => Promise} [parseWithConfig] + */ + +/** @type {BankCsvAdapter[]} */ const ADAPTERS = [belfius, revolut, ing, bnp, kbc, vision, sabb, wise, generic]; const REGISTRY = new Map(ADAPTERS.map((adapter) => [adapter.name, adapter])); +/** + * Look an adapter up by internal name (case- and whitespace-insensitive). + * + * @param {string|null|undefined} name + * @returns {BankCsvAdapter|null} + */ export function getAdapter(name) { if (!name) return null; const key = String(name).toLowerCase().replace(/\s+/g, '_'); return REGISTRY.get(key) || null; } +/** + * @returns {string[]} internal adapter names, excluding the generic fallback + */ export function getSupportedBanks() { // Mirrors legacy order for UI selects. Exclude generic (internal fallback). return ADAPTERS @@ -36,7 +66,10 @@ export function getSupportedBanks() { /** * Single source of truth for the frontend adapter catalog: { key, name } per * non-generic adapter, derived from the registry so adding an adapter exposes it + * in the UI automatically (no separate hardcoded list to drift). + * + * @returns {Array<{ key: string, name: string }>} */ export function listAdapters() { return ADAPTERS @@ -65,17 +98,20 @@ export function detectBank(csvSample) { * Legacy-compatible factory: returns a `(filePath) => transactions[]` callable. * * @param {string} bankName — display name or internal name - * @param {object | null} customConfig — when present, uses generic adapter + + * @param {CustomTransactionParserConfig | null} customConfig — when present, uses generic adapter + * @returns {(filePath: string) => Promise} + * @throws {Error} when no adapter matches `bankName` and no customConfig was given */ export function createAdapter(bankName, customConfig = null) { if (customConfig) { - return (filePath) => generic.parseWithConfig(filePath, customConfig); + return (/** @type {string} */ filePath) => generic.parseWithConfig(filePath, customConfig); } const adapter = getAdapter(bankName); if (!adapter) { throw new Error(`No configuration found for bank: ${bankName}`); } - return (filePath) => adapter.parse(filePath); + return (/** @type {string} */ filePath) => adapter.parse(filePath); } export { ADAPTERS, REGISTRY }; diff --git a/apps/node-backend/src/services/importPipeline/adapters/ing.js b/apps/node-backend/src/services/importPipeline/adapters/ing.js index 83de5c3d..643ac148 100644 --- a/apps/node-backend/src/services/importPipeline/adapters/ing.js +++ b/apps/node-backend/src/services/importPipeline/adapters/ing.js @@ -20,14 +20,27 @@ import { normalizeToUppercase } from '../../../lib/textNormalization.js'; import { logger } from '../../../config/logger.js'; import { parseDayMonthYear, parseCommaDecimal, buildOptionalComment, splitCsvLines, splitDelimitedRecord, canonicalIban, readTextWithEncodingFallback } from './_shared.js'; +/** + * @typedef {import('./_shared.js').ParsedBankTransaction} ParsedBankTransaction + * @typedef {import('./_shared.js').ParsedBankTransactions} ParsedBankTransactions + */ + const NAME = 'ing'; const BANK_LABEL = 'ING'; const MIN_FIELDS = 9; +/** + * @param {string} line + * @returns {boolean} + */ function isHeaderLine(line) { return line.includes('Omzetnummer') && line.includes('Boekingsdatum'); } +/** + * @param {string} line one ';'-delimited statement record + * @returns {ParsedBankTransaction|null} null when too short or unparseable + */ function parseLine(line) { const parts = splitDelimitedRecord(line); if (!parts || parts.length < MIN_FIELDS) return null; @@ -72,6 +85,10 @@ function parseLine(line) { }; } +/** + * @param {string|null|undefined} csvSample raw head of the uploaded file + * @returns {boolean} + */ export function detect(csvSample) { if (!csvSample) return false; const lines = splitCsvLines(csvSample).slice(0, 3); @@ -80,10 +97,14 @@ export function detect(csvSample) { ); } +/** + * @param {string} filePath + * @returns {Promise} + */ export async function parse(filePath) { const content = await readTextWithEncodingFallback(filePath); const lines = splitCsvLines(content); - const transactions = /** @type {any[] & { skipped?: number }} */ ([]); + const transactions = /** @type {ParsedBankTransactions} */ ([]); let skipped = 0; let headerSeen = false; diff --git a/apps/node-backend/src/services/importPipeline/adapters/kbc.js b/apps/node-backend/src/services/importPipeline/adapters/kbc.js index d6e4fab1..07aee496 100644 --- a/apps/node-backend/src/services/importPipeline/adapters/kbc.js +++ b/apps/node-backend/src/services/importPipeline/adapters/kbc.js @@ -7,9 +7,19 @@ import { logger } from '../../../config/logger.js'; import { parseDayMonthYear, parseCommaDecimal, buildOptionalComment, splitCsvLines, splitDelimitedRecord, canonicalIban, readTextWithEncodingFallback } from './_shared.js'; const NAME = 'kbc'; +/** + * @typedef {import('./_shared.js').ParsedBankTransaction} ParsedBankTransaction + * @typedef {import('./_shared.js').ParsedBankTransactions} ParsedBankTransactions + */ + const BANK_LABEL = 'KBC'; const MIN_FIELDS = 15; +/** + * @param {string|undefined} creditStr + * @param {string|undefined} debitStr + * @returns {'CREDIT'|'DEBIT'|null} null when neither column holds a non-zero amount + */ function classifyTransactionType(creditStr, debitStr) { if (creditStr && creditStr.trim()) { const cv = parseCommaDecimal(creditStr); @@ -22,6 +32,10 @@ function classifyTransactionType(creditStr, debitStr) { return null; } +/** + * @param {string} line one ','-delimited statement record + * @returns {ParsedBankTransaction|null} null when too short or unparseable + */ function parseLine(line) { const parts = splitDelimitedRecord(line); if (!parts || parts.length < MIN_FIELDS) return null; @@ -80,12 +94,20 @@ function parseLine(line) { }; } +/** + * @param {string} line + * @returns {boolean} + */ function isNonDataLine(line) { return line.startsWith('Rekeningnummer') || line.includes('Vrije Mededeling') || line.startsWith(',,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,'); } +/** + * @param {string|null|undefined} csvSample raw head of the uploaded file + * @returns {boolean} + */ export function detect(csvSample) { if (!csvSample) return false; const lines = splitCsvLines(csvSample).slice(0, 5); @@ -93,10 +115,14 @@ export function detect(csvSample) { || lines.some((line) => line.includes('Vrije Mededeling')); } +/** + * @param {string} filePath + * @returns {Promise} + */ export async function parse(filePath) { const content = await readTextWithEncodingFallback(filePath); const lines = splitCsvLines(content); - const transactions = /** @type {any[] & { skipped?: number }} */ ([]); + const transactions = /** @type {ParsedBankTransactions} */ ([]); let skipped = 0; for (const rawLine of lines) { diff --git a/apps/node-backend/src/services/importPipeline/adapters/revolut.js b/apps/node-backend/src/services/importPipeline/adapters/revolut.js index bdc17f57..50993b8d 100644 --- a/apps/node-backend/src/services/importPipeline/adapters/revolut.js +++ b/apps/node-backend/src/services/importPipeline/adapters/revolut.js @@ -7,16 +7,29 @@ import { logger } from '../../../config/logger.js'; import { parseCsvFile, buildOptionalComment, parseDecimalSafe, parseDateFlexibleUtc } from './_shared.js'; import { toDecimal, roundMoney } from '../../../lib/money.js'; +/** + * @typedef {import('./_shared.js').ParsedBankTransaction} ParsedBankTransaction + * @typedef {import('./_shared.js').ParsedBankTransactions} ParsedBankTransactions + */ + const NAME = 'revolut'; const BANK_LABEL = 'Revolut'; const MIN_FIELDS = 10; +/** + * @param {string} completedDateStr + * @returns {Date|null} UTC-midnight Date + */ function parseRevolutDate(completedDateStr) { // "YYYY-MM-DD HH:MM:SS" and plain ISO are both handled by the shared parser; // any other shape is rebuilt at UTC midnight rather than local (day-shift). return parseDateFlexibleUtc(completedDateStr); } +/** + * @param {string|null|undefined} product Revolut's "Product" column (CURRENT / SAVINGS / …) + * @returns {string} the ADR-088 account label + */ function buildBankAccount(product) { const upper = (product || '').toUpperCase(); if (upper === 'SAVINGS') return 'REVOLUT SAVINGS'; @@ -24,6 +37,14 @@ function buildBankAccount(product) { return `REVOLUT ${upper}`.trim(); } +/** + * Rebuild the source record with both date columns normalized, so `rawData` + * (and the dedup hash derived from it) is stable across export variants. + * + * @param {string[]} parts + * @param {string} normalizedDate + * @returns {string} + */ function buildNormalizedRawData(parts, normalizedDate) { const normalized = [...parts]; normalized[2] = normalizedDate; @@ -33,6 +54,10 @@ function buildNormalizedRawData(parts, normalizedDate) { .join(','); } +/** + * @param {string[]} parts one split CSV record + * @returns {ParsedBankTransaction|null} null when too short or unparseable + */ function parseRow(parts) { if (parts.length < MIN_FIELDS) return null; @@ -90,6 +115,10 @@ function parseRow(parts) { }; } +/** + * @param {string|null|undefined} csvSample raw head of the uploaded file + * @returns {boolean} + */ export function detect(csvSample) { if (!csvSample) return false; const firstLine = csvSample.split('\n')[0] || ''; @@ -97,13 +126,17 @@ export function detect(csvSample) { return lower.startsWith('type,') && lower.includes('completed date') && lower.includes('state'); } +/** + * @param {string} filePath + * @returns {Promise} + */ export async function parse(filePath) { const records = await parseCsvFile(filePath, { columns: false, skip_empty_lines: true, relax_column_count: true, }); - const transactions = /** @type {any[] & { skipped?: number }} */ ([]); + const transactions = /** @type {ParsedBankTransactions} */ ([]); let skipped = 0; for (let i = 0; i < records.length; i++) { diff --git a/apps/node-backend/src/services/importPipeline/adapters/sabb.js b/apps/node-backend/src/services/importPipeline/adapters/sabb.js index c0382507..a91e0ef5 100644 --- a/apps/node-backend/src/services/importPipeline/adapters/sabb.js +++ b/apps/node-backend/src/services/importPipeline/adapters/sabb.js @@ -6,6 +6,11 @@ import { cleanRecipientName, normalizeToUppercase } from '../../../lib/textNorma import { logger } from '../../../config/logger.js'; import { parseCsvFile, buildOptionalComment, buildRawRowString, parseAmountField, parseDayMonthYear, parseDateFlexibleUtc } from './_shared.js'; +/** + * @typedef {import('./_shared.js').ParsedBankTransaction} ParsedBankTransaction + * @typedef {import('./_shared.js').ParsedBankTransactions} ParsedBankTransactions + */ + const NAME = 'sabb'; const BANK_LABEL = 'SABB'; @@ -17,6 +22,10 @@ const BANK_LABEL = 'SABB'; // silently dropping a settled transaction. const NON_COMPLETED_STATUS_RE = /pending|declin|reject|refus|revers|fail|cancel/i; +/** + * @param {Record} row a `columns: true` csv-parse record + * @returns {ParsedBankTransaction|null} null for a non-settled status or an unusable row + */ function rowToTransaction(row) { const status = (row['Status'] || '').trim(); if (NON_COMPLETED_STATUS_RE.test(status)) return null; @@ -74,12 +83,20 @@ function rowToTransaction(row) { }; } +/** + * @param {string|null|undefined} csvSample raw head of the uploaded file + * @returns {boolean} + */ export function detect(csvSample) { if (!csvSample) return false; const firstLine = (csvSample.split('\n')[0] || '').toLowerCase(); return firstLine.includes('transaction date') && firstLine.includes('amount(sar)'); } +/** + * @param {string} filePath + * @returns {Promise} + */ export async function parse(filePath) { const records = await parseCsvFile(filePath, { columns: true, @@ -88,7 +105,7 @@ export async function parse(filePath) { trim: true, }); - const transactions = /** @type {any[] & { skipped?: number }} */ ([]); + const transactions = /** @type {ParsedBankTransactions} */ ([]); let skipped = 0; for (const row of records) { try { diff --git a/apps/node-backend/src/services/importPipeline/adapters/vision.js b/apps/node-backend/src/services/importPipeline/adapters/vision.js index 176740bb..84f7e2bd 100644 --- a/apps/node-backend/src/services/importPipeline/adapters/vision.js +++ b/apps/node-backend/src/services/importPipeline/adapters/vision.js @@ -6,9 +6,18 @@ import { cleanRecipientName, normalizeToUppercase } from '../../../lib/textNorma import { logger } from '../../../config/logger.js'; import { parseCsvFile, buildOptionalComment, buildRawRowString, parseAmountField, parseDateFlexibleUtc, normalizeIsoCurrency } from './_shared.js'; +/** + * @typedef {import('./_shared.js').ParsedBankTransaction} ParsedBankTransaction + * @typedef {import('./_shared.js').ParsedBankTransactions} ParsedBankTransactions + */ + const NAME = 'vision'; const BANK_LABEL = 'Vision'; +/** + * @param {Record} row a `columns: true` csv-parse record + * @returns {ParsedBankTransaction|null} null when the row is unusable (no date / non-numeric amount) + */ function rowToTransaction(row) { const dateStr = (row['Date'] || '').trim(); if (!dateStr) return null; @@ -68,6 +77,10 @@ function rowToTransaction(row) { // extended exports still detect. const EXPORT_HEADER_PREFIX = ['date', 'bank account', 'recipient', 'memo', 'amount', 'currency']; +/** + * @param {string|null|undefined} csvSample raw head of the uploaded file + * @returns {boolean} + */ export function detect(csvSample) { if (!csvSample) return false; // Strip a UTF-8 BOM — detect() receives raw file content, not the @@ -77,6 +90,10 @@ export function detect(csvSample) { return EXPORT_HEADER_PREFIX.every((name, i) => cols[i] === name); } +/** + * @param {string} filePath + * @returns {Promise} + */ export async function parse(filePath) { const records = await parseCsvFile(filePath, { columns: true, @@ -85,7 +102,7 @@ export async function parse(filePath) { trim: true, }); - const transactions = /** @type {any[] & { skipped?: number }} */ ([]); + const transactions = /** @type {ParsedBankTransactions} */ ([]); let skipped = 0; for (const row of records) { try { diff --git a/apps/node-backend/src/services/importPipeline/adapters/wise.js b/apps/node-backend/src/services/importPipeline/adapters/wise.js index b149708d..fc6c0ff5 100644 --- a/apps/node-backend/src/services/importPipeline/adapters/wise.js +++ b/apps/node-backend/src/services/importPipeline/adapters/wise.js @@ -6,9 +6,19 @@ import { cleanRecipientName, normalizeToUppercase } from '../../../lib/textNorma import { logger } from '../../../config/logger.js'; import { parseCsvFile, buildOptionalComment, buildRawRowString, parseAmountField, parseDateFlexibleUtc } from './_shared.js'; +/** + * @typedef {import('./_shared.js').ParsedBankTransaction} ParsedBankTransaction + * @typedef {import('./_shared.js').ParsedBankTransactions} ParsedBankTransactions + */ + const NAME = 'wise'; const BANK_LABEL = 'Wise'; +/** + * @param {string} amountStr + * @param {string} direction Wise's 'IN' / 'OUT' / 'NEUTRAL' column + * @returns {number|null} signed amount, or null when unparseable + */ function resolveAmount(amountStr, direction) { const parsed = parseAmountField(amountStr); if (isNaN(parsed)) return null; @@ -17,6 +27,15 @@ function resolveAmount(amountStr, direction) { return parsed; } +/** + * @param {Record} row + * @param {string} direction + * @param {string} sourceAmountStr + * @param {string} targetAmountStr + * @param {string} sourceCurrency + * @param {string} targetCurrency + * @returns {string|null} + */ function buildWiseComment(row, direction, sourceAmountStr, targetAmountStr, sourceCurrency, targetCurrency) { const sourceFee = (row['Source fee amount'] || '').trim(); const sourceFeeCurrency = (row['Source fee currency'] || '').trim(); @@ -36,6 +55,10 @@ function buildWiseComment(row, direction, sourceAmountStr, targetAmountStr, sour return buildOptionalComment(commentParts); } +/** + * @param {Record} row a `columns: true` csv-parse record + * @returns {ParsedBankTransaction|null} null unless the row is COMPLETED and parseable + */ function rowToTransaction(row) { const status = (row['Status'] || '').trim().toUpperCase(); if (status !== 'COMPLETED') return null; @@ -98,6 +121,10 @@ function rowToTransaction(row) { }; } +/** + * @param {string|null|undefined} csvSample raw head of the uploaded file + * @returns {boolean} + */ export function detect(csvSample) { if (!csvSample) return false; const firstLine = (csvSample.split('\n')[0] || '').toLowerCase(); @@ -106,6 +133,10 @@ export function detect(csvSample) { && firstLine.includes('source amount'); } +/** + * @param {string} filePath + * @returns {Promise} + */ export async function parse(filePath) { const records = await parseCsvFile(filePath, { columns: true, @@ -114,7 +145,7 @@ export async function parse(filePath) { trim: true, }); - const transactions = /** @type {any[] & { skipped?: number }} */ ([]); + const transactions = /** @type {ParsedBankTransactions} */ ([]); let skipped = 0; for (const row of records) { try { diff --git a/apps/node-backend/src/services/importPipeline/brokerageRouting.js b/apps/node-backend/src/services/importPipeline/brokerageRouting.js index 37744a66..0faba2ff 100644 --- a/apps/node-backend/src/services/importPipeline/brokerageRouting.js +++ b/apps/node-backend/src/services/importPipeline/brokerageRouting.js @@ -55,7 +55,7 @@ export function classifyBrokerageRow(row) { * @returns {string} */ export function tradeDedupKey(row) { - const norm = (v) => (v == null ? '' : String(v).trim()); + const norm = (/** @type {unknown} */ v) => (v == null ? '' : String(v).trim()); return [ norm(row.account_id), norm(row.investment_id), diff --git a/apps/node-backend/src/services/importPipeline/commit.js b/apps/node-backend/src/services/importPipeline/commit.js index 893e6f3b..83c231ea 100644 --- a/apps/node-backend/src/services/importPipeline/commit.js +++ b/apps/node-backend/src/services/importPipeline/commit.js @@ -22,8 +22,21 @@ import { formatDateToYmd } from '../../lib/dateFormat.js'; import { refreshAggregations } from '../aggregationRefresh.js'; import { autoLinkTransactions } from '../plannedMatchService.js'; +/** + * @typedef {import('../../types/rows.js').ImportStagingRow} ImportStagingRow + * @typedef {import('./index.js').ImportBatchId} ImportBatchId + * @typedef {import('./index.js').ImportProgressCallback} ImportProgressCallback + */ + const COMMIT_CHUNK = 1000; +/** + * Run the commit phase: drain 'matched' staging rows into `transactions` with + * per-row dedup and per-row SAVEPOINTs, then auto-link planned payments. + * + * @param {{ batchId: ImportBatchId, onProgress?: ImportProgressCallback }} args + * @returns {Promise<{ imported: number, duplicates: number, errors: number, autoLinkedCount: number }>} + */ export async function commitBatch({ batchId, onProgress }) { await query(`UPDATE import_batches SET status = 'committing' WHERE id = $1`, [batchId]); @@ -60,9 +73,11 @@ export async function commitBatch({ batchId, onProgress }) { // tx_hashes already written to `transactions` by this run — guards against // two identical rows inside the same CSV both passing the field-based dup // check (neither is in `transactions` yet when the first is processed). + /** @type {Set} */ const committedHashes = new Set(); // Inserted rows fed to planned-payment auto-link after the whole batch // commits (so matching sees the full import — both ambiguity directions). + /** @type {Array<{ id: number, recipient_id: number|null, amount: string|null, transaction_date: string }>} */ const insertedRows = []; if (onProgress) onProgress({ phase: 'committing', current: 0, total }); @@ -76,6 +91,7 @@ export async function commitBatch({ batchId, onProgress }) { let chunkImported = 0; let chunkDuplicates = 0; let chunkErrors = 0; + /** @type {typeof insertedRows} */ let chunkInserted = []; await withTransaction(async (client) => { // Reset inside the callback so a withTransaction retry recounts cleanly. diff --git a/apps/node-backend/src/services/importPipeline/index.js b/apps/node-backend/src/services/importPipeline/index.js index 18bb63ac..70374946 100644 --- a/apps/node-backend/src/services/importPipeline/index.js +++ b/apps/node-backend/src/services/importPipeline/index.js @@ -30,12 +30,39 @@ import { reconcileTransfers } from '../transferReconciliationService.js'; export { createBatch, stageBatch, validateBatch, matchBatch, commitBatch }; +/** + * Progress reporter threaded through every pipeline phase. `imported` / + * `duplicates` / `errors` are only supplied by the commit phase. + * + * @typedef {(progress: { + * phase: 'staging'|'validating'|'matching'|'committing', + * current: number, + * total: number, + * imported?: number, + * duplicates?: number, + * errors?: number, + * }) => void} ImportProgressCallback + */ + +/** + * An `import_batches` id as it is actually passed around. + * + * Always a NUMBER. Two producers feed it and both now agree: `createBatch` + * (stage.js) normalizes node-postgres's BIGSERIAL string at the boundary, and + * the review/commit routes parse it out of the URL through `coercedIdSchema` + * (lib/importBatchIds.js:17), which already yielded a number. It was a + * `string|number` union until the two disagreed on the wire — see the note on + * `createBatch` for why number won. + * + * @typedef {number} ImportBatchId + */ + /** * Stage → validate → match a batch that already exists. * Decides whether the batch needs user review or can be auto-committed. * - * @param {{ batchId: number, filePath: string, adapterName: string, customConfig?: object, filename?: string, sizeBytes?: number, onProgress?: Function }} args - * @returns {Promise<{ batchId: number, rowsTotal: number, requiresReview: boolean, matchSourceCounts: object, validateErrors: number }>} + * @param {{ batchId: ImportBatchId, filePath: string, adapterName: string, customConfig?: object, filename?: string, sizeBytes?: number, onProgress?: ImportProgressCallback }} args + * @returns {Promise<{ batchId: ImportBatchId, rowsTotal: number, requiresReview: boolean, matchSourceCounts: object, validateErrors: number }>} */ export async function prepareImport({ batchId, filePath, adapterName, customConfig, filename: _filename, sizeBytes: _sizeBytes, onProgress }) { const { rowsTotal } = await stageBatch({ batchId, filePath, adapterName, customConfig, onProgress }); @@ -71,7 +98,7 @@ export async function prepareImport({ batchId, filePath, adapterName, customConf * Commit a prepared (or reviewed) batch. * Applies user_override_recipient_id before writing transactions. * - * @param {{ batchId: number, onProgress?: Function }} args + * @param {{ batchId: ImportBatchId, onProgress?: ImportProgressCallback }} args * @returns {Promise<{ imported: number, duplicates: number, errors: number, autoLinkedCount: number }>} */ export async function commitImport({ batchId, onProgress }) { @@ -111,8 +138,8 @@ export async function commitImport({ batchId, onProgress }) { * auto-commits (all rows exact) or leaves the batch in 'awaiting_review' * for the frontend to present the ImportReviewPage. * - * @param {{ filePath: string, adapterName: string, customConfig?: object, filename?: string, sizeBytes?: number, onProgress?: Function }} args - * @returns {Promise<{ batchId: number, total: number, requiresReview: boolean, imported?: number, duplicates?: number, errors?: number, matchSourceCounts?: object, autoLinkedCount?: number }>} + * @param {{ filePath: string, adapterName: string, customConfig?: object, filename?: string, sizeBytes?: number, onProgress?: ImportProgressCallback }} args + * @returns {Promise<{ batchId: ImportBatchId, total: number, requiresReview: boolean, imported?: number, duplicates?: number, errors?: number, matchSourceCounts?: object, autoLinkedCount?: number }>} */ export async function runImportPipeline({ filePath, adapterName, customConfig, filename, sizeBytes, onProgress }) { const batchId = await createBatch({ adapterName, filename, sizeBytes, customConfig }); diff --git a/apps/node-backend/src/services/importPipeline/match.js b/apps/node-backend/src/services/importPipeline/match.js index 6f61f186..579a2ec7 100644 --- a/apps/node-backend/src/services/importPipeline/match.js +++ b/apps/node-backend/src/services/importPipeline/match.js @@ -24,8 +24,22 @@ import { } from '../calculations/normalization.js'; import { loadActivePatterns, applyPatterns } from '../recipientPatternService.js'; +/** + * @typedef {import('../../types/rows.js').ImportStagingRow} ImportStagingRow + * @typedef {import('./index.js').ImportBatchId} ImportBatchId + * @typedef {import('./index.js').ImportProgressCallback} ImportProgressCallback + */ + const MATCH_UPDATE_CHUNK = 500; +/** + * Run the match phase: resolve every validated row's `recipient_raw` to a + * recipient id via patterns → exact/fuzzy → new-recipient upsert, and stamp + * the provenance columns the review UI reads. + * + * @param {{ batchId: ImportBatchId, onProgress?: ImportProgressCallback }} args + * @returns {Promise<{ matched: number, unresolved: number, matchSourceCounts: Record }>} + */ export async function matchBatch({ batchId, onProgress }) { await query(`UPDATE import_batches SET status = 'matching' WHERE id = $1`, [batchId]); @@ -40,8 +54,13 @@ export async function matchBatch({ batchId, onProgress }) { const total = staged.length; if (onProgress) onProgress({ phase: 'matching', current: 0, total }); + /** @type {string[]} */ const distinctRaw = [ - ...new Set(staged.map((r) => r.recipient_raw).filter((n) => n && String(n).trim().length)), + ...new Set( + /** @type {Pick[]} */ (staged) + .map((r) => r.recipient_raw) + .filter((n) => n && String(n).trim().length), + ), ]; // --- Phase 1: pattern match --- @@ -92,7 +111,9 @@ export async function matchBatch({ batchId, onProgress }) { RETURNING id, normalized_name`, [upperNames, normalizedNames], ); - const insertedByNorm = new Map(inserted.rows.map((r) => [r.normalized_name, r.id])); + const insertedByNorm = new Map( + /** @type {{ id: number, normalized_name: string }[]} */ (inserted.rows).map((r) => [r.normalized_name, r.id]), + ); // Fetch ids for names that already existed (conflict — not returned above). const conflicted = normalizedNames.filter((n) => !insertedByNorm.has(n)); @@ -101,7 +122,9 @@ export async function matchBatch({ batchId, onProgress }) { `SELECT id, normalized_name FROM recipients WHERE normalized_name = ANY($1::text[])`, [conflicted], ); - for (const r of existing.rows) insertedByNorm.set(r.normalized_name, r.id); + for (const r of /** @type {{ id: number, normalized_name: string }[]} */ (existing.rows)) { + insertedByNorm.set(r.normalized_name, r.id); + } } for (const { raw, normalized } of toUpsert) { @@ -156,6 +179,7 @@ export async function matchBatch({ batchId, onProgress }) { if (onProgress) onProgress({ phase: 'matching', current: seen, total }); } + /** @type {Record} */ const matchSourceCounts = { pattern: 0, exact: 0, fuzzy: 0, new: 0 }; for (const info of resolved.values()) matchSourceCounts[info.matchSource] = (matchSourceCounts[info.matchSource] || 0) + 1; diff --git a/apps/node-backend/src/services/importPipeline/stage.js b/apps/node-backend/src/services/importPipeline/stage.js index 032da8ed..0b5eadad 100644 --- a/apps/node-backend/src/services/importPipeline/stage.js +++ b/apps/node-backend/src/services/importPipeline/stage.js @@ -13,10 +13,34 @@ import { parsedDateToYmd } from '../../lib/importDates.js'; import { getAdapter } from './adapters/index.js'; import generic from './adapters/generic.js'; +/** + * @typedef {import('../../types/rows.js').ImportStagingRow} ImportStagingRow + * @typedef {import('./index.js').ImportBatchId} ImportBatchId + * @typedef {import('./index.js').ImportProgressCallback} ImportProgressCallback + */ + const STAGE_INSERT_CHUNK = 500; /** - * Create a new import batch row. Returns its id. + * Create a new import batch row. + * + * @param {{ adapterName: string, filename?: string|null, sizeBytes?: number|null, customConfig?: object|null }} args + * @returns {Promise} the new batch id, as a NUMBER. + * + * `import_batches.id` is BIGSERIAL and node-postgres hands BIGINT back as a + * STRING, so this used to leak a string all the way to the wire: POST + * /api/import/csv answered `batch_id: "12"` while the review-commit route + * (routes/importRoutes.js:570), which reads the id back off the URL through + * `coercedIdSchema` (lib/importBatchIds.js:17), answered `batch_id: 12` — + * same JSON field, two types, so strict-equality across the two responses + * broke. Normalizing here, at the single boundary where the id enters the + * application, makes NUMBER the one wire type; it matches the coerced input + * schema and the frontend runtime guards (`batch_id: z.number()` in + * apps/frontend/src/lib/api/imports.ts). + * + * Safe for this app: BIGSERIAL starts at 1 and increments per CSV import, so + * reaching 2^53 is not physically attainable. If that ever changes, the fix + * is to make the WIRE type a string everywhere, not to reintroduce the split. */ export async function createBatch({ adapterName, filename, sizeBytes, customConfig }) { const result = await query( @@ -26,11 +50,15 @@ export async function createBatch({ adapterName, filename, sizeBytes, customConf RETURNING id`, [adapterName, filename || null, sizeBytes || null, customConfig ? JSON.stringify(customConfig) : null] ); - return result.rows[0].id; + return Number(result.rows[0].id); } /** * Run the stage phase: parse the file, bulk-insert staging rows. + * + * @param {{ batchId: ImportBatchId, filePath: string, adapterName: string, customConfig?: object|null, onProgress?: ImportProgressCallback }} args + * @returns {Promise<{ rowsTotal: number, rowsSkipped: number }>} `rowsSkipped` is + * the adapter's own count of data rows it could not interpret. */ export async function stageBatch({ batchId, filePath, adapterName, customConfig, onProgress }) { await query( @@ -73,10 +101,21 @@ export async function stageBatch({ batchId, filePath, adapterName, customConfig, return { rowsTotal: total, rowsSkipped: skipped }; } +/** + * Bulk-insert one chunk of parsed rows as `import_staging_rows` (status + * 'pending') in a single multi-VALUES statement. + * + * @param {ImportBatchId} batchId + * @param {import('./adapters/_shared.js').ParsedBankTransaction[]} rows + * @param {number} startIndex the chunk's offset, written to `row_index` + * @returns {Promise} + */ async function insertStagingChunk(batchId, rows, startIndex) { if (!rows.length) return; await withTransaction(async (client) => { + /** @type {any[]} */ const values = []; + /** @type {string[]} */ const placeholders = []; rows.forEach((r, i) => { const idx = startIndex + i; diff --git a/apps/node-backend/src/services/importPipeline/validate.js b/apps/node-backend/src/services/importPipeline/validate.js index 25433662..85ad9af6 100644 --- a/apps/node-backend/src/services/importPipeline/validate.js +++ b/apps/node-backend/src/services/importPipeline/validate.js @@ -13,8 +13,31 @@ import { query } from '../../database/connection.js'; import { logger } from '../../config/logger.js'; import { parsedDateToYmd } from '../../lib/importDates.js'; +/** + * @typedef {import('../../types/rows.js').ImportStagingRow} ImportStagingRow + * @typedef {import('./index.js').ImportBatchId} ImportBatchId + * @typedef {import('./index.js').ImportProgressCallback} ImportProgressCallback + */ + const VALIDATE_CHUNK = 500; +/** + * The projection validate.js reads. `tx_date` is `to_char`-ed to a + * 'YYYY-MM-DD' string rather than selected raw (see the comment below), so it + * overrides the raw DATE on {@link ImportStagingRow}. + * + * @typedef {Pick + * & { tx_date: string|null }} PendingStagingRow + */ + +/** + * Run the validate phase: reject unusable rows, hash the rest, and flag + * intra-batch duplicates. + * + * @param {{ batchId: ImportBatchId, onProgress?: ImportProgressCallback }} args + * @returns {Promise<{ validated: number, duplicates: number, errors: number }>} + */ export async function validateBatch({ batchId, onProgress }) { await query(`UPDATE import_batches SET status = 'validating' WHERE id = $1`, [batchId]); @@ -38,17 +61,22 @@ export async function validateBatch({ batchId, onProgress }) { // tx_hashes seen so far in this batch — a repeat is an intra-batch duplicate // (the same row twice in one CSV) and is dropped here rather than inserted // twice at commit time. + /** @type {Set} */ const seenHashes = new Set(); if (onProgress) onProgress({ phase: 'validating', current: 0, total }); for (let start = 0; start < total; start += VALIDATE_CHUNK) { const chunk = pending.slice(start, start + VALIDATE_CHUNK); + /** @type {string[]} */ const ids = []; + /** @type {string[]} */ const statuses = []; + /** @type {(string|null)[]} */ const txHashes = []; + /** @type {(string|null)[]} */ const errorMessages = []; - for (const row of chunk) { + for (const row of /** @type {PendingStagingRow[]} */ (chunk)) { const issue = validateRow(row); ids.push(row.id); if (issue) { @@ -102,6 +130,10 @@ export async function validateBatch({ batchId, onProgress }) { return { validated, duplicates, errors }; } +/** + * @param {PendingStagingRow} row + * @returns {string|null} the rejection reason, or null when the row is usable + */ function validateRow(row) { if (!row.tx_date) return 'missing tx_date'; if (row.amount == null) return 'missing amount'; @@ -110,6 +142,13 @@ function validateRow(row) { return null; } +/** + * sha256 of the literal source record, falling back to a + * date|amount|recipient|memo field hash when the adapter kept no raw record. + * + * @param {PendingStagingRow} row + * @returns {string} lowercase hex digest + */ function computeRowHash(row) { let raw; if (row.raw_data) { diff --git a/apps/node-backend/src/services/info/cache.js b/apps/node-backend/src/services/info/cache.js index 08e0669e..5807c6e4 100644 --- a/apps/node-backend/src/services/info/cache.js +++ b/apps/node-backend/src/services/info/cache.js @@ -15,6 +15,30 @@ export const BANK_BALANCES_CACHE_TTL_MS = 60_000; // 1min — live SQL, short TT export const STATISTICS_CACHE_TTL_MS = 300_000; // 5min safety net export const MAX_CACHE_ENTRIES = 100; +/** + * One entry of an /api/info TTL cache. + * + * `data` holds the last resolved response, `inflight` the in-progress promise + * that de-dupes concurrent misses (undefined when settled), and `expiresAt` is + * an epoch-ms deadline — 0 while a cold miss is in flight. + * + * The payload type differs per cache (net-worth response, performance + * response, portfolio summary, statistics pivots) and these helpers never look + * inside it, so it is deliberately `any` rather than a guessed union. + * + * @typedef {object} InfoCacheEntry + * @property {any} data + * @property {Promise|undefined} inflight + * @property {number} expiresAt + */ + +/** + * A keyed TTL cache of {@link InfoCacheEntry}. Keys are the request-varying + * dimension as a string (target currency, or a composed cache key). + * + * @typedef {Map} InfoCache + */ + export const netWorthResponseCache = new Map(); export const perfResponseCache = new Map(); export const portfolioSummaryCache = new Map(); @@ -46,6 +70,13 @@ export function invalidateStatisticsCaches() { statisticsResponseCache.clear(); } +/** + * Drop settled entries whose TTL has elapsed. In-flight entries are kept so a + * concurrent miss can still join them. + * + * @param {InfoCache} cache + * @returns {void} + */ function pruneExpiredCacheEntries(cache) { const now = Date.now(); for (const [key, value] of cache.entries()) { @@ -56,6 +87,14 @@ function pruneExpiredCacheEntries(cache) { } } +/** + * Evict settled entries (insertion order) until the cache is back under + * `maxEntries`. In-flight entries are never evicted. + * + * @param {InfoCache} cache + * @param {number} [maxEntries] + * @returns {void} + */ function enforceCacheSizeLimit(cache, maxEntries = MAX_CACHE_ENTRIES) { if (cache.size <= maxEntries) return; @@ -73,6 +112,15 @@ function enforceCacheSizeLimit(cache, maxEntries = MAX_CACHE_ENTRIES) { } } +/** + * Return the cached payload when it is still within its TTL, else undefined + * (deleting the stale entry unless a refresh is already in flight). + * + * @param {InfoCache} cache + * @param {string} key + * @param {{ requireData?: boolean }} [options] `requireData` also rejects a fresh-but-empty entry + * @returns {any} the cached payload, or undefined on miss/stale + */ function getFreshCachedData(cache, key, { requireData = false } = {}) { pruneExpiredCacheEntries(cache); const cached = cache.get(key); @@ -84,6 +132,15 @@ function getFreshCachedData(cache, key, { requireData = false } = {}) { return undefined; } +/** + * Store a resolved payload with a fresh TTL, clearing any inflight marker. + * + * @param {InfoCache} cache + * @param {string} key + * @param {any} data the route's response payload — shape varies per cache + * @param {number} ttlMs + * @returns {void} + */ export function setCachedData(cache, key, data, ttlMs) { pruneExpiredCacheEntries(cache); cache.set(key, { @@ -94,6 +151,16 @@ export function setCachedData(cache, key, data, ttlMs) { enforceCacheSizeLimit(cache); } +/** + * Publish the in-flight promise so concurrent callers join it instead of + * starting a second load. + * + * @param {InfoCache} cache + * @param {string} key + * @param {Promise} inflight + * @param {{ keepPreviousData?: boolean }} [options] keep serving the previous (stale) payload while the refresh runs + * @returns {void} + */ function setInflightCache(cache, key, inflight, { keepPreviousData = false } = {}) { pruneExpiredCacheEntries(cache); const current = cache.get(key); @@ -105,6 +172,23 @@ function setInflightCache(cache, key, inflight, { keepPreviousData = false } = { enforceCacheSizeLimit(cache); } +/** + * Cache-or-load with single-flight de-duplication: serve a fresh entry, else + * join an in-flight load, else start one (caching the result and evicting the + * entry on rejection). + * + * The payload is `any` — every caller passes a differently-shaped loader and + * these helpers never inspect the value. + * + * @param {InfoCache} cache + * @param {string} key + * @param {object} options + * @param {number} options.ttlMs + * @param {boolean} [options.requireData=false] treat a fresh-but-empty entry as a miss + * @param {boolean} [options.keepPreviousData=false] keep serving the stale payload while the refresh runs + * @param {() => Promise} options.loader + * @returns {Promise} + */ export async function resolveCacheWithInflight(cache, key, { ttlMs, requireData = false, keepPreviousData = false, loader }) { const cachedData = getFreshCachedData(cache, key, { requireData }); if (cachedData !== undefined) { diff --git a/apps/node-backend/src/services/info/liveSummary.js b/apps/node-backend/src/services/info/liveSummary.js index 887c8f1b..976b58a1 100644 --- a/apps/node-backend/src/services/info/liveSummary.js +++ b/apps/node-backend/src/services/info/liveSummary.js @@ -40,7 +40,12 @@ export function resolveLiveSummary(targetCurrency = 'EUR') { */ export async function resolveLivePortfolioValue(targetCurrency = 'EUR') { try { - const summary = await resolveLiveSummary(targetCurrency); + // `resolveLiveSummary` is declared as `Promise` (the cache helper is + // payload-agnostic); narrow to the one field read here. getPortfolioSummary + // builds `totals.totalPortfolioValue` as a rounded number. + const summary = /** @type {{ totals?: { totalPortfolioValue?: number } }} */ ( + await resolveLiveSummary(targetCurrency) + ); const value = summary?.totals?.totalPortfolioValue; return Number.isFinite(value) ? value : undefined; } catch (err) { diff --git a/apps/node-backend/src/services/info/performanceHelpers.js b/apps/node-backend/src/services/info/performanceHelpers.js index baf78de1..2cc213d1 100644 --- a/apps/node-backend/src/services/info/performanceHelpers.js +++ b/apps/node-backend/src/services/info/performanceHelpers.js @@ -17,6 +17,32 @@ import { resolveCacheWithInflight, } from './cache.js'; +/** + * @typedef {import('@vision/shared-utils/money').DecimalInput} DecimalInput + * @typedef {import('../../types/rows.js').PortfolioPerformanceSnapshotRow} PortfolioPerformanceSnapshotRow + */ + +/** + * The snapshot shape this module actually receives: `getSnapshots` in + * portfolioPerformanceSnapshotService re-projects the raw row, dropping `id` / + * `computed_at` / `cumulative_inflation` / `real_return_pct` and applying `??` + * defaults — so `inflation_adjusted_value` may fall back to `value` and the + * three `*_invested` columns may fall back to the number `0` on rows written + * before those columns existed. `value_fx_neutral` becomes `undefined` (not + * null) when absent. + * + * @typedef {Pick & { + * inflation_adjusted_value: string, + * stocks_etfs_invested: string|number, + * crypto_invested: string|number, + * metals_invested: string|number, + * value_fx_neutral?: string|undefined, + * }} PerformanceSnapshot + */ + +/** @type {Record} Period key → lookback window in days. */ const PERIOD_OFFSETS = { '1m': 30, '3m': 90, @@ -25,10 +51,21 @@ const PERIOD_OFFSETS = { '3y': 1095, }; +/** + * @param {DecimalInput} value a NUMERIC column (pg string), or a `??` fallback number + * @returns {number} + */ function parseSnapshotNumber(value) { return toNumber(toDecimal(value)); } +/** + * Snapshot row → client-facing shape: NUMERIC strings become numbers and the + * DATE becomes a calendar-day string. + * + * @param {PerformanceSnapshot} snapshot + * @returns {Record} the wire shape (`value_fx_neutral` is conditionally spread in) + */ export function mapPortfolioPerformanceSnapshot(snapshot) { return { // DATE column: calendar-day string, not a raw pg Date. @@ -53,6 +90,14 @@ export function mapPortfolioPerformanceSnapshot(snapshot) { }; } +/** + * Restrict a snapshot series to the trailing window named by `period`. + * Unknown periods (including 'all') pass the series through unchanged. + * + * @param {PerformanceSnapshot[]} snapshots + * @param {string|null|undefined} period one of PERIOD_OFFSETS' keys, or 'all' + * @returns {PerformanceSnapshot[]} + */ function filterSnapshotsByPeriod(snapshots, period) { if (!period || period === 'all' || !PERIOD_OFFSETS[period]) return snapshots; const daysBack = PERIOD_OFFSETS[period]; @@ -63,6 +108,22 @@ function filterSnapshotsByPeriod(snapshots, period) { }); } +/** + * Assemble the /portfolio-performance response: cleaned + period-filtered + * series, snapshot-derived metrics overlaid with the live summary's current + * totals, the heatmap, and the per-investment breakdown. + * + * `metrics`, `heatmap`, `breakdownSummary` and `totals` are `any`/loose on + * purpose — they are re-exports of computeMetrics / computeHeatmap / + * getPortfolioSummary output, none of which is typed at its source. + * + * @param {string} targetCurrency + * @param {string} startDate 'YYYY-MM-DD' + * @param {string} endDate 'YYYY-MM-DD' + * @param {PerformanceSnapshot[]} allSnapshots full stored series, oldest first + * @param {string|null|undefined} period one of PERIOD_OFFSETS' keys, or 'all' + * @returns {Promise>} + */ export async function buildPortfolioPerformancePayload(targetCurrency, startDate, endDate, allSnapshots, period) { // Smooth isolated one-day price needles (kinesis data-quality issue) BEFORE // metrics/heatmap/series, mirroring the protection the net-worth path already @@ -110,7 +171,7 @@ export async function buildPortfolioPerformancePayload(targetCurrency, startDate snapshots, metrics, heatmap, - breakdownSummary: liveSummary.summaries.map((s) => ({ + breakdownSummary: liveSummary.summaries.map((/** @type {Record} */ s) => ({ id: s.id, name: s.name, symbol: s.symbol, diff --git a/apps/node-backend/src/services/marketLookupService.js b/apps/node-backend/src/services/marketLookupService.js index e682a8f5..d4629d01 100644 --- a/apps/node-backend/src/services/marketLookupService.js +++ b/apps/node-backend/src/services/marketLookupService.js @@ -36,10 +36,19 @@ const inFlightQuotes = new Map(); // whatever data came back instead of failing the whole request. const NO_VALIDATE = /** @type {{ validateResult: false }} */ ({ validateResult: false }); +/** + * @param {string} message + * @param {unknown} [cause] + * @returns {AppError} + */ function upstreamError(message, cause) { return new AppError(message, { status: 502, code: ApiErrorCode.BAD_GATEWAY, cause }); } +/** + * @param {unknown} url + * @returns {string|null} + */ function normalizeThumbnailUrl(url) { if (!url || typeof url !== 'string') return null; const trimmed = url.trim(); @@ -50,6 +59,10 @@ function normalizeThumbnailUrl(url) { return null; } +/** + * @param {any} thumbnail raw Yahoo `news[].thumbnail` — shape is upstream-controlled (NO_VALIDATE). + * @returns {string|null} + */ function pickBestThumbnail(thumbnail) { const resolutions = Array.isArray(thumbnail?.resolutions) ? thumbnail.resolutions : []; for (let i = resolutions.length - 1; i >= 0; i -= 1) { @@ -61,6 +74,8 @@ function pickBestThumbnail(thumbnail) { /** * Convert a range string (e.g. '1mo', '5y') to a Date for period1. + * @param {string} range + * @returns {Date} */ function rangeToDate(range) { const now = new Date(); @@ -114,6 +129,9 @@ function mapQuoteCore(q) { * only (one quote() call); the default (full) additionally fetches quoteSummary * for fundamentals/analyst data — roughly 2× the outbound calls. Returns null * when the upstream quote is unavailable. + * @param {string} sym + * @param {boolean} [basic] + * @returns {Promise} */ async function buildQuote(sym, basic) { const yahooFinance = await getYahooClient(); @@ -157,7 +175,7 @@ async function buildQuote(sym, basic) { const priceToBook = ks.priceToBook ?? q.priceToBook; const trendBuckets = s?.recommendationTrend?.trend || []; - const currentTrend = trendBuckets.find((t) => t.period === '0m') || trendBuckets[0] || null; + const currentTrend = trendBuckets.find((/** @type {any} */ t) => t.period === '0m') || trendBuckets[0] || null; const analystConsensus = currentTrend ? { strongBuy: currentTrend.strongBuy ?? 0, @@ -170,7 +188,7 @@ async function buildQuote(sym, basic) { const recentAnalystActions = (s?.upgradeDowngradeHistory?.history || []) .slice(0, 10) - .map((h) => ({ + .map((/** @type {any} */ h) => ({ date: h.epochGradeDate, firm: h.firm, toGrade: h.toGrade, @@ -198,10 +216,13 @@ async function buildQuote(sym, basic) { * outbound call; concurrent identical fetches share one in-flight promise. Only * successful (non-null) quotes are cached. Never throws — returns null so one bad * symbol can't fail a multi-symbol request. + * @param {string} sym + * @param {boolean} [basic] + * @returns {Promise} */ async function getCachedQuote(sym, basic) { const key = `${basic ? 'basic' : 'full'}:${sym}`; - const cached = quoteCache.get(key); + const cached = /** @type {object|null|undefined} */ (quoteCache.get(key)); if (cached !== undefined) return cached; const existing = inFlightQuotes.get(key); @@ -245,8 +266,8 @@ export async function searchSymbols(q) { } const items = (results.quotes || []) - .filter((r) => r.symbol) - .map((r) => ({ + .filter((/** @type {any} */ r) => r.symbol) + .map((/** @type {any} */ r) => ({ symbol: r.symbol, name: r.shortname || r.longname || r.symbol, type: r.quoteType || 'UNKNOWN', @@ -313,8 +334,8 @@ export async function getChart(symbol, { range = '1mo', interval = '1d' } = {}) if (!result) return { items: [], total: 0 }; const points = (result.quotes || []) - .filter((p) => p.close != null) - .map((p) => ({ + .filter((/** @type {any} */ p) => p.close != null) + .map((/** @type {any} */ p) => ({ time: new Date(p.date).getTime(), close: p.close, high: p.high, @@ -351,7 +372,7 @@ export async function getNews(symbols, count) { quotesCount: 0, newsCount, }, NO_VALIDATE); - return ((/** @type {any} */ (results)).news || []).map((n) => ({ + return ((/** @type {any} */ (results)).news || []).map((/** @type {any} */ n) => ({ title: n.title, link: n.link, publisher: n.publisher, diff --git a/apps/node-backend/src/services/materializedViewService.js b/apps/node-backend/src/services/materializedViewService.js index f79276aa..f3d9f17a 100644 --- a/apps/node-backend/src/services/materializedViewService.js +++ b/apps/node-backend/src/services/materializedViewService.js @@ -19,6 +19,10 @@ import { invalidateStatisticsCaches } from './info/cache.js'; * session-level SET on a dedicated client and restore the pool default before * the client is reused. */ +/** + * @param {string} sql + * @returns {Promise<{ rows: any[], rowCount: number|null }>} + */ async function runMaintenanceStatement(sql) { const client = await getClient(); try { @@ -51,7 +55,13 @@ const MATERIALIZED_VIEWS = [ export async function createMaterializedViews() { logger.info('Creating materialized views (if not exist)…'); - // 1. Monthly income / spending / net per month (last 12 months) + // 1. Monthly income / spending / net per month (last 12 months). + // Effective category resolves 3 levels (own → recipient default → PRIMARY + // recipient's default) so the MV's category grain agrees with the monthly + // live path and the transactions surfaces. Changing this definition + // requires a migration that DROPs the MV (see 0085): IF NOT EXISTS never + // redefines an existing view, so already-migrated installs keep the old SQL + // otherwise. await query(` CREATE MATERIALIZED VIEW IF NOT EXISTS mv_monthly_summary AS SELECT @@ -68,7 +78,8 @@ export async function createMaterializedViews() { COALESCE(c.general || ':' || c.detail, 'UNCATEGORISED') AS category_name FROM transactions t LEFT JOIN recipients r ON t.recipient_id = r.id - LEFT JOIN categories c ON COALESCE(t.category_id, r.default_category_id) = c.id + LEFT JOIN recipients pr ON r.primary_recipient_id = pr.id + LEFT JOIN categories c ON COALESCE(t.category_id, r.default_category_id, pr.default_category_id) = c.id WHERE t.is_active = true AND t.is_transfer = false AND t.date >= date_trunc('month', CURRENT_DATE) - interval '12 months' GROUP BY month_start, month, year, t.currency, c.id, category_name @@ -249,7 +260,9 @@ export async function refreshMaterializedViews() { export const REFRESH_DEBOUNCE_MS = 5000; export const REFRESH_MAX_WAIT_MS = 10000; +/** @type {ReturnType|null} */ let debounceTimer = null; +/** @type {number|null} */ let debounceDeadline = null; // epoch ms the current burst must flush by export function scheduleRefresh() { diff --git a/apps/node-backend/src/services/plannedExecutionService.js b/apps/node-backend/src/services/plannedExecutionService.js index d4da45f5..358a8a63 100644 --- a/apps/node-backend/src/services/plannedExecutionService.js +++ b/apps/node-backend/src/services/plannedExecutionService.js @@ -30,6 +30,13 @@ export async function executePlanned({ id, executedTransactionId, executionDate if (!existing) throw new NotFoundError(`Planned transaction ${id} not found`); const execDate = executionDate || todayAppDateString(); + /** + * Sanitized update payload for `plannedTransactionRepository.executeAndAdvance` + * — write-side values, NOT the read-side row shape: both dates go in as + * 'YYYY-MM-DD' strings (pg coerces on bind), whereas a fetched row's same + * columns come back as `Date` (see `PlannedTransactionRow` in types/rows.js). + * @type {{ is_executed: boolean, last_executed_date: string, planned_date?: string }} + */ const updateFields = { is_executed: !existing.is_recurring, last_executed_date: execDate, diff --git a/apps/node-backend/src/services/plannedMatchService.js b/apps/node-backend/src/services/plannedMatchService.js index 64227fad..f86ce363 100644 --- a/apps/node-backend/src/services/plannedMatchService.js +++ b/apps/node-backend/src/services/plannedMatchService.js @@ -28,15 +28,39 @@ const AMOUNT_TOLERANCE_PCT = 5; const DATE_WINDOW_DAYS = 5; const SUGGESTION_LOOKBACK_DAYS = 45; +/** + * The fields `matchesTolerance` reads off a planned-payment side. Deliberately + * loose (not `HydratedPlannedTransactionRow`): callers also pass the + * synthetic `{ id, amount, transaction_date, recipient_cluster_id }` shape + * built in `autoLinkTransactions`, so only the fields this predicate touches + * are required. + * @typedef {{ recipient_cluster_id?: number|string|null, amount: number|string, planned_date?: Date|string|null }} PlannedLike + */ + +/** + * The fields `matchesTolerance` reads off a transaction side — same looseness + * rationale as `PlannedLike`. + * @typedef {{ recipient_cluster_id?: number|string|null, amount: number|string, transaction_date?: Date|string|null }} TxLike + */ + // Normalize a DATE-ish value (pg Date at local midnight, or a 'YYYY-MM-DD' / // ISO string) to a plain 'YYYY-MM-DD' string. pg returns DATE columns as a JS // Date built from local Y/M/D, so local getters recover the calendar date. +/** + * @param {Date|string|null|undefined} value + * @returns {string|undefined} + */ function toYmd(value) { if (value == null) return undefined; if (value instanceof Date) return formatDateToYmd(value); return String(value).slice(0, 10); } +/** + * @param {string} a 'YYYY-MM-DD' + * @param {string} b 'YYYY-MM-DD' + * @returns {number} + */ function ymdDiffDays(a, b) { const [ay, am, ad] = a.split('-').map(Number); const [by, bm, bd] = b.split('-').map(Number); @@ -47,6 +71,10 @@ function ymdDiffDays(a, b) { * Pure predicate: does `tx` fall within tolerance of planned payment `planned`? * Both objects must carry `recipient_cluster_id`, `amount`, and a date * (`planned_date` / `transaction_date`). + * + * @param {PlannedLike} planned + * @param {TxLike} tx + * @returns {boolean} */ export function matchesTolerance(planned, tx) { if (planned?.recipient_cluster_id == null || tx?.recipient_cluster_id == null) return false; @@ -94,9 +122,11 @@ async function isAutoClearEnabled() { * date (`transaction_date` or `date`). Never throws — per-pair failures are * logged and skipped so an import/create never fails because of auto-link. * + * @param {Array<{ id: number, recipient_id?: number|null, amount: number|string, transaction_date?: Date|string|null, date?: Date|string|null }>} txRows * @returns {Promise<{ autoLinkedCount: number, links: Array<{plannedTransactionId:number, transactionId:number}> }>} */ export async function autoLinkTransactions(txRows) { + /** @type {{ autoLinkedCount: number, links: Array<{plannedTransactionId:number, transactionId:number}> }} */ const result = { autoLinkedCount: 0, links: [] }; if (!Array.isArray(txRows) || txRows.length === 0) return result; if (!(await isAutoClearEnabled())) return result; diff --git a/apps/node-backend/src/services/portfolioImportBatchService.js b/apps/node-backend/src/services/portfolioImportBatchService.js index 1c3670e7..c553e1e5 100644 --- a/apps/node-backend/src/services/portfolioImportBatchService.js +++ b/apps/node-backend/src/services/portfolioImportBatchService.js @@ -29,6 +29,10 @@ export { * Create a new investment from a staging row's symbol/name and the batch's * default asset class, then point the row at it. Used by the review "create * new holding" action. + * + * @param {{ batchId: number, rowId: number }} args + * @returns {Promise} the created investment row, or + * `undefined` when the staging row does not exist. */ export async function createInvestmentForRow({ batchId, rowId }) { const row = await getRowForInvestmentCreation({ batchId, rowId }); @@ -70,6 +74,9 @@ export async function createInvestmentForRow({ batchId, rowId }) { * transaction id — the sequences are independent, so deleting every id * through the portfolio repo removed UNRELATED trades that happened to share * a cash row's number (and left the imported cash row in the ledger). + * + * @param {number} batchId + * @returns {Promise<{ deleted: number }>} */ export async function rollbackBatch(batchId) { const rows = await getCommittedRows(batchId); diff --git a/apps/node-backend/src/services/portfolioImportPipeline/commit.js b/apps/node-backend/src/services/portfolioImportPipeline/commit.js index 162ce025..85fa4645 100644 --- a/apps/node-backend/src/services/portfolioImportPipeline/commit.js +++ b/apps/node-backend/src/services/portfolioImportPipeline/commit.js @@ -23,16 +23,50 @@ import { classifyBrokerageRow } from '../importPipeline/brokerageRouting.js'; // 500-1000 row brokerage CSV from thousands of sequential round trips to a // handful, while a per-row SAVEPOINT keeps crash-isolation (a bad row rolls back // to its savepoint without poisoning the chunk). +/** + * @typedef {import('../../types/rows.js').PortfolioImportStagingRow} PortfolioImportStagingRow + * @typedef {import('./index.js').PortfolioImportBatchId} PortfolioImportBatchId + * @typedef {import('./index.js').PortfolioImportProgressCallback} PortfolioImportProgressCallback + */ + +/** + * The projection commit.js drains. `tx_date` is `to_char`-ed to a 'YYYY-MM-DD' + * string, `investment_id` is `COALESCE(user_override_investment_id, + * resolved_investment_id)`, and the two `inv.*` columns come from the LEFT JOIN + * on `investments` (null for cash rows and unresolved instruments). + * + * @typedef {Pick + * & { + * tx_date: string|null, + * investment_id: number|null, + * asset_class: string|null, + * investment_currency: string|null, + * }} MatchedPortfolioStagingRow + */ + const COMMIT_CHUNK = 1000; // Staging stores cash magnitudes ABSOLUTE (adapter contract); the ledger sign // comes from the kind. Without this every withdrawal was credited as a // deposit (+500 instead of −500) — the sleeve error grew 2× per withdrawal. +/** + * @param {Pick} row + * @returns {number} the ledger-signed cash amount (negative for withdrawals) + */ function signedCashAmount(row) { const { direction } = classifyBrokerageRow({ kind: row.type_raw }); return (direction ?? 1) * Math.abs(Number(row.amount)); } +/** + * Run the commit phase: drain 'matched' staging rows into + * `portfolio_transactions` (and, for brokerage cash rows, `transactions`), with + * field-based dedup, an intra-batch hash guard, and a per-row SAVEPOINT. + * + * @param {{ batchId: PortfolioImportBatchId, onProgress?: PortfolioImportProgressCallback }} args + * @returns {Promise<{ imported: number, duplicates: number, errors: number }>} + */ export async function commitBatch({ batchId, onProgress }) { await query(`UPDATE portfolio_import_batches SET status = 'committing' WHERE id = $1`, [batchId]); @@ -76,13 +110,20 @@ export async function commitBatch({ batchId, onProgress }) { let imported = 0; let duplicates = 0; let errors = 0; + /** @type {Set} */ const committedHashes = new Set(); // Per-batch FX cache: the on-or-before stored-rate lookup is deterministic for // a given (currency, tx_date), so resolve each pair once instead of issuing a // fresh uncached lookup on every row — a single-currency CSV collapses ~N // lookups to one per distinct trade date. + /** @type {Map} */ const fxCache = new Map(); + /** + * @param {string|null} currency + * @param {string|null} date 'YYYY-MM-DD' + * @returns {Promise} + */ async function resolveFx(currency, date) { const key = `${String(currency || 'EUR').toUpperCase()}|${date}`; if (fxCache.has(key)) return fxCache.get(key); @@ -256,11 +297,21 @@ export async function commitBatch({ batchId, onProgress }) { // precision, so validate against a digits-only regex to keep the interpolated // identifier injection-safe. Returns null for a non-numeric id so the caller // can skip the row rather than emit an unsafe savepoint name. +/** + * @param {string|number} id + * @returns {string|null} null when the id is not digits-only (unsafe to interpolate) + */ function savepointFor(id) { const idStr = String(id); return /^\d+$/.test(idStr) ? `sp_prow_${idStr}` : null; } +/** + * @param {string|number} id + * @param {'committed'|'duplicate'|'error'} status + * @param {string|null} [message] + * @returns {Promise} + */ async function markRow(id, status, message) { await query( `UPDATE portfolio_import_staging_rows SET status = $2, error_message = $3 WHERE id = $1`, @@ -270,6 +321,11 @@ async function markRow(id, status, message) { // Field-based dedup for an external brokerage cash row, so re-importing a // statement is a no-op (cash rows have no tx_hash partial-unique of their own here). +/** + * @param {number} accountId the batch's brokerage sleeve account + * @param {MatchedPortfolioStagingRow} row + * @returns {Promise} + */ async function isCashFieldDuplicate(accountId, row) { const memo = row.note || (row.type_raw ? String(row.type_raw).toUpperCase() : 'BROKERAGE CASH'); const signed = signedCashAmount(row); @@ -294,6 +350,11 @@ async function isCashFieldDuplicate(accountId, row) { return dup.rows.length > 0; } +/** + * @param {MatchedPortfolioStagingRow} row + * @param {number|undefined} batchAccountId + * @returns {Promise} + */ async function isFieldDuplicate(row, batchAccountId) { // account_id and currency are part of the identity: the same-shaped fill on // a different account (or in a different currency) is a distinct trade, not diff --git a/apps/node-backend/src/services/portfolioImportPipeline/index.js b/apps/node-backend/src/services/portfolioImportPipeline/index.js index 0787c16e..08dd1a8a 100644 --- a/apps/node-backend/src/services/portfolioImportPipeline/index.js +++ b/apps/node-backend/src/services/portfolioImportPipeline/index.js @@ -33,6 +33,42 @@ import { commitBatch } from './commit.js'; export { createBatch, stageBatch, validateBatch, matchBatch, commitBatch }; +/** + * Progress reporter threaded through every portfolio-pipeline phase. + * + * `imported` / `duplicates` / `errors` are only supplied by the commit phase. + * + * @typedef {(progress: { + * phase: 'staging'|'validating'|'matching'|'committing', + * current: number, + * total: number, + * imported?: number, + * duplicates?: number, + * errors?: number, + * }) => void} PortfolioImportProgressCallback + */ + +/** + * A `portfolio_import_batches` id as it is actually passed around. + * + * Always a NUMBER — `createBatch` (stage.js) normalizes node-postgres's + * BIGSERIAL string at the boundary, and the review/commit routes parse it out + * of the URL through `coercedIdSchema` (lib/importBatchIds.js:17), which + * already yielded a number. Formerly a `string|number` union that let the two + * import responses disagree on the wire. + * + * @typedef {number} PortfolioImportBatchId + */ + +/** + * Stage → validate → match a batch that already exists, then decide whether it + * needs user review. + * + * @param {{ batchId: PortfolioImportBatchId, filePath: string, customConfig: object, onProgress?: PortfolioImportProgressCallback }} args + * @returns {Promise<{ batchId: PortfolioImportBatchId, rowsTotal: number, rowsSkipped: number, requiresReview: boolean, matchSourceCounts: Record, validateErrors: number }>} + * @throws {ValidationError} when the column mapping / date format parsed zero rows + */ + export async function prepareImport({ batchId, filePath, customConfig, onProgress }) { const { rowsTotal, rowsSkipped } = await stageBatch({ batchId, filePath, customConfig, onProgress }); @@ -74,7 +110,10 @@ export async function prepareImport({ batchId, filePath, customConfig, onProgres } /** - * @param {{ batchId: number, onProgress?: Function }} args + * Commit a prepared (or reviewed) batch and settle its final status. + * + * @param {{ batchId: PortfolioImportBatchId, onProgress?: PortfolioImportProgressCallback }} args + * @returns {Promise<{ imported: number, duplicates: number, errors: number }>} */ export async function commitPortfolioImport({ batchId, onProgress }) { const { imported, duplicates, errors } = await commitBatch({ batchId, onProgress }); @@ -112,7 +151,12 @@ export async function commitPortfolioImport({ batchId, onProgress }) { } /** - * @param {{ filePath: string, adapterName: string, customConfig: object, defaultAssetClass?: string, defaultType?: string, filename?: string, sizeBytes?: number, isBrokerage?: boolean, accountId?: number, onProgress?: Function }} args + * Full one-shot portfolio import: create the batch, prepare it, and either + * auto-commit (every row matched by exact symbol, nothing errored) or leave it + * in 'awaiting_review'. + * + * @param {{ filePath: string, adapterName: string, customConfig: object, defaultAssetClass?: string, defaultType?: string, filename?: string, sizeBytes?: number, isBrokerage?: boolean, accountId?: number, onProgress?: PortfolioImportProgressCallback }} args + * @returns {Promise<{ batchId: PortfolioImportBatchId, total: number, skipped: number, requiresReview: boolean, matchSourceCounts?: Record, imported?: number, duplicates?: number, errors?: number }>} */ export async function runPortfolioImportPipeline({ filePath, adapterName, customConfig, defaultAssetClass, defaultType, filename, sizeBytes, isBrokerage, accountId, onProgress }) { const batchId = await createBatch({ adapterName, filename, sizeBytes, customConfig, defaultAssetClass, defaultType, isBrokerage, accountId }); diff --git a/apps/node-backend/src/services/portfolioImportPipeline/matchInvestments.js b/apps/node-backend/src/services/portfolioImportPipeline/matchInvestments.js index 312a7282..d3af2b6e 100644 --- a/apps/node-backend/src/services/portfolioImportPipeline/matchInvestments.js +++ b/apps/node-backend/src/services/portfolioImportPipeline/matchInvestments.js @@ -14,6 +14,26 @@ import { query } from '../../database/connection.js'; import { logger } from '../../config/logger.js'; +/** + * @typedef {import('../../types/rows.js').PortfolioImportStagingRow} PortfolioImportStagingRow + * @typedef {import('./index.js').PortfolioImportBatchId} PortfolioImportBatchId + * @typedef {import('./index.js').PortfolioImportProgressCallback} PortfolioImportProgressCallback + */ + +/** + * One resolution outcome for a (symbol, name) pair. + * + * @typedef {{ investmentId: number|null, matchSource: 'symbol'|'name_exact'|null }} InstrumentMatch + */ + +/** + * Run the match phase: resolve each validated row to an existing investment by + * unambiguous symbol, then unambiguous exact name. Cash rows (brokerage + * deposits/withdrawals) skip resolution entirely. + * + * @param {{ batchId: PortfolioImportBatchId, onProgress?: PortfolioImportProgressCallback }} args + * @returns {Promise<{ matchSourceCounts: Record, unresolved: number, total: number }>} + */ export async function matchBatch({ batchId, onProgress }) { await query(`UPDATE portfolio_import_batches SET status = 'matching' WHERE id = $1`, [batchId]); @@ -53,9 +73,16 @@ export async function matchBatch({ batchId, onProgress }) { // Resolve once per distinct (symbol, name) pair — brokerage exports repeat the // same instrument across many rows. + /** @type {Map} */ const cache = new Map(); - const resolveKey = (symbol, name) => `${symbol || ''}\x00${name || ''}`; - + const resolveKey = (/** @type {string|null} */ symbol, /** @type {string|null} */ name) => + `${symbol || ''}\x00${name || ''}`; + + /** + * @param {string|null} symbolRaw + * @param {string|null} nameRaw + * @returns {InstrumentMatch} + */ const classify = (symbolRaw, nameRaw) => { const symbol = String(symbolRaw || '').trim(); if (symbol) { @@ -77,13 +104,17 @@ export async function matchBatch({ batchId, onProgress }) { return { investmentId: null, matchSource: null }; }; + /** @type {string[]} */ const ids = []; + /** @type {(number|null)[]} */ const investmentIds = []; + /** @type {(string|null)[]} */ const matchSources = []; + /** @type {Record} */ const counts = { symbol: 0, name_exact: 0, unresolved: 0 }; let seen = 0; - for (const row of rows) { + for (const row of /** @type {Pick[]} */ (rows)) { const key = resolveKey(row.symbol_raw, row.name_raw); let resolved = cache.get(key); if (resolved === undefined) { @@ -128,15 +159,17 @@ export async function matchBatch({ batchId, onProgress }) { * match, >1 → ambiguous/unresolved" rule; MIN(id) is the resolved id when the * count is 1 (identical to the old ORDER BY id LIMIT 1 on a single match). * - * @param {{ symbol_raw: string }[]} rows + * @param {{ symbol_raw: string|null }[]} rows * @returns {Promise>} keyed by lowercased symbol */ async function resolveBySymbolBatch(rows) { + /** @type {Set} */ const symbols = new Set(); for (const row of rows) { const symbol = String(row.symbol_raw || '').trim(); if (symbol) symbols.add(symbol.toLowerCase()); } + /** @type {Map} */ const map = new Map(); if (symbols.size === 0) return map; const r = await query( @@ -158,11 +191,12 @@ async function resolveBySymbolBatch(rows) { * that carry no symbol — reach the name path; a row with an ambiguous symbol is * excluded, mirroring the old function's early return before name matching. * - * @param {{ symbol_raw: string, name_raw: string }[]} rows + * @param {{ symbol_raw: string|null, name_raw: string|null }[]} rows * @param {Map} symbolMatches * @returns {Promise>} keyed by lowercased trimmed name */ async function resolveByNameBatch(rows, symbolMatches) { + /** @type {Set} */ const names = new Set(); for (const row of rows) { const symbol = String(row.symbol_raw || '').trim(); @@ -171,6 +205,7 @@ async function resolveByNameBatch(rows, symbolMatches) { const name = String(row.name_raw || '').trim(); if (name) names.add(name.toLowerCase()); } + /** @type {Map} */ const map = new Map(); if (names.size === 0) return map; const r = await query( diff --git a/apps/node-backend/src/services/portfolioImportPipeline/portfolioGenericAdapter.js b/apps/node-backend/src/services/portfolioImportPipeline/portfolioGenericAdapter.js index 004c093d..486a9488 100644 --- a/apps/node-backend/src/services/portfolioImportPipeline/portfolioGenericAdapter.js +++ b/apps/node-backend/src/services/portfolioImportPipeline/portfolioGenericAdapter.js @@ -12,7 +12,55 @@ import { logger } from '../../config/logger.js'; import { parseCsvFile, buildRawRowString, parseAmountField, SUPPORTED_DATE_FORMATS, parseDateWithFormat } from '../importPipeline/adapters/_shared.js'; -// Absolute magnitude of a numeric cell, or null when blank/unparseable. +/** + * One raw row as this adapter extracts it — field names are the staging + * columns' camelCase equivalents, and every numeric is stored as an ABSOLUTE + * magnitude (direction is carried by the later-normalized type). + * + * @typedef {object} ParsedPortfolioRow + * @property {Date} date UTC-midnight (see parseDateWithFormat). + * @property {string} typeRaw the CSV's own type label; '' when unmapped. + * @property {string} symbolRaw + * @property {string} nameRaw + * @property {number|null} units + * @property {number|null} pricePerUnit + * @property {number|null} amount + * @property {number|null} fees + * @property {number|null} taxes + * @property {string|null} currency + * @property {number|null} fxRateToEur + * @property {string} note + * @property {string} rawData source record, kept for dedup + provenance. + */ + +/** + * A parsed row list carrying the adapter's count of rows it could not + * interpret (the counter rides on the array, matching the transaction + * adapters' contract). + * + * @typedef {ParsedPortfolioRow[] & { skipped?: number }} ParsedPortfolioRows + */ + +/** + * The custom-parser definition a portfolio import runs on. It comes from the + * upload route or a saved `custom_parser_configs.config_json` row and is not + * re-validated here, so everything beyond `column_mapping` is optional. + * + * @typedef {object} PortfolioParserConfig + * @property {string} [date_format] must be one of SUPPORTED_DATE_FORMATS + * @property {string} [separator] CSV delimiter; defaults to ',' + * @property {number} [skip_rows] + * @property {BufferEncoding} [encoding] defaults to 'utf-8' + * @property {Record} [type_mapping] raw type label → canonical portfolio_txn_type (read by validate.js) + * @property {{ date?: string, type?: string, symbol?: string, name?: string, units?: string, price?: string, amount?: string, fees?: string, taxes?: string, currency?: string, fx_rate?: string, note?: string }} [column_mapping] source column NAMES, not indices + */ + +/** + * Absolute magnitude of a numeric cell, or null when blank/unparseable. + * + * @param {unknown} raw + * @returns {number|null} + */ function parseMagnitude(raw) { if (raw === undefined || raw === null || String(raw).trim() === '') return null; const n = parseAmountField(raw); @@ -20,11 +68,21 @@ function parseMagnitude(raw) { return Math.abs(n); } +/** + * @param {Record} row a `columns: true` csv-parse record + * @param {string|undefined} key the mapped source column name; '' when unmapped + * @returns {string} trimmed cell value, '' when the column is unmapped or absent + */ function cell(row, key) { if (!key) return ''; return String(row[key] ?? '').trim(); } +/** + * @param {Record} row a `columns: true` csv-parse record + * @param {PortfolioParserConfig} config + * @returns {ParsedPortfolioRow|null} null when the mapped date cell is missing or unparseable + */ function rowToParsed(row, config) { const colMap = config.column_mapping || {}; const dateStr = cell(row, colMap.date); @@ -52,6 +110,12 @@ function rowToParsed(row, config) { }; } +/** + * @param {string} filePath + * @param {PortfolioParserConfig} config + * @returns {Promise} + * @throws {Error} when `date_format` is not one of SUPPORTED_DATE_FORMATS + */ export async function parseWithConfig(filePath, config) { const dateFormat = config.date_format || ''; if (!SUPPORTED_DATE_FORMATS.includes(dateFormat)) { @@ -72,7 +136,7 @@ export async function parseWithConfig(filePath, config) { config.encoding || 'utf-8', ); - const rows = /** @type {any[] & { skipped?: number }} */ ([]); + const rows = /** @type {ParsedPortfolioRows} */ ([]); let skipped = 0; for (const record of records) { try { diff --git a/apps/node-backend/src/services/portfolioImportPipeline/portfolioTypeNormalizer.js b/apps/node-backend/src/services/portfolioImportPipeline/portfolioTypeNormalizer.js index 81ca0214..31580d32 100644 --- a/apps/node-backend/src/services/portfolioImportPipeline/portfolioTypeNormalizer.js +++ b/apps/node-backend/src/services/portfolioImportPipeline/portfolioTypeNormalizer.js @@ -12,6 +12,7 @@ import { VALID_PORTFOLIO_TXN_TYPES } from '../../lib/portfolioTxnTypes.js'; // Lowercased raw → canonical. Covers common English/Dutch/German brokerage labels. +/** @type {Record} */ export const BUILTIN_TYPE_ALIASES = { buy: 'buy', purchase: 'buy', bought: 'buy', koop: 'buy', aankoop: 'buy', kauf: 'buy', sell: 'sell', sale: 'sell', sold: 'sell', verkoop: 'sell', verkauf: 'sell', diff --git a/apps/node-backend/src/services/portfolioImportPipeline/stage.js b/apps/node-backend/src/services/portfolioImportPipeline/stage.js index 6ba5af26..16d64948 100644 --- a/apps/node-backend/src/services/portfolioImportPipeline/stage.js +++ b/apps/node-backend/src/services/portfolioImportPipeline/stage.js @@ -12,8 +12,29 @@ import { logger } from '../../config/logger.js'; import { parsedDateToYmd } from '../../lib/importDates.js'; import { parseWithConfig } from './portfolioGenericAdapter.js'; +/** + * @typedef {import('../../types/rows.js').PortfolioImportStagingRow} PortfolioImportStagingRow + * @typedef {import('./index.js').PortfolioImportBatchId} PortfolioImportBatchId + * @typedef {import('./index.js').PortfolioImportProgressCallback} PortfolioImportProgressCallback + */ + const STAGE_INSERT_CHUNK = 500; +/** + * Create a new portfolio import batch row. + * + * @param {{ adapterName: string, filename?: string|null, sizeBytes?: number|null, customConfig?: object|null, defaultAssetClass?: string|null, defaultType?: string|null, isBrokerage?: boolean, accountId?: number|string|null }} args + * @returns {Promise} the new batch id, as a NUMBER. + * + * `portfolio_import_batches.id` is BIGSERIAL and node-postgres hands BIGINT + * back as a STRING. Normalized here for the same reason as the transaction + * pipeline (see importPipeline/stage.js `createBatch`): the streaming/immediate + * import responses would otherwise emit `batch_id: "12"` while the review-commit + * route (routes/portfolioImportRoutes.js:491) emits `batch_id: 12`. NUMBER is + * the single wire type — it matches `coercedIdSchema` (lib/importBatchIds.js:17) + * and the frontend guards (`batch_id: z.number()` in + * apps/frontend/src/lib/api/portfolioImports.ts). + */ export async function createBatch({ adapterName, filename, sizeBytes, customConfig, defaultAssetClass, defaultType, isBrokerage = false, accountId }) { const result = await query( `INSERT INTO portfolio_import_batches @@ -31,9 +52,16 @@ export async function createBatch({ adapterName, filename, sizeBytes, customConf accountId != null ? Number(accountId) : null, ], ); - return result.rows[0].id; + return Number(result.rows[0].id); } +/** + * Run the stage phase: parse the file, bulk-insert staging rows. + * + * @param {{ batchId: PortfolioImportBatchId, filePath: string, customConfig: import('./portfolioGenericAdapter.js').PortfolioParserConfig, onProgress?: PortfolioImportProgressCallback }} args + * @returns {Promise<{ rowsTotal: number, rowsSkipped: number }>} `rowsSkipped` is + * the adapter's own count of data rows it could not interpret. + */ export async function stageBatch({ batchId, filePath, customConfig, onProgress }) { await query(`UPDATE portfolio_import_batches SET status = 'staging' WHERE id = $1`, [batchId]); @@ -54,10 +82,21 @@ export async function stageBatch({ batchId, filePath, customConfig, onProgress } return { rowsTotal: total, rowsSkipped: skipped }; } +/** + * Bulk-insert one chunk of parsed rows as `portfolio_import_staging_rows` + * (status 'pending' via the column default) in one multi-VALUES statement. + * + * @param {PortfolioImportBatchId} batchId + * @param {import('./portfolioGenericAdapter.js').ParsedPortfolioRow[]} rows + * @param {number} startIndex the chunk's offset, written to `row_index` + * @returns {Promise} + */ async function insertStagingChunk(batchId, rows, startIndex) { if (!rows.length) return; await withTransaction(async (client) => { + /** @type {any[]} */ const values = []; + /** @type {string[]} */ const placeholders = []; rows.forEach((r, i) => { const idx = startIndex + i; diff --git a/apps/node-backend/src/services/portfolioImportPipeline/validate.js b/apps/node-backend/src/services/portfolioImportPipeline/validate.js index 045e5362..bd09412e 100644 --- a/apps/node-backend/src/services/portfolioImportPipeline/validate.js +++ b/apps/node-backend/src/services/portfolioImportPipeline/validate.js @@ -18,8 +18,30 @@ import { UNIT_BASED_ASSET_CLASSES } from '../../repositories/portfolioTxRepo.com import { normalizeType } from './portfolioTypeNormalizer.js'; import { classifyBrokerageRow } from '../importPipeline/brokerageRouting.js'; +/** + * @typedef {import('../../types/rows.js').PortfolioImportStagingRow} PortfolioImportStagingRow + * @typedef {import('./index.js').PortfolioImportBatchId} PortfolioImportBatchId + * @typedef {import('./index.js').PortfolioImportProgressCallback} PortfolioImportProgressCallback + */ + +/** + * The projection validate.js reads. `tx_date` is selected RAW here (unlike the + * transaction pipeline), so it really is a pg local-midnight `Date` — + * `resolveAndCheck` formats it with LOCAL getters (`toYmd`) on purpose. + * + * @typedef {Pick} PendingPortfolioStagingRow + */ + const VALIDATE_CHUNK = 500; +/** + * Run the validate phase: normalize each pending row's type, pre-check its + * numeric fields, hash it, and mark it validated / duplicate / error. + * + * @param {{ batchId: PortfolioImportBatchId, onProgress?: PortfolioImportProgressCallback }} args + * @returns {Promise<{ validated: number, duplicates: number, errors: number }>} + */ export async function validateBatch({ batchId, onProgress }) { await query(`UPDATE portfolio_import_batches SET status = 'validating' WHERE id = $1`, [batchId]); @@ -53,20 +75,27 @@ export async function validateBatch({ batchId, onProgress }) { let seen = 0; let errors = 0; let duplicates = 0; + /** @type {Set} */ const seenHashes = new Set(); if (onProgress) onProgress({ phase: 'validating', current: 0, total }); for (let start = 0; start < total; start += VALIDATE_CHUNK) { const chunk = pending.slice(start, start + VALIDATE_CHUNK); + /** @type {string[]} */ const ids = []; + /** @type {string[]} */ const statuses = []; + /** @type {(string|null|undefined)[]} */ const types = []; + /** @type {(string|null|undefined)[]} */ const routes = []; + /** @type {(string|null)[]} */ const txHashes = []; + /** @type {(string|null)[]} */ const errorMessages = []; - for (const row of chunk) { + for (const row of /** @type {PendingPortfolioStagingRow[]} */ (chunk)) { ids.push(row.id); const { type, route, error } = resolveAndCheck(row, { typeMapping, defaultType, unitBased, isBrokerage, today }); if (error) { @@ -131,6 +160,14 @@ export async function validateBatch({ batchId, onProgress }) { return { validated, duplicates, errors }; } +/** + * Resolve one staging row's canonical type and brokerage route, or report why + * it cannot be committed. + * + * @param {PendingPortfolioStagingRow} row + * @param {{ typeMapping: Record, defaultType: string|undefined, unitBased: boolean, isBrokerage: boolean, today: string }} options + * @returns {{ type?: string, route?: string, error?: string }} exactly one of `type`/`error` is meaningful + */ function resolveAndCheck(row, { typeMapping, defaultType, unitBased, isBrokerage, today }) { if (!row.tx_date) return { error: 'missing date' }; @@ -176,6 +213,15 @@ function resolveAndCheck(row, { typeMapping, defaultType, unitBased, isBrokerage return { type, route: isBrokerage ? 'portfolio' : undefined }; } +/** + * sha256 of route|type|raw record, falling back to the parsed fields when the + * adapter kept no raw record. + * + * @param {PendingPortfolioStagingRow} row + * @param {string|undefined} type + * @param {string|undefined} route + * @returns {string} lowercase hex digest + */ function computeRowHash(row, type, route) { let raw; if (row.raw_data) { diff --git a/apps/node-backend/src/services/portfolioPerformanceSnapshotService.js b/apps/node-backend/src/services/portfolioPerformanceSnapshotService.js index 6b2d2832..892b08a0 100644 --- a/apps/node-backend/src/services/portfolioPerformanceSnapshotService.js +++ b/apps/node-backend/src/services/portfolioPerformanceSnapshotService.js @@ -10,12 +10,36 @@ import { computeMetrics, computeHeatmap } from '../utils/portfolioMath.js'; import { computeAndStoreSnapshots } from './portfolio/snapshotBuilder.js'; import { getPortfolioSummary, getBreakdownSummary } from './portfolio/portfolioSummaryService.js'; +/** @typedef {import('../types/rows.js').PortfolioPerformanceSnapshotRow} PortfolioPerformanceSnapshotRow */ + // Re-export the imported bindings so both `import { x } from` consumers and the // default-object consumers below share a single declaration each (SIMP-51). export { computeMetrics, computeHeatmap }; export { computeAndStoreSnapshots }; export { getPortfolioSummary, getBreakdownSummary }; +/** + * @param {string} startDate 'YYYY-MM-DD' + * @param {string} endDate 'YYYY-MM-DD' + * @param {string} [currency] + * @returns {Promise>} + */ export async function getSnapshots(startDate, endDate, currency = 'EUR') { // SELECT * + shape in JS: value_fx_neutral only exists once migration 0039 // is applied, and enumerating it in SQL would break un-migrated databases. @@ -27,7 +51,7 @@ export async function getSnapshots(startDate, endDate, currency = 'EUR') { ORDER BY snapshot_date ASC `, [currency, startDate, endDate]); - return result.rows.map((row) => ({ + return result.rows.map((/** @type {PortfolioPerformanceSnapshotRow} */ row) => ({ snapshot_date: row.snapshot_date, invested: row.invested, value: row.value, diff --git a/apps/node-backend/src/services/priceProviderService.js b/apps/node-backend/src/services/priceProviderService.js index 1ba2d2d2..5b1a2a36 100644 --- a/apps/node-backend/src/services/priceProviderService.js +++ b/apps/node-backend/src/services/priceProviderService.js @@ -33,6 +33,22 @@ import { } from './prices/priceProviderRegistry.js'; import { getYahooClient } from './prices/yahooClient.js'; +/** @typedef {import('../types/rows.js').InvestmentRow} InvestmentRow */ +/** @typedef {import('../types/rows.js').PricePoint} PricePoint */ +/** @typedef {import('./prices/priceProviderRegistry.js').LivePriceQuote} LivePriceQuote */ + +/** + * A resolved live-price result for one investment. `source` reflects which + * tier of the fetch → cache → cached-price → historical-fallback chain + * produced it: a `LivePriceQuote.source` value ('live'|'close') when it came + * straight off a provider/cache hit, or 'cached'/'historical_fallback' for + * the two degraded tiers below. + * @typedef {object} ResolvedPrice + * @property {number} price + * @property {string} source + * @property {number|null} [stale_as_of_ms] only set for 'historical_fallback'. + */ + export { saveHistoricalPointsToDatabase, resetPriceCache as __resetPriceCache, @@ -50,6 +66,10 @@ export const SUPPORTED_PROVIDERS = [ // ─── Live price fetching ────────────────────────────────────────────────────── +/** + * @param {InvestmentRow[]} investments + * @returns {Promise>} + */ export async function fetchLivePrices(investments) { const detailed = await fetchLivePricesDetailed(investments); return Object.fromEntries( @@ -57,8 +77,15 @@ export async function fetchLivePrices(investments) { ); } +/** + * @param {InvestmentRow[]} investments + * @param {{ cachedPricesByInvestmentId?: Record }} [opts] + * @returns {Promise>} + */ export async function fetchLivePricesDetailed(investments, { cachedPricesByInvestmentId = {} } = {}) { + /** @type {Record} */ const results = {}; + /** @type {Record} */ const stale = { binance: [], yahoo: [], custom: [], kinesis: [] }; for (const inv of investments) { @@ -86,6 +113,12 @@ export async function fetchLivePricesDetailed(investments, { cachedPricesByInves // Provider fetch tasks. Two shapes (SIMP-33): id-based providers batch by a // resolved symbol and key results/cache by that symbol; investment-based // providers pass the investments through and key by inv.id. + /** + * @param {string} key + * @param {string} label + * @param {() => Promise} task + * @returns {Promise} + */ const runProviderTask = async (key, label, task) => { try { await task(); @@ -97,11 +130,13 @@ export async function fetchLivePricesDetailed(investments, { cachedPricesByInves }; // { key, resolveId, batchFn, label } + /** @type {Array<{ key: string, resolveId: (inv: InvestmentRow) => string, batchFn: (ids: string[]) => Promise>, label: string }>} */ const idBasedProviders = [ { key: 'binance', resolveId: (inv) => (inv.price_provider_id || '').toUpperCase(), batchFn: PROVIDERS.binance, label: 'Binance batch fetch' }, { key: 'yahoo', resolveId: resolveYahooSymbol, batchFn: PROVIDERS.yahoo, label: 'Yahoo Finance batch fetch' }, ]; // { key, batchFn, label } — kinesis already converts EUR-symbol prices out of USD. + /** @type {Array<{ key: string, batchFn: (investments: InvestmentRow[]) => Promise>, label: string }>} */ const investmentBasedProviders = [ { key: 'custom', batchFn: PROVIDERS.custom, label: 'Custom price fetch' }, { key: 'kinesis', batchFn: PROVIDERS.kinesis, label: 'Kinesis price fetch' }, @@ -187,6 +222,10 @@ const BINANCE_DAY_MS = 24 * 60 * 60 * 1000; const BINANCE_PAGE_LIMIT = 1000; const BINANCE_MAX_PAGES = 30; // 30 × 1000 daily candles ≈ 82 years — a runaway guard, not a real bound +/** + * @param {number} ms + * @returns {number} + */ function _dayKey(ms) { return Math.floor(Number(ms) / BINANCE_DAY_MS); } @@ -228,6 +267,12 @@ async function _fetchBinanceKlines(binanceSymbol, startMs, endMs) { return collected; } +/** + * @param {PricePoint[]} points + * @param {number} [fromMs] + * @param {number} [toMs] + * @returns {PricePoint[]} + */ function _filterHistoricalPoints(points, fromMs, toMs) { return filterPointsByRange(points, { fromMs, toMs }); } @@ -307,8 +352,8 @@ export async function fetchHistoricalPrices(investment, { fromMs, toMs, dbOnly = includePrePost: false, }); - points = normalizeHistoryPoints((chart?.quotes || []) - .map((q) => ({ + points = normalizeHistoryPoints((/** @type {any[]} */ (chart?.quotes) || []) + .map((/** @type {any} */ q) => ({ timestampMs: q?.date ? new Date(q.date).getTime() : Number.NaN, price: toNumber(q?.close), })) diff --git a/apps/node-backend/src/services/prices/priceCache.js b/apps/node-backend/src/services/prices/priceCache.js index f350975a..e9057cb3 100644 --- a/apps/node-backend/src/services/prices/priceCache.js +++ b/apps/node-backend/src/services/prices/priceCache.js @@ -9,19 +9,46 @@ import { query } from '../../database/connection.js'; import { logger } from '../../config/logger.js'; import { epochMsToUtcYmd } from '../../lib/dateFormat.js'; +/** + * @typedef {import('../../types/rows.js').AssetPriceHistoryRow} AssetPriceHistoryRow + * @typedef {import('../../types/rows.js').PricePoint} PricePoint + */ + +/** + * A raw, unvalidated price point as it arrives from a provider adapter or a DB + * projection — both fields may be missing, non-finite, or the wrong type; + * `normalizeHistoryPoints` is the gate that turns these into {@link PricePoint}s. + * + * @typedef {{ timestampMs?: any, price?: any }} RawPricePoint + */ + export const PRICE_CACHE_TTL_MS = 5 * 60_000; const HISTORY_DAY_MS = 24 * 60 * 60 * 1000; // Key: `${provider}:${providerId}` — Value: { data, expiresAt } +// The payload differs per call site (a live quote, a point array, a provider +// response) and this module never inspects it, so it stays `any`. +/** @type {Map} */ const _cache = new Map(); // ─── Shared numeric helpers ─────────────────────────────────────────────────── +/** + * Coerce to a finite number, or undefined. Deliberately accepts anything — + * it is applied to raw provider JSON and to NUMERIC columns (pg strings). + * + * @param {any} value + * @returns {number|undefined} + */ export function toNumber(value) { const num = Number(value); return Number.isFinite(num) ? num : undefined; } +/** + * @param {any} value + * @returns {boolean} true only for a finite, strictly-positive number + */ export function isValidPrice(value) { const num = toNumber(value); return num !== undefined && num > 0; @@ -29,11 +56,19 @@ export function isValidPrice(value) { // ─── Date / timestamp helpers ───────────────────────────────────────────────── +/** + * @param {number} timestampMs + * @returns {string|undefined} the UTC calendar day as 'YYYY-MM-DD' + */ export function toDateOnly(timestampMs) { if (!Number.isFinite(timestampMs)) return undefined; return epochMsToUtcYmd(timestampMs); } +/** + * @param {string|Date|null|undefined} dateOnly a 'YYYY-MM-DD' string or a pg DATE (local-midnight `Date`) + * @returns {number} epoch ms at UTC noon of that day, or NaN when unparseable + */ export function dateOnlyToTimestampMs(dateOnly) { if (!dateOnly) return Number.NaN; // pg returns DATE columns as local-midnight Date objects. String() on one is @@ -53,8 +88,16 @@ export function dateOnlyToTimestampMs(dateOnly) { // ─── History point helpers ──────────────────────────────────────────────────── +/** + * Validate, de-duplicate by calendar day (last one wins) and date-sort a raw + * point series. Non-array input and malformed points are dropped, not thrown. + * + * @param {RawPricePoint[]|null|undefined} points + * @returns {PricePoint[]} date-ascending, one point per day + */ export function normalizeHistoryPoints(points) { if (!Array.isArray(points) || points.length === 0) return []; + /** @type {Map} */ const byDate = new Map(); for (const point of points) { @@ -106,6 +149,14 @@ export function needsHistoryRefresh(points, { fromMs, toMs } = {}) { return false; } +/** + * Count positionally-aligned points whose price actually moved. Used to report + * how much a refetch changed; non-array or malformed input counts as 0. + * + * @param {RawPricePoint[]|null|undefined} beforePoints + * @param {RawPricePoint[]|null|undefined} afterPoints + * @returns {number} + */ export function countChangedPointPrices(beforePoints, afterPoints) { if (!Array.isArray(beforePoints) || !Array.isArray(afterPoints)) return 0; const len = Math.min(beforePoints.length, afterPoints.length); @@ -121,6 +172,10 @@ export function countChangedPointPrices(beforePoints, afterPoints) { // ─── In-memory cache ────────────────────────────────────────────────────────── +/** + * @param {string} key `${provider}:${providerId}` + * @returns {any} the cached payload, or undefined when absent/expired + */ export function cacheGet(key) { const entry = _cache.get(key); if (!entry) return undefined; @@ -128,6 +183,11 @@ export function cacheGet(key) { return entry.data; } +/** + * @param {string} key `${provider}:${providerId}` + * @param {any} data payload shape varies per call site + * @returns {void} + */ export function cacheSet(key, data) { _cache.set(key, { data, expiresAt: Date.now() + PRICE_CACHE_TTL_MS }); } @@ -175,7 +235,8 @@ export async function loadHistoricalPointsFromDatabase(investmentId, { fromMs, t ); return normalizeHistoryPoints( - result.rows.map((row) => ({ + /** @type {Pick[]} */ + (result.rows).map((row) => ({ timestampMs: dateOnlyToTimestampMs(row.price_date), price: toNumber(row.close_price), })) @@ -207,8 +268,9 @@ export async function loadLatestHistoricalPointByInvestmentIds(investmentIds) { [ids] ); + /** @type {Map} */ const byId = new Map(); - for (const row of result.rows) { + for (const row of /** @type {Pick[]} */ (result.rows)) { byId.set(row.investment_id, { timestampMs: dateOnlyToTimestampMs(row.price_date), price: toNumber(row.close_price), @@ -242,6 +304,15 @@ async function _dropForeignKey() { } } +/** + * Upsert a normalized point series for one investment. Silently no-ops when the + * table is absent (42P01) or nothing survives normalization. + * + * @param {number} investmentId + * @param {RawPricePoint[]|null|undefined} points + * @param {string|null|undefined} source provider id; defaults to 'provider' + * @returns {Promise} + */ export async function saveHistoricalPointsToDatabase(investmentId, points, source) { const normalized = normalizeHistoryPoints(points); if (!Number.isFinite(Number(investmentId)) || normalized.length === 0) return; diff --git a/apps/node-backend/src/services/prices/priceProviderRegistry.js b/apps/node-backend/src/services/prices/priceProviderRegistry.js index f8458266..90e57ec4 100644 --- a/apps/node-backend/src/services/prices/priceProviderRegistry.js +++ b/apps/node-backend/src/services/prices/priceProviderRegistry.js @@ -18,13 +18,52 @@ import { convertToCurrency } from '../currency/currencyConversionService.js'; import { assertPublicHttpUrl } from '../../lib/urlSafety.js'; import { getYahooClient } from './yahooClient.js'; +/** + * @typedef {import('../../types/rows.js').InvestmentRow} InvestmentRow + * @typedef {import('../../types/rows.js').PricePoint} PricePoint + */ + +/** + * The custom-provider history extraction config, as resolved from an + * investment's `price_provider_history_*` columns. Every field is a + * dot-separated path into the provider's JSON except `historyUrl`. + * + * @typedef {object} CustomHistoryConfig + * @property {string} historyUrl + * @property {string} historyPath path to the point array + * @property {string} timestampPath path to a point's epoch-millis timestamp + * @property {string} pricePath path to a point's price + */ + +/** + * One provider strategy's answer for a single key. `currency` and `source` are + * optional because the `custom` strategy has no way to know either — it only + * ever returns `{ price }`. + * + * @typedef {object} LivePriceQuote + * @property {number} price + * @property {string} [currency] + * @property {'live'|'close'} [source] + */ + // ─── Path helpers ───────────────────────────────────────────────────────────── +/** + * @param {unknown} path dot-separated path, e.g. `data.points` + * @returns {string[]} trimmed non-empty segments; [] for non-string input + */ function _splitPath(path) { if (typeof path !== 'string') return []; return path.trim().split('.').map(seg => seg.trim()).filter(Boolean); } +/** + * Walk a dot-separated path into arbitrary provider JSON. + * + * @param {any} input parsed provider JSON — shape is user-configured, so genuinely unknown + * @param {unknown} path an empty path returns `input` itself + * @returns {any} undefined as soon as any hop is null/undefined + */ function _readPathValue(input, path) { const segments = _splitPath(path); if (segments.length === 0) return input; @@ -38,6 +77,13 @@ function _readPathValue(input, path) { // ─── Config resolvers ───────────────────────────────────────────────────────── +/** + * The Yahoo ticker for an investment: the explicit `price_provider_id`, else + * the instrument symbol, uppercased. Empty string when neither is set. + * + * @param {Partial|null|undefined} inv + * @returns {string} + */ export function resolveYahooSymbol(inv) { const providerId = (inv?.price_provider_id || '').trim(); if (providerId) return providerId.toUpperCase(); @@ -45,12 +91,20 @@ export function resolveYahooSymbol(inv) { return symbol ? symbol.toUpperCase() : ''; } +/** + * @param {Partial|null|undefined} inv + * @returns {{ latestUrl: string, latestPath: string }} empty `latestUrl` when no URL column is set + */ function _resolveCustomLatestConfig(inv) { const latestUrl = (inv?.price_provider_latest_url || inv?.price_provider_url || inv?.price_provider_history_url || '').trim(); const latestPath = (inv?.price_provider_latest_path || inv?.price_provider_id || 'price').trim(); return { latestUrl, latestPath }; } +/** + * @param {Partial|null|undefined} inv + * @returns {CustomHistoryConfig} empty `historyUrl` when no URL column is set + */ export function resolveCustomHistoryConfig(inv) { const historyUrl = (inv?.price_provider_history_url || inv?.price_provider_latest_url || inv?.price_provider_url || '').trim(); const historyPath = (inv?.price_provider_history_path || 'points').trim(); @@ -60,6 +114,7 @@ export function resolveCustomHistoryConfig(inv) { } // Kinesis API only provides USD-denominated symbols. Map EUR variants to USD. +/** @type {Record} */ const KINESIS_EUR_TO_USD = { 'KAU_EUR': 'KAU_USD', 'KAG_EUR': 'KAG_USD', @@ -69,6 +124,16 @@ const KINESIS_EUR_TO_USD = { 'XPD_EUR': 'XPD_USD', }; +/** + * Resolve the Kinesis symbol + trendline window for an investment, remapping a + * EUR symbol to its USD variant (Kinesis only quotes USD) and falling back to + * the name-keyed asset catalogue. + * + * @param {Partial|null|undefined} inv + * @returns {{ symbol: string, timeframe: number, fromDate: string, needsUsdToEur: boolean }} + * `symbol` is '' when nothing could be resolved; `needsUsdToEur` flags that + * the fetched prices are USD and must be converted before use. + */ export function resolveKinesisConfig(inv) { const providerId = (inv?.price_provider_id || '').trim(); const assetName = (inv?.name || inv?.symbol || '').toLowerCase().trim(); @@ -124,6 +189,9 @@ function _assertResponseWithinCap(res, provider) { * (scheme + private/loopback/link-local block, DNS-resolved). Redirects are * followed manually so a public host cannot 302 the request to an internal * address, and the response body is size-capped. + * + * @param {string} url + * @returns {Promise} parsed provider JSON — shape is user-configured */ async function _fetchJson(url) { let current = String(url); @@ -152,10 +220,19 @@ async function _fetchJson(url) { // ─── Custom endpoint parsing ────────────────────────────────────────────────── +/** + * Extract a date-ascending point series from a custom provider's JSON using the + * configured paths. Malformed rows and non-positive prices are dropped. + * + * @param {any} data parsed provider JSON + * @param {CustomHistoryConfig} config + * @returns {PricePoint[]} + */ function _parseCustomHistoryPoints(data, config) { const listValue = _readPathValue(data, config.historyPath); if (!Array.isArray(listValue)) return []; + /** @type {PricePoint[]} */ const points = []; for (const row of listValue) { const timestampMs = Number(_readPathValue(row, config.timestampPath)); @@ -168,12 +245,27 @@ function _parseCustomHistoryPoints(data, config) { return points; } +/** + * Last price of a custom provider's history payload — the fallback when the + * latest-price endpoint yields nothing usable. + * + * @param {any} data parsed provider JSON + * @param {CustomHistoryConfig} historyConfig + * @returns {number|undefined} + */ function _deriveLatestPriceFromHistoryPayload(data, historyConfig) { const points = _parseCustomHistoryPoints(data, historyConfig); if (!points.length) return undefined; return points[points.length - 1]?.price; } +/** + * Public wrapper over the custom-provider history parser. + * + * @param {any} data parsed provider JSON + * @param {CustomHistoryConfig} config + * @returns {PricePoint[]} + */ export function parseCustomHistoryPoints(data, config) { return _parseCustomHistoryPoints(data, config); } @@ -183,8 +275,14 @@ export function parseCustomHistoryPoints(data, config) { // Removes runs of ≥ minRunLength consecutive identical prices (stale API data). // Keeps only the first point of each such run so the series resumes at the // correct price when real updates arrive. +/** + * @param {PricePoint[]} points + * @param {number} [minRunLength] run length at/above which the repeats are dropped + * @returns {PricePoint[]} a new array (the input is not mutated) + */ function _removeStaleRuns(points, minRunLength = 8) { if (points.length < 2) return [...points]; + /** @type {PricePoint[]} */ const result = []; let i = 0; while (i < points.length) { @@ -200,6 +298,14 @@ function _removeStaleRuns(points, minRunLength = 8) { return result; } +/** + * Kinesis-specific data cleanup: drop stale repeated-price runs, then flatten + * isolated one-point needles (edges against their single neighbour, interior + * against a MAD-based robust threshold). + * + * @param {PricePoint[]|null|undefined} points + * @returns {PricePoint[]} a shallow-copied, repaired series + */ export function sanitizeKinesisIsolatedSpikes(points) { if (!Array.isArray(points) || points.length < 2) return points || []; @@ -230,6 +336,7 @@ export function sanitizeKinesisIsolatedSpikes(points) { if (sanitized.length < 5) return sanitized; + /** @type {number[]} */ const logReturns = []; for (let i = 1; i < sanitized.length; i += 1) { const prev = toNumber(sanitized[i - 1]?.price); @@ -267,9 +374,17 @@ export function sanitizeKinesisIsolatedSpikes(points) { return sanitized; } +/** + * Kinesis trendline rows → sanitized point series. Kinesis dates its points + * with an ISO `createdAt` rather than an epoch, so it needs its own parser. + * + * @param {any} rawPoints the `data[symbol]` array of the Kinesis response + * @returns {PricePoint[]} + */ function _parseKinesisTrendlinePoints(rawPoints) { if (!Array.isArray(rawPoints)) return []; + /** @type {PricePoint[]} */ const points = []; for (const point of rawPoints) { const createdAt = point?.createdAt; @@ -286,6 +401,13 @@ function _parseKinesisTrendlinePoints(rawPoints) { // ─── Yahoo helper ───────────────────────────────────────────────────────────── +/** + * Most recent valid daily close from Yahoo's chart endpoint (5-day window) — + * the per-symbol fallback when the batched quote call yields nothing. + * + * @param {string} symbol + * @returns {Promise} + */ async function _fetchYahooLatestClose(symbol) { const yahooFinance = await getYahooClient(); const chart = await yahooFinance.chart(symbol, { @@ -304,6 +426,14 @@ async function _fetchYahooLatestClose(symbol) { // ─── Historical price point lookup ─────────────────────────────────────────── +/** + * Binary-search a date-ascending series for the last price at or before a + * timestamp. + * + * @param {PricePoint[]|null|undefined} points must be date-ascending + * @param {number} timestampMs + * @returns {number|undefined} undefined when the series is empty or starts later + */ export function getHistoricalPriceAt(points, timestampMs) { if (!Array.isArray(points) || points.length === 0) return undefined; @@ -328,11 +458,21 @@ export function getHistoricalPriceAt(points, timestampMs) { // ─── Provider strategies ────────────────────────────────────────────────────── +/** + * Provider strategies. The two shapes are deliberate (SIMP-33): `binance` and + * `yahoo` batch by a resolved provider id and key their result by that id; + * `custom` and `kinesis` take investment rows and key by `inv.id`. + */ export const PROVIDERS = { + /** + * @param {string[]} providerIds Binance ticker symbols + * @returns {Promise>} keyed by symbol; missing symbols are simply absent + */ async binance(providerIds) { const uniqueSymbols = [...new Set(providerIds.map(id => (id || '').toUpperCase()))].filter(Boolean); if (uniqueSymbols.length === 0) return {}; + /** @type {Record} */ const prices = {}; try { const res = await fetch('https://data-api.binance.vision/api/v3/ticker/price', { @@ -343,6 +483,7 @@ export const PROVIDERS = { _assertResponseWithinCap(res, 'Binance'); const data = await res.json(); + /** @type {Record} */ const priceMap = {}; for (const item of data) { if (!item.symbol || !item.price) continue; @@ -365,8 +506,14 @@ export const PROVIDERS = { return prices; }, + /** + * @param {string[]} providerIds Yahoo ticker symbols + * @returns {Promise>} keyed by uppercased symbol + */ async yahoo(providerIds) { + /** @type {Record} */ const prices = {}; + /** @type {Set} */ const resolved = new Set(); const symbols = [...new Set(providerIds.map((s) => (s || '').toUpperCase()).filter(Boolean))]; @@ -421,10 +568,15 @@ export const PROVIDERS = { return prices; }, + /** + * @param {InvestmentRow[]} investments holdings whose price_provider is 'custom' + * @returns {Promise>} keyed by investment id; `{ price }` only + */ async custom(investments) { // Per-holding fetches run concurrently — each iteration self-catches, and // serially one hung endpoint (10s timeout, up to 2 fetches per holding on // the fallback path) stalled every holding behind it. + /** @type {Record} */ const prices = {}; await Promise.all(investments.map(async (inv) => { const { latestUrl, latestPath } = _resolveCustomLatestConfig(inv); @@ -466,9 +618,14 @@ export const PROVIDERS = { return prices; }, + /** + * @param {InvestmentRow[]} investments holdings whose price_provider is 'kinesis' + * @returns {Promise>} keyed by investment id + */ async kinesis(investments) { // Same concurrency rationale as custom(): these ran one sequential fetch // per holding with a 15s timeout — worst case ~75s for 5 holdings. + /** @type {Record} */ const prices = {}; await Promise.all(investments.map(async (inv) => { const { symbol, timeframe, fromDate, needsUsdToEur } = resolveKinesisConfig(inv); diff --git a/apps/node-backend/src/services/providerHealthService.js b/apps/node-backend/src/services/providerHealthService.js index f2ffd331..0e24fc2c 100644 --- a/apps/node-backend/src/services/providerHealthService.js +++ b/apps/node-backend/src/services/providerHealthService.js @@ -13,6 +13,8 @@ import finnhubAdapter from './research/adapters/finnhubAdapter.js'; import fmpAdapter from './research/adapters/fmpAdapter.js'; import alphaVantageAdapter from './research/adapters/alphaVantageAdapter.js'; +/** @typedef {import('../repositories/providerHealthRepository.js').ProviderHealth} ProviderHealth */ + // ─── Constants ──────────────────────────────────────────────────────────────── const PROBE_TIMEOUT_MS = 10_000; @@ -89,6 +91,7 @@ const PROVIDER_DEFINITIONS = { // ─── Probe helpers ──────────────────────────────────────────────────────────── +/** @param {string} url */ async function probeUrl(url) { const res = await fetch(url, { signal: AbortSignal.timeout(PROBE_TIMEOUT_MS) }); if (!res.ok) throw new Error(`HTTP ${res.status}`); @@ -126,7 +129,7 @@ async function probeKinesis() { * @param {string} provider Key from PROVIDER_DEFINITIONS */ export async function recordSuccess(provider) { - const def = PROVIDER_DEFINITIONS[provider]; + const def = PROVIDER_DEFINITIONS[/** @type {keyof typeof PROVIDER_DEFINITIONS} */ (provider)]; if (!def) return; try { await providerHealthRepository.recordSuccess(provider, def.kind); @@ -142,7 +145,7 @@ export async function recordSuccess(provider) { * @param {Error|string} error */ export async function recordError(provider, error) { - const def = PROVIDER_DEFINITIONS[provider]; + const def = PROVIDER_DEFINITIONS[/** @type {keyof typeof PROVIDER_DEFINITIONS} */ (provider)]; if (!def) return; const message = error instanceof Error ? error.message : String(error); try { @@ -184,7 +187,7 @@ export async function listProviderHealth() { * @returns {Promise<{ ok: boolean, error?: string, provider: Object }>} */ export async function probeProvider(provider) { - const def = PROVIDER_DEFINITIONS[provider]; + const def = PROVIDER_DEFINITIONS[/** @type {keyof typeof PROVIDER_DEFINITIONS} */ (provider)]; if (!def) { throw Object.assign(new Error(`Unknown provider: ${provider}`), { status: 404 }); } @@ -205,6 +208,11 @@ export async function probeProvider(provider) { return { ok: true, provider: enrichRow(provider, def, row) }; } +/** + * @param {string} key + * @param {{ label: string, kind: string }} def + * @param {ProviderHealth|null} row + */ function enrichRow(key, def, row) { return { provider: key, diff --git a/apps/node-backend/src/services/quoteBackfillService.js b/apps/node-backend/src/services/quoteBackfillService.js index 5bf0fad5..2ad6e9dd 100644 --- a/apps/node-backend/src/services/quoteBackfillService.js +++ b/apps/node-backend/src/services/quoteBackfillService.js @@ -23,6 +23,65 @@ import { saveHistoricalPointsToDatabase, } from './priceProviderService.js'; +/** + * Provider-config subset of `InvestmentRow` (types/rows.js) this module reads + * off `HOLDING_WINDOW_SELECT` — a bespoke projection, not `SELECT i.*`, so it + * only lists the columns actually selected there. Matches + * `fetchHistoricalPrices`'s own `investment` param shape (priceProviderService.js). + * @typedef {object} HoldingWindowInvestment + * @property {number} id + * @property {string} asset_class + * @property {string} currency + * @property {string} price_provider + * @property {string|null} price_provider_id + * @property {string|null} symbol + * @property {string|null} price_provider_url + * @property {string|null} price_provider_latest_url + * @property {string|null} price_provider_latest_path + * @property {string|null} price_provider_history_url + * @property {string|null} price_provider_history_path + * @property {string|null} price_provider_history_ts_path + * @property {string|null} price_provider_history_price_path + */ + +/** + * One buy/gift/sell leg, as `HOLDING_WINDOW_SELECT` projects it for + * `computeHoldingWindows`. + * @typedef {object} HoldingWindowTx + * @property {number} id + * @property {string} type `portfolio_txn_type` enum, filtered to 'buy'|'gift'|'sell'. + * @property {string} date 'YYYY-MM-DD' — `to_char`-formatted in the query. + * @property {number} units + */ + +/** + * A raw `HOLDING_WINDOW_SELECT` row: `HoldingWindowInvestment`'s columns + * (unconverted — pg-raw) plus the `tx_*`-prefixed transaction columns. + * @typedef {object} HoldingWindowRow + * @property {number} id + * @property {string} asset_class + * @property {string} currency + * @property {string} price_provider + * @property {string|null} price_provider_id + * @property {string|null} symbol + * @property {string|null} price_provider_url + * @property {string|null} price_provider_latest_url + * @property {string|null} price_provider_latest_path + * @property {string|null} price_provider_history_url + * @property {string|null} price_provider_history_path + * @property {string|null} price_provider_history_ts_path + * @property {string|null} price_provider_history_price_path + * @property {number} tx_id + * @property {string} tx_type + * @property {string} tx_date 'YYYY-MM-DD' + * @property {string} tx_units NUMERIC — coerced with `Number()` by `mapRowToHoldingTx`. + */ + +/** + * A holding window: a continuous period where net units > 0. + * @typedef {{ fromDate: string, toDate: string|null }} HoldingWindow + */ + // ─── Constants ────────────────────────────────────────────────────────────── const SPIKE_RATIO_THRESHOLD = 3; // 3× single-day jump = spike @@ -191,7 +250,11 @@ const HOLDING_WINDOW_SELECT = ` // Priceable asset classes — the only ones with a provider quote series to backfill. const HOLDING_ASSET_CLASS_FILTER = `i.asset_class IN ('stock', 'etf', 'crypto', 'metals')`; -/** Map a HOLDING_WINDOW_SELECT row's investment columns to the provider-config object. */ +/** + * Map a HOLDING_WINDOW_SELECT row's investment columns to the provider-config object. + * @param {HoldingWindowRow} row + * @returns {HoldingWindowInvestment} + */ function mapRowToInvestment(row) { return { id: Number(row.id), @@ -210,7 +273,11 @@ function mapRowToInvestment(row) { }; } -/** Map a HOLDING_WINDOW_SELECT row's transaction columns to a holding-window tx. */ +/** + * Map a HOLDING_WINDOW_SELECT row's transaction columns to a holding-window tx. + * @param {HoldingWindowRow} row + * @returns {HoldingWindowTx} + */ function mapRowToHoldingTx(row) { return { id: Number(row.tx_id), @@ -226,7 +293,7 @@ function mapRowToHoldingTx(row) { * * Includes ALL investments with transactions, regardless of is_active flag. * - * @returns {Promise>} + * @returns {Promise>} */ export async function getInvestmentsWithHoldingWindows() { const result = await query( @@ -236,7 +303,9 @@ export async function getInvestmentsWithHoldingWindows() { [] ); + /** @type {HoldingWindowRow[]} */ const rows = result.rows || []; + /** @type {Map} */ const investmentMap = new Map(); for (const row of rows) { @@ -253,6 +322,7 @@ export async function getInvestmentsWithHoldingWindows() { } // Compute holding windows per investment + /** @type {Map} */ const resultMap = new Map(); for (const [invId, { investment, transactions }] of investmentMap) { const holdingWindows = computeHoldingWindows(transactions); @@ -268,7 +338,7 @@ export async function getInvestmentsWithHoldingWindows() { * Fetch holding windows for a single investment by ID. * * @param {number} investmentId - * @returns {Promise<{ investment: object, holdingWindows: Array } | null>} + * @returns {Promise<{ investment: HoldingWindowInvestment, holdingWindows: HoldingWindow[] } | null>} */ async function getInvestmentWithHoldingWindows(investmentId) { const result = await query( @@ -279,6 +349,7 @@ async function getInvestmentWithHoldingWindows(investmentId) { [Number(investmentId)] ); + /** @type {HoldingWindowRow[]} */ const rows = result.rows || []; if (rows.length === 0) return null; @@ -305,9 +376,14 @@ async function getStoredPriceDates(investmentId) { ORDER BY price_date`, [Number(investmentId)] ); - return (result.rows || []).map((row) => row.d); + return (result.rows || []).map((/** @type {{ d: string }} */ row) => row.d); } +/** + * @param {string} aYmd 'YYYY-MM-DD' + * @param {string} bYmd 'YYYY-MM-DD' + * @returns {number} + */ function _daysBetween(aYmd, bYmd) { const a = Date.parse(`${aYmd}T00:00:00.000Z`); const b = Date.parse(`${bYmd}T00:00:00.000Z`); @@ -354,8 +430,8 @@ export function holdingWindowsNeedBackfill(holdingWindows, storedDates, { thresh * Backfill quotes for a single investment across all its holding windows. * Fetches historical prices, sanitizes spikes, and persists cleaned data. * - * @param {object} investment - Investment object with provider config - * @param {Array<{ fromDate: string, toDate: string | null }>} holdingWindows + * @param {HoldingWindowInvestment} investment - Investment object with provider config + * @param {HoldingWindow[]} holdingWindows * @param {{ force?: boolean }} [opts] - force re-queries the provider even when the stored * series already spans the window endpoints (needed to repopulate interior gaps). * @returns {Promise<{ hasHistory: boolean, windowCount: number }>} @@ -594,7 +670,7 @@ export async function refreshQuotesForInvestment(investmentId) { * Delete asset_price_history rows that fall outside any holding window * for the given investments. * - * @param {Map} investmentWindows + * @param {Map} investmentWindows * @returns {Promise} Total rows deleted */ export async function cleanupStaleQuotes(investmentWindows) { @@ -651,6 +727,7 @@ export async function cleanupStaleQuotes(investmentWindows) { // ─── Private Helpers ──────────────────────────────────────────────────────── +/** @param {number|null|undefined} value */ function _isPositive(value) { return Number.isFinite(value) && value > 0; } diff --git a/apps/node-backend/src/services/recipientClusterService.js b/apps/node-backend/src/services/recipientClusterService.js index 38f26b6b..5a5807dc 100644 --- a/apps/node-backend/src/services/recipientClusterService.js +++ b/apps/node-backend/src/services/recipientClusterService.js @@ -34,6 +34,7 @@ const MAX_CLUSTERS = 50; * }>>} */ export async function findRecipientClusters({ minCount = 2 } = {}) { + /** @type {{ rows: Array<{ id: number, name: string, default_category_id: number|null }> }} */ const { rows } = await query(` SELECT id, name, default_category_id FROM recipients @@ -42,6 +43,7 @@ export async function findRecipientClusters({ minCount = 2 } = {}) { ORDER BY name `); + /** @type {Map>} */ const buckets = new Map(); for (const r of rows) { const upper = r.name.trim().toUpperCase(); diff --git a/apps/node-backend/src/services/recipientPatternService.js b/apps/node-backend/src/services/recipientPatternService.js index d28353b1..9ae64a6c 100644 --- a/apps/node-backend/src/services/recipientPatternService.js +++ b/apps/node-backend/src/services/recipientPatternService.js @@ -15,6 +15,15 @@ import { buildSetClauses } from '../lib/sqlClauses.js'; import { logger } from '../config/logger.js'; import { ValidationError, NotFoundError } from '../middleware/errorHandler.js'; +/** @typedef {import('../types/rows.js').RecipientMatchPatternRow} RecipientMatchPatternRow */ + +/** + * `RecipientMatchPatternRow` as `loadActivePatterns` projects it: `updated_at` + * is explicitly cast to text in that query (used verbatim in the compile-cache + * key), so it is `string` here rather than the raw row's `Date`. + * @typedef {Omit & { updated_at: string }} ActivePatternRow + */ + const MIN_LCP_LENGTH = 8; const STOP_PATTERNS = new Set([ @@ -24,20 +33,31 @@ const STOP_PATTERNS = new Set([ 'FROM', 'TO', ]); -/** Simple fixed-size LRU cache keyed by string. */ +/** + * Simple fixed-size LRU cache keyed by string. Only ever holds compiled + * `RegExp`s (the pattern cache below) — typed to that instead of a generic + * `` cache since there is exactly one caller. + * @param {number} maxSize + */ function makeLruCache(maxSize) { + /** @type {Map} */ const map = new Map(); return { + /** @param {string} k */ get(k) { if (!map.has(k)) return undefined; const v = map.get(k); map.delete(k); - map.set(k, v); + map.set(k, /** @type {RegExp} */ (v)); return v; }, + /** + * @param {string} k + * @param {RegExp} v + */ set(k, v) { if (map.has(k)) map.delete(k); - else if (map.size >= maxSize) map.delete(map.keys().next().value); + else if (map.size >= maxSize) map.delete(/** @type {string} */ (map.keys().next().value)); map.set(k, v); }, }; @@ -96,6 +116,8 @@ export function compilePattern(row) { * * This is a conservative static heuristic — not exhaustive, but catches the * patterns that cause catastrophic backtracking in practice. + * @param {string} pattern + * @returns {boolean} */ function hasRedosRisk(pattern) { // Remove character classes so [...+*] doesn't confuse the group scanner. @@ -138,7 +160,7 @@ export function validatePattern(row) { * Load all active patterns from DB, ordered by (priority ASC, id ASC). * Returns raw rows — callers compile as needed. * - * @returns {Promise>} + * @returns {Promise} */ export async function loadActivePatterns() { const { rows } = await query( @@ -159,7 +181,7 @@ export async function loadActivePatterns() { * are left for findBestRecipientMatches. * * @param {string[]} distinctRaw - * @param {Array} [preloadedPatterns] optional: pass already-loaded patterns to avoid a DB round-trip + * @param {ActivePatternRow[]} [preloadedPatterns] optional: pass already-loaded patterns to avoid a DB round-trip * @returns {Promise>} */ export async function applyPatterns(distinctRaw, preloadedPatterns) { @@ -256,6 +278,10 @@ function buildSqlRegexPattern({ pattern, pattern_kind }) { } } +// Cap the regex-preview scan: it pulls recipient rows into memory and filters +// in JS, so an unbounded SELECT could load the whole table for one preview. +const PREVIEW_REGEX_SCAN_CAP = 10_000; + /** * Preview how many recipients in the DB have a normalized_name or name * that the given pattern would match. @@ -264,12 +290,8 @@ function buildSqlRegexPattern({ pattern, pattern_kind }) { * outside the intended merge set. * * @param {{ pattern: string, pattern_kind: string, case_sensitive: boolean }} patternRow - * @returns {Promise<{ matchCount: number, recipientIds: number[] }>} + * @returns {Promise<{ matchCount: number, recipientIds: number[], truncated?: boolean }>} */ -// Cap the regex-preview scan: it pulls recipient rows into memory and filters -// in JS, so an unbounded SELECT could load the whole table for one preview. -const PREVIEW_REGEX_SCAN_CAP = 10_000; - export async function previewPatternMatches(patternRow) { const validation = validatePattern(patternRow); if (!validation.valid) { @@ -294,8 +316,8 @@ export async function previewPatternMatches(patternRow) { ); const truncated = rows.length > PREVIEW_REGEX_SCAN_CAP; const scanned = truncated ? rows.slice(0, PREVIEW_REGEX_SCAN_CAP) : rows; - const matched = scanned.filter((r) => re.test(String(r.name ?? '').toUpperCase())); - return { matchCount: matched.length, recipientIds: matched.map((r) => r.id), truncated }; + const matched = scanned.filter((/** @type {{ id: number, name: string|null }} */ r) => re.test(String(r.name ?? '').toUpperCase())); + return { matchCount: matched.length, recipientIds: matched.map((/** @type {{ id: number }} */ r) => r.id), truncated }; } const sqlPattern = buildSqlRegexPattern(patternRow); @@ -308,7 +330,7 @@ export async function previewPatternMatches(patternRow) { return { matchCount: rows.length, - recipientIds: rows.map((r) => r.id), + recipientIds: rows.map((/** @type {{ id: number }} */ r) => r.id), }; } @@ -395,7 +417,7 @@ export async function deletePattern(patternId) { * List all patterns for a recipient. * * @param {number} recipientId - * @returns {Promise} + * @returns {Promise} */ export async function listPatternsForRecipient(recipientId) { const { rows } = await query( diff --git a/apps/node-backend/src/services/reconcileService.js b/apps/node-backend/src/services/reconcileService.js index 6d34efa2..1c4dd4bd 100644 --- a/apps/node-backend/src/services/reconcileService.js +++ b/apps/node-backend/src/services/reconcileService.js @@ -119,7 +119,7 @@ export async function reconcileAccount(accountId, body) { drift: 0, statement_balance: Number(upd.rows[0].statement_balance), computed_balance: computed, - transaction: null, + transaction: /** @type {object|null} */ (null), }; } diff --git a/apps/node-backend/src/services/recurringDetectionService.js b/apps/node-backend/src/services/recurringDetectionService.js index 15df19ad..c22d75eb 100644 --- a/apps/node-backend/src/services/recurringDetectionService.js +++ b/apps/node-backend/src/services/recurringDetectionService.js @@ -14,6 +14,55 @@ import { logger } from '../config/logger.js'; import { addAll, divide, roundMoney, toDecimal } from '../lib/money.js'; import { median } from '../lib/math.js'; +/** + * The bespoke projection `detectRecurringPatterns`' query selects — not a + * plain `SELECT t.*`, so distinct from `TransactionRow` in types/rows.js. + * @typedef {object} RecurringCandidateRow + * @property {number} id + * @property {Date} date DATE + * @property {string} amount NUMERIC + * @property {string|null} currency + * @property {string|null} memo + * @property {string|null} bank_account + * @property {number|null} recipient_id + * @property {string|null} recipient_name + * @property {number|null} effective_category_id + * @property {string|null} category_name + */ + +/** + * @typedef {object} RecurringGroup + * @property {number|null} recipientId + * @property {string} recipientName + * @property {'income'|'expense'} direction + * @property {RecurringCandidateRow[]} transactions + */ + +/** + * One entry of `detectRecurringPatterns`'s result — a detected recurring + * pattern for one (recipient, direction) group. + * @typedef {object} RecurringPattern + * @property {number|null} recipientId + * @property {string} recipientName + * @property {'income'|'expense'} direction + * @property {string} detectedPattern + * @property {number} intervalDays + * @property {number} consistency + * @property {number} occurrences + * @property {number} averageAmount + * @property {number} latestAmount + * @property {string} currency + * @property {number|null} categoryId + * @property {string|null} categoryName + * @property {string|null} bankAccount + * @property {string|null} firstSeen 'YYYY-MM-DD' + * @property {string|null} lastSeen 'YYYY-MM-DD' + * @property {string} predictedNext 'YYYY-MM-DD' + * @property {Array<{ date: Date, previousAmount: number, newAmount: number, percentChange: number, direction: 'increased'|'decreased' }>} amountChanges + * @property {boolean} isAlreadyPlanned + * @property {number} confidence + */ + const MIN_OCCURRENCES = 3; // Minimum transactions to consider a pattern const INTERVAL_TOLERANCE = 0.25; // 25% tolerance for interval matching @@ -26,7 +75,7 @@ const INTERVAL_TOLERANCE = 0.25; // 25% tolerance for interval matching // short TTL is used: results are eventually consistent within RECURRING_CACHE_TTL_MS // of a transaction change, which is acceptable for a suggestion feature. const RECURRING_CACHE_TTL_MS = 3 * 60_000; // 3 minutes -/** @type {{ value: { patterns: any[], total: number }, expiresAt: number } | null} */ +/** @type {{ value: { patterns: RecurringPattern[], total: number }, expiresAt: number } | null} */ let recurringCache = null; /** Test-only: drop the cached recurring-patterns result. */ @@ -45,6 +94,8 @@ const INTERVAL_PATTERNS = [ /** * Detect the most likely recurrence pattern from a series of intervals. + * @param {number[]} intervals + * @returns {{ pattern: string, avgDays: number, medianDays: number, consistency: number, customIntervalDays?: number }|null} */ function detectInterval(intervals) { if (intervals.length === 0) return null; @@ -92,10 +143,13 @@ function detectInterval(intervals) { /** * Detect amount changes in a recurring pattern. + * @param {RecurringCandidateRow[]} transactions + * @returns {Array<{ date: Date, previousAmount: number, newAmount: number, percentChange: number, direction: 'increased'|'decreased' }>} */ function detectAmountChanges(transactions) { if (transactions.length < 2) return []; + /** @type {Array<{ date: Date, previousAmount: number, newAmount: number, percentChange: number, direction: 'increased'|'decreased' }>} */ const changes = []; const sorted = [...transactions].sort((a, b) => { const aTime = new Date(a?.date).getTime(); @@ -147,14 +201,20 @@ export async function detectRecurringPatterns() { // history adds scan cost without changing the result. Without the bound // this was a full-table scan on every call (including via the aiChat // getRecurringDetected tool). + // Effective category is the canonical 3-level resolution (own → recipient + // default → PRIMARY recipient's default), matching transactionRepository — + // a row recorded under an alias whose PRIMARY carries the default category + // must not report a null category_name here while the transactions list + // shows it categorised. const result = await query(` SELECT t.id, t.date, t.amount, t.currency, t.memo, t.bank_account, t.recipient_id, r.name AS recipient_name, - t.category_id, + COALESCE(t.category_id, r.default_category_id, pr.default_category_id) AS effective_category_id, COALESCE(c.general || ':' || c.detail, NULL) AS category_name FROM transactions t LEFT JOIN recipients r ON t.recipient_id = r.id - LEFT JOIN categories c ON COALESCE(t.category_id, r.default_category_id) = c.id + LEFT JOIN recipients pr ON r.primary_recipient_id = pr.id + LEFT JOIN categories c ON COALESCE(t.category_id, r.default_category_id, pr.default_category_id) = c.id WHERE t.is_active = true AND t.recipient_id IS NOT NULL AND t.date >= CURRENT_DATE - INTERVAL '3 years' @@ -162,6 +222,7 @@ export async function detectRecurringPatterns() { `); if (result.rows.length === 0) { + /** @type {{ patterns: RecurringPattern[], total: number }} */ const empty = { patterns: [], total: 0 }; recurringCache = { value: empty, expiresAt: Date.now() + RECURRING_CACHE_TTL_MS }; return empty; @@ -178,8 +239,9 @@ export async function detectRecurringPatterns() { // that is also occasionally reimbursed) into one averaged "pattern" that // matched neither real flow — amounts go through .abs() below, so the // sign distinction would otherwise be lost entirely. + /** @type {Record} */ const byRecipient = {}; - for (const row of result.rows) { + for (const row of /** @type {RecurringCandidateRow[]} */ (result.rows)) { const direction = Number(row.amount) < 0 ? 'expense' : 'income'; const key = `${row.recipient_id}:${direction}`; if (!byRecipient[key]) { @@ -209,6 +271,7 @@ export async function detectRecurringPatterns() { for (const row of plannedResult.rows) plannedRecipientIds.add(row.recipient_id); } + /** @type {RecurringPattern[]} */ const patterns = []; for (const group of Object.values(byRecipient)) { @@ -272,7 +335,7 @@ export async function detectRecurringPatterns() { averageAmount: roundMoney(avgAmount), latestAmount: roundMoney(latestAmount), currency, - categoryId: txns[txns.length - 1].category_id, + categoryId: txns[txns.length - 1].effective_category_id, categoryName: txns[txns.length - 1].category_name, bankAccount: txns[txns.length - 1].bank_account, // DATE columns: calendar-day strings, not raw pg Dates (which diff --git a/apps/node-backend/src/services/routeManifest.js b/apps/node-backend/src/services/routeManifest.js index 274ec4de..50705269 100644 --- a/apps/node-backend/src/services/routeManifest.js +++ b/apps/node-backend/src/services/routeManifest.js @@ -5,6 +5,28 @@ * a flat list of { method, path } entries. Consumed by GET /api/admin/endpoints. */ +/** + * The slice of an Express `Layer` this module actually reads. Deliberately + * structural rather than `import('express').Layer`: express ships no type + * declarations and `@types/express` is not a dependency, so referencing its + * types resolves to an implicit `any` (TS7016) under `noImplicitAny` — same + * reasoning as `QueryRunner` in types/rows.js for `pg`. + * + * @typedef {object} ExpressLayer + * @property {{ source?: string }} [regexp] + * @property {string} [_mountPath] + * @property {{ path: string, methods: Record }} [route] + * @property {{ stack?: ExpressLayer[], _mountPath?: string }} [handle] + */ + +/** + * The slice of an Express `Application` this module actually reads/calls. + * @typedef {object} ExpressApp + * @property {{ stack?: ExpressLayer[] }} [router] + * @property {{ stack?: ExpressLayer[] }} [_router] + * @property {(path: string, ...fns: any[]) => void} use + */ + /** @type {{ method: string, path: string }[]} */ let manifest = []; @@ -13,7 +35,7 @@ let manifest = []; * Express converts the mount path into a regexp with source like: * ^\/api\/transactions\/?(?=\/|$) * We strip the ^ prefix and the \/?(?=\/|$) suffix, then unescape \/ → /. - * @param {import('express').IRouterMatcher} layer + * @param {ExpressLayer} layer * @returns {string|null} */ function extractPrefix(layer) { @@ -30,11 +52,12 @@ function extractPrefix(layer) { /** * Recursively scan a Layer array and collect route definitions. - * @param {any[]} stack + * @param {ExpressLayer[]} stack * @param {string} prefix * @returns {{ method: string, path: string }[]} */ function scanStack(stack, prefix) { + /** @type {{ method: string, path: string }[]} */ const routes = []; for (const layer of stack) { @@ -59,7 +82,7 @@ function scanStack(stack, prefix) { /** * Scan the Express app's router stack and store the manifest. * Call once after all routes are registered in main.js. - * @param {import('express').Application} app + * @param {ExpressApp} app */ export function buildRouteManifest(app) { const router = app.router ?? app._router; @@ -69,7 +92,7 @@ export function buildRouteManifest(app) { /** * Mount a router at a path and tag it with _mountPath so scanStack can resolve * the prefix in Express v5 (which no longer exposes layer.regexp). - * @param {import('express').Application} app + * @param {ExpressApp} app * @param {string} path * @param {...any} fns */ diff --git a/apps/node-backend/src/services/tagService.js b/apps/node-backend/src/services/tagService.js index 55da09e2..7faa8e69 100644 --- a/apps/node-backend/src/services/tagService.js +++ b/apps/node-backend/src/services/tagService.js @@ -71,7 +71,10 @@ export const tagService = { return updated; }, - /** Soft-delete (deactivate) a tag. */ + /** + * Soft-delete (deactivate) a tag. + * @param {number} id + */ async softDelete(id) { const deleted = await tagRepository.softDelete(id); if (!deleted) throw new NotFoundError(`Tag ${id} not found`); diff --git a/apps/node-backend/src/services/tax/deductionCandidatesService.js b/apps/node-backend/src/services/tax/deductionCandidatesService.js index a34a1424..b6a06d90 100644 --- a/apps/node-backend/src/services/tax/deductionCandidatesService.js +++ b/apps/node-backend/src/services/tax/deductionCandidatesService.js @@ -20,6 +20,9 @@ import { transactionRepository } from '../../repositories/transactionRepository. import { toDecimal, roundToCents } from '../../lib/money.js'; import { classifyDeduction } from './deductionClassifier.js'; +/** A decimal.js money value, as produced by the shared `toDecimal` helper. */ +/** @typedef {ReturnType} Money */ + /** * Compute per-deduction-type candidate totals for one calendar year. * @@ -51,6 +54,7 @@ export async function computeDeductionCandidates({ year }) { }); // Pass 1: group outflows by raw category name, classifying each category. + /** @type {Map} */ const byCategory = new Map(); for (const row of rows) { const amount = toDecimal(row.amount ?? 0); @@ -75,6 +79,7 @@ export async function computeDeductionCandidates({ year }) { // Pass 2: roll categories up per deduction type, keeping the contributors // nested (totals stay Decimal until the final rounding). + /** @type {Map }>} */ const typeAgg = new Map(); for (const entry of byCategory.values()) { const agg = typeAgg.get(entry.deductionType) || { total: toDecimal(0), categories: [] }; diff --git a/apps/node-backend/src/services/tax/deductionClassifier.js b/apps/node-backend/src/services/tax/deductionClassifier.js index c42969d0..8738e74f 100644 --- a/apps/node-backend/src/services/tax/deductionClassifier.js +++ b/apps/node-backend/src/services/tax/deductionClassifier.js @@ -62,12 +62,31 @@ function tokenize(value) { .filter(Boolean); } -/** True when any token in `tokens` is exactly one of `words`. */ +/** + * Tokenized category parts a rule tests against: `all` is general + detail + * concatenated. + * + * @typedef {{ general: string[], detail: string[], all: string[] }} CategoryTokens + */ + +/** + * True when any token in `tokens` is exactly one of `words`. + * + * @param {string[]} tokens + * @param {readonly string[]} words + * @returns {boolean} + */ function hasWord(tokens, words) { return tokens.some((t) => words.includes(t)); } -/** True when `phrase` (an array of words) appears as consecutive tokens. */ +/** + * True when `phrase` (an array of words) appears as consecutive tokens. + * + * @param {string[]} tokens + * @param {readonly string[]} phrase + * @returns {boolean} + */ function hasPhrase(tokens, phrase) { for (let i = 0; i + phrase.length <= tokens.length; i += 1) { let matched = true; @@ -86,6 +105,10 @@ function hasPhrase(tokens, phrase) { * Phrase lookup within EITHER the general or the detail part (phrases do not * span the general/detail boundary — that boundary is a deliberate split the * user made, not adjacent words). + * + * @param {CategoryTokens} cat + * @param {readonly (readonly string[])[]} phrases + * @returns {boolean} */ function hasAnyPhrase(cat, phrases) { return phrases.some((p) => hasPhrase(cat.general, p) || hasPhrase(cat.detail, p)); @@ -152,6 +175,8 @@ const MORTGAGE_INTEREST_WORDS = Object.freeze(['HYPOTHEEKRENTE']); * - groupInsurance is checked BEFORE lifeInsurance so an employer-scheme * name like "GROUP LIFE INSURANCE" lands in the more specific * groupInsurance (2nd pillar), not lifeInsurance. + * + * @type {ReadonlyArray<{ type: string, test: (cat: CategoryTokens) => boolean }>} */ const RULES = Object.freeze([ { diff --git a/apps/node-backend/src/services/transactionBulkService.js b/apps/node-backend/src/services/transactionBulkService.js index 2fce7e5f..eb995873 100644 --- a/apps/node-backend/src/services/transactionBulkService.js +++ b/apps/node-backend/src/services/transactionBulkService.js @@ -36,8 +36,11 @@ export async function bulkTagTransactions({ transactionIds, addSlugs, removeSlug throw new ValidationError('transaction_ids contains no valid IDs'); } + /** @type {number[]} */ const addTagIds = []; + /** @type {number[]} */ const removeTagIds = []; + /** @type {string[]} */ const allUnknown = []; if (addSlugs.length > 0) { @@ -45,7 +48,7 @@ export async function bulkTagTransactions({ transactionIds, addSlugs, removeSlug 'SELECT id, slug FROM tags WHERE slug = ANY($1::text[]) AND is_active = true', [addSlugs], ); - const found = new Map(r.rows.map((row) => [row.slug, row.id])); + const found = new Map(r.rows.map((/** @type {{ id: number, slug: string }} */ row) => [row.slug, row.id])); for (const s of addSlugs) { if (!found.has(s)) allUnknown.push(s); else addTagIds.push(found.get(s)); @@ -57,7 +60,7 @@ export async function bulkTagTransactions({ transactionIds, addSlugs, removeSlug 'SELECT id, slug FROM tags WHERE slug = ANY($1::text[])', [removeSlugs], ); - const found = new Map(r.rows.map((row) => [row.slug, row.id])); + const found = new Map(r.rows.map((/** @type {{ id: number, slug: string }} */ row) => [row.slug, row.id])); for (const s of removeSlugs) { if (!found.has(s)) allUnknown.push(s); else removeTagIds.push(found.get(s)); @@ -84,7 +87,7 @@ export async function bulkTagTransactions({ transactionIds, addSlugs, removeSlug [txIds, addTagIds], ); added = r.rows.length; - r.rows.forEach((row) => affectedTxIds.add(row.transaction_id)); + r.rows.forEach((/** @type {{ transaction_id: number }} */ row) => affectedTxIds.add(row.transaction_id)); } if (removeTagIds.length > 0) { @@ -95,7 +98,7 @@ export async function bulkTagTransactions({ transactionIds, addSlugs, removeSlug [txIds, removeTagIds], ); removed = r.rows.length; - r.rows.forEach((row) => affectedTxIds.add(row.transaction_id)); + r.rows.forEach((/** @type {{ transaction_id: number }} */ row) => affectedTxIds.add(row.transaction_id)); } return { added, removed, transactions_affected: affectedTxIds.size }; @@ -139,6 +142,7 @@ export async function bulkUpdateTransactions({ ids, filter, fields }) { const txIds = await resolveBulkSelection({ ids, filter }); + /** @type {string[]} */ const setClauses = []; /** @type {Array} */ const params = [txIds]; diff --git a/apps/node-backend/src/services/transactionExport.js b/apps/node-backend/src/services/transactionExport.js index e941e817..7f013fc0 100644 --- a/apps/node-backend/src/services/transactionExport.js +++ b/apps/node-backend/src/services/transactionExport.js @@ -12,6 +12,40 @@ import { toDecimal } from '../lib/money.js'; import { toYmd } from '../utils/portfolioMath.js'; import { escapeCsvValue } from '../lib/csv.js'; +/** + * The slice of an Express `Response` this module actually calls. Deliberately + * structural rather than `import('express').Response`: express ships no type + * declarations and `@types/express` is not a dependency, so referencing its + * types resolves to an implicit `any` (TS7016) under `noImplicitAny` — same + * reasoning as `ExpressApp` in services/routeManifest.js and `QueryRunner` in + * types/rows.js for `pg`. + * @typedef {object} ExpressResponse + * @property {(name: string, value: string) => void} setHeader + * @property {(chunk: string) => boolean} write + * @property {(event: string, cb: () => void) => void} [once] + * @property {boolean} [headersSent] + * @property {() => void} end + */ + +/** + * A row as selected by `buildExportChunkSql` — a projection of + * `EnrichedTransactionRow`, not the full row (no `is_active`, `recipient_id`, + * etc. — only the columns the export needs). + * @typedef {object} ExportTransactionRow + * @property {number} id + * @property {Date} date DATE — local-midnight `Date`; read via `toYmd`, never `String()`/`toISOString()`. + * @property {string|null} bank_account + * @property {number|null} [account_id] + * @property {string|null} recipient_name + * @property {string|null} memo + * @property {string} amount NUMERIC(18,4) — pg emits NUMERIC as a string. + * @property {string|null} currency + * @property {string|null} balance NUMERIC(15,2) — pg emits NUMERIC as a string; null on manual rows. + * @property {string} category_name '' when the transaction has no resolved category. + * @property {string|null} comment + * @property {string[]} tags tag slugs, `{}` (empty array) when none. + */ + export const EXPORT_CHUNK_SIZE = 1000; export const EXPORT_MAX_LIST_SIZE = 50; @@ -39,6 +73,10 @@ export function buildNdjsonFilename() { return `transactions_export_${buildExportTimestamp()}.ndjson`; } +/** + * @param {string} whereSql + * @returns {string} + */ function buildExportProbeSql(whereSql) { return `SELECT 1 ${EXPORT_JOINS_SQL} WHERE ${whereSql} LIMIT 1`; } @@ -48,7 +86,7 @@ function buildExportProbeSql(whereSql) { * buffer is full `res.write` returns false — without this a slow client lets * rows buffer unboundedly in memory on a large export. * - * @param {import('express').Response} res + * @param {ExpressResponse} res * @param {string} chunk * @returns {Promise} */ @@ -59,6 +97,13 @@ function writeWithBackpressure(res, chunk) { return new Promise((resolve) => res.once('drain', resolve)); } +/** + * @param {string} whereSql + * @param {number} limitParamIdx + * @param {number} [cursorDateParamIdx] + * @param {number} [cursorIdParamIdx] + * @returns {string} + */ function buildExportChunkSql(whereSql, limitParamIdx, cursorDateParamIdx, cursorIdParamIdx) { // Keyset pagination: each chunk continues strictly after the previous chunk's // last (date, id) instead of OFFSET. OFFSET across separate pool queries (new @@ -72,10 +117,17 @@ function buildExportChunkSql(whereSql, limitParamIdx, cursorDateParamIdx, cursor SELECT t.id, t.date, t.bank_account, t.account_id, COALESCE(pr.name, r.name) AS recipient_name, t.memo, t.amount, t.currency, t.balance, + -- Same branch order as transactionRepository's CATEGORY_NAME_SQL: + -- own (c) → recipient default (rc) → primary-recipient default (pc), + -- mirroring COALESCE(t.category_id, r.default_category_id, + -- pr.default_category_id). It used to test pc before rc, so an + -- ALIAS recipient with its own default under a differently-defaulted + -- PRIMARY exported the primary's category name while the + -- transactions list showed the alias's. CASE WHEN c.id IS NOT NULL THEN c.general || ':' || c.detail - WHEN pc.id IS NOT NULL THEN pc.general || ':' || pc.detail WHEN rc.id IS NOT NULL THEN rc.general || ':' || rc.detail + WHEN pc.id IS NOT NULL THEN pc.general || ':' || pc.detail ELSE '' END AS category_name, t.comment, @@ -95,6 +147,11 @@ function buildExportChunkSql(whereSql, limitParamIdx, cursorDateParamIdx, cursor `; } +/** + * @param {ExportTransactionRow & { running_balance?: number }} row + * @param {{ includeBalance?: boolean }} [opts] + * @returns {string} + */ function buildCsvRow(row, { includeBalance = false } = {}) { const cols = [ // toYmd, not the raw pg Date: String() of it is "Wed Jul 01 2026 …" — @@ -114,6 +171,10 @@ function buildCsvRow(row, { includeBalance = false } = {}) { return cols.join(','); } +/** + * @param {ExportTransactionRow} row + * @returns {string} + */ function buildNdjsonRow(row) { return JSON.stringify({ id: row.id, @@ -136,17 +197,18 @@ function buildNdjsonRow(row) { * Build a probe + iterate-in-chunks pipeline that streams export rows to `res`. * Returns `{ rowCount }` when the stream completes cleanly. * - * @param {import('express').Response} res + * @param {ExpressResponse} res * @param {{ * whereSql: string, * params: any[], * nextParamIdx: number, * contentType: string, * filename: string, - * writeHeader?: (res: import('express').Response) => void, - * formatRow: (row: any, rowIndex: number) => string, + * writeHeader?: (res: ExpressResponse) => void, + * formatRow: (row: ExportTransactionRow, rowIndex: number) => string, * label: string, * }} opts + * @returns {Promise<{ rowCount: number }>} */ async function streamExport(res, { whereSql, params, nextParamIdx, contentType, filename, writeHeader, formatRow, label }) { const probe = await dbQuery(buildExportProbeSql(whereSql), params); @@ -160,11 +222,14 @@ async function streamExport(res, { whereSql, params, nextParamIdx, contentType, if (writeHeader) writeHeader(res); // Keyset cursor (last streamed (date, id)). First chunk has no cursor. + /** @type {string|null} */ let cursorDate = null; + /** @type {number|null} */ let cursorId = null; let rowCount = 0; try { while (true) { + /** @type {{ rows: ExportTransactionRow[] }} */ const chunk = cursorDate == null ? await dbQuery(buildExportChunkSql(whereSql, nextParamIdx), [...params, EXPORT_CHUNK_SIZE]) : await dbQuery( @@ -195,12 +260,18 @@ async function streamExport(res, { whereSql, params, nextParamIdx, contentType, } } +/** + * @param {ExpressResponse} res + * @param {{ whereSql: string, params: any[], nextParamIdx: number, includeBalance?: boolean }} args + * @returns {Promise<{ rowCount: number }>} + */ export async function streamCsvExport(res, { whereSql, params, nextParamIdx, includeBalance = false }) { // Partitioned by account_id (ADR-088): the list endpoint's window partitions // by account because a stream spanning multiple accounts otherwise sums them // into one meaningless cross-account total. Kept as Decimals across the whole // stream — collapsing to a JS number each row re-ingested a drifted float // into the next step's running balance. + /** @type {Map} */ const runningBalances = new Map(); return streamExport(res, { whereSql, @@ -227,6 +298,11 @@ export async function streamCsvExport(res, { whereSql, params, nextParamIdx, inc }); } +/** + * @param {ExpressResponse} res + * @param {{ whereSql: string, params: any[], nextParamIdx: number }} args + * @returns {Promise<{ rowCount: number }>} + */ export async function streamNdjsonExport(res, { whereSql, params, nextParamIdx }) { return streamExport(res, { whereSql, @@ -245,6 +321,9 @@ export async function streamNdjsonExport(res, { whereSql, params, nextParamIdx } * Helper used by the POST /bulk-export route: builds a WHERE clause that * targets a fixed list of ids while leaving the param numbering compatible * with the chunk-SQL builder (which appends LIMIT/OFFSET). + * + * @param {number[]} ids + * @returns {{ whereSql: string, params: [number[]], nextParamIdx: number }} */ export function buildIdListWhere(ids) { return { diff --git a/apps/node-backend/src/services/transactionService.js b/apps/node-backend/src/services/transactionService.js index 48367f70..444a3714 100644 --- a/apps/node-backend/src/services/transactionService.js +++ b/apps/node-backend/src/services/transactionService.js @@ -30,6 +30,18 @@ import { logger } from '../config/logger.js'; * unambiguously matches one; an auto-link failure must never fail the create, * so it is caught and logged. * + * @param {{ + * transaction_date?: string, + * date?: string, + * bank_account?: string|null, + * recipient_id?: number|null, + * amount: number|string, + * memo?: string|null, + * currency?: string|null, + * category_id?: number|null, + * comment?: string|null, + * tags?: string[]|null, + * }} data zod-validated POST body — loose passthrough (see module doc above). * @returns {Promise<{ transaction: object, autoLink: { autoLinkedCount: number, links: Array<{ plannedTransactionId?: number }> } }>} */ async function createManualTransaction(data) { @@ -77,6 +89,7 @@ async function createManualTransaction(data) { // Auto-clear a matching planned payment if this transaction unambiguously // matches one. Never let an auto-link failure fail the create. + /** @type {{ autoLinkedCount: number, links: Array<{ plannedTransactionId?: number, transactionId?: number }> }} */ let autoLink = { autoLinkedCount: 0, links: [] }; try { autoLink = await autoLinkTransactions([transaction]); diff --git a/apps/node-backend/src/services/transferReconciliationService.js b/apps/node-backend/src/services/transferReconciliationService.js index f39aee8d..0cc1ff7d 100644 --- a/apps/node-backend/src/services/transferReconciliationService.js +++ b/apps/node-backend/src/services/transferReconciliationService.js @@ -89,6 +89,10 @@ export async function getTransferSuggestions({ windowDays = DEFAULT_WINDOW_DAYS * - the two legs sit on different accounts (a transfer moves money between * accounts; same-account is meaningless) * - opposite signs (one outflow, one inflow) + * + * @param {number} aId + * @param {number} bId + * @returns {Promise<{ok: boolean}>} */ export async function markTransfer(aId, bId) { await withTransaction(async () => { @@ -133,6 +137,9 @@ export async function markTransfer(aId, bId) { * auto-pair with its true counterpart C. Excluding just the pair gives both: A↔B * never comes back on its own, A↔C still can. A manual markTransfer of a * dismissed pair still works — dismissals only gate auto-detection. + * + * @param {number} id + * @returns {Promise<{ok: boolean}>} */ export async function unmarkTransfer(id) { await withTransaction(async () => { @@ -175,7 +182,9 @@ export async function unmarkTransfer(id) { export const RECONCILE_DEBOUNCE_MS = 5000; export const RECONCILE_MAX_WAIT_MS = 10000; +/** @type {ReturnType|undefined} */ let reconcileTimer; +/** @type {number|null} */ let reconcileDeadline = null; // epoch ms the current burst must flush by export function scheduleReconcile() { diff --git a/apps/node-backend/src/types/rows.js b/apps/node-backend/src/types/rows.js index 3a301839..ea5806c6 100644 --- a/apps/node-backend/src/types/rows.js +++ b/apps/node-backend/src/types/rows.js @@ -293,6 +293,26 @@ * }} EnrichedRecipientRow */ +/** + * A row of `recipient_match_patterns` as returned by `SELECT *` (migration + * 0015). `pattern_kind` is CHECK-constrained to 'regex'|'glob'|'literal_prefix', + * `source` to 'user'|'suggested'|'system' — kept as plain `string` here since + * Postgres CHECK constraints are not reflected in the driver's row shape. + * + * @typedef {object} RecipientMatchPatternRow + * @property {number} id SERIAL + * @property {number} recipient_id FK → recipients, ON DELETE CASCADE + * @property {string} pattern + * @property {string} pattern_kind 'regex'|'glob'|'literal_prefix', DEFAULT 'literal_prefix' + * @property {boolean} case_sensitive DEFAULT false + * @property {number} priority DEFAULT 100 + * @property {boolean} is_active DEFAULT true + * @property {string} source 'user'|'suggested'|'system', DEFAULT 'user' + * @property {string|null} notes + * @property {Date} created_at TIMESTAMPTZ + * @property {Date} updated_at TIMESTAMPTZ + */ + // --------------------------------------------------------------------------- // Categories // --------------------------------------------------------------------------- @@ -763,4 +783,268 @@ * @property {number} transactions_remaining */ +// --------------------------------------------------------------------------- +// Import staging +// --------------------------------------------------------------------------- + +/** + * A row of `import_staging_rows` — the transaction import pipeline's work + * table (migration 0001; `match_source` / `matched_pattern_id` / + * `match_similarity` / `user_override_recipient_id` added by 0015, + * `override_category_id` by 0020). + * + * Everything the adapter produced is nullable here on purpose: the STAGE phase + * writes whatever it parsed and the VALIDATE phase is what rejects rows. The + * pipeline phases select column subsets, so use `Pick<>` at the call site. + * + * `amount` and `balance` are NUMERIC → pg strings. `tx_date` is a DATE → a + * local-midnight `Date`; validate.js and commit.js both project it as + * `to_char(tx_date, 'YYYY-MM-DD')` instead, precisely so the fallback hash and + * the insert can't shift a day (see the comment at the top of validate.js). + * `match_similarity` is REAL, which pg DOES emit as a number. + * + * @typedef {object} ImportStagingRow + * @property {string} id BIGSERIAL — string, not number. + * @property {string} batch_id BIGINT — string. + * @property {number} row_index INTEGER + * @property {'pending'|'validated'|'matched'|'committed'|'duplicate'|'error'} status + * @property {Date|null} tx_date DATE — local-midnight `Date` when selected raw. + * @property {string|null} bank_account + * @property {string|null} recipient_raw + * @property {string|null} memo + * @property {string|null} amount NUMERIC(20,4) — string. + * @property {string|null} currency + * @property {string|null} balance NUMERIC(20,4) — string. + * @property {string|null} recipient_account + * @property {string|null} recipient_address + * @property {string|null} recipient_bank_name + * @property {string|null} comment + * @property {string|null} raw_data + * @property {string|null} tx_hash sha256 hex of raw_data (or the field fallback). + * @property {number|null} resolved_recipient_id + * @property {number|null} resolved_bank_account_id + * @property {string|null} error_message + * @property {'pattern'|'exact'|'fuzzy'|'new'|null} [match_source] migration 0015. + * @property {number|null} [matched_pattern_id] migration 0015. + * @property {number|null} [match_similarity] REAL — a number, not a string (migration 0015). + * @property {number|null} [user_override_recipient_id] migration 0015. + * @property {number|null} [override_category_id] migration 0020. + * @property {Date} created_at TIMESTAMPTZ + */ + +/** + * A row of `portfolio_import_batches` (migration 0040; `account_id` added by + * 0057, `is_brokerage` by 0060, the 'complete_with_errors' status by 0081). + * + * @typedef {object} PortfolioImportBatchRow + * @property {string} id BIGSERIAL — string, not number. + * @property {string} adapter_name + * @property {string|null} source_filename + * @property {string|null} source_size_bytes BIGINT — string. + * @property {any} custom_config JSONB — pg hands it back already parsed. + * @property {string|null} default_asset_class `asset_class` enum. + * @property {string|null} default_type `portfolio_txn_type` enum. + * @property {'pending'|'staging'|'validating'|'matching'|'awaiting_review'|'committing'|'complete'|'complete_with_errors'|'failed'|'aborted'} status + * @property {number} rows_total + * @property {number} rows_imported + * @property {number} rows_duplicate + * @property {number} rows_error + * @property {string|null} error_summary + * @property {Date} started_at + * @property {Date|null} completed_at + * @property {number|null} account_id FK → accounts (migration 0057). + * @property {boolean} is_brokerage migration 0060. + */ + +/** + * A row of `portfolio_import_staging_rows` — the portfolio import pipeline's + * work table (migration 0040; `route` added by 0060). + * + * Every parsed field is nullable: STAGE writes what the adapter produced and + * VALIDATE is what rejects rows. All NUMERIC columns are pg strings; + * `match_similarity` is REAL, which pg DOES emit as a number. `tx_date` is a + * DATE, so raw selects hand back a local-midnight `Date` — validate.js formats + * it with LOCAL getters (`toYmd`) on purpose. + * + * @typedef {object} PortfolioImportStagingRow + * @property {string} id BIGSERIAL — string, not number. + * @property {string} batch_id BIGINT — string. + * @property {number} row_index INTEGER + * @property {'pending'|'validated'|'matched'|'committed'|'duplicate'|'error'} status + * @property {Date|null} tx_date DATE — local-midnight `Date` when selected raw. + * @property {string|null} type_raw the CSV's own type label, pre-normalization. + * @property {string|null} type `portfolio_txn_type` enum — stamped by VALIDATE. + * @property {string|null} symbol_raw + * @property {string|null} name_raw + * @property {string|null} units NUMERIC(18,8) — string. + * @property {string|null} price_per_unit NUMERIC(18,6) — string. + * @property {string|null} amount NUMERIC(18,4) — string. + * @property {string|null} fees NUMERIC(18,4) — string. + * @property {string|null} taxes NUMERIC(18,4) — string. + * @property {string|null} currency + * @property {string|null} fx_rate_to_eur NUMERIC(20,10) — string. + * @property {string|null} note + * @property {string|null} raw_data + * @property {string|null} tx_hash + * @property {number|null} resolved_investment_id + * @property {number|null} user_override_investment_id + * @property {'symbol'|'name_exact'|null} match_source + * @property {number|null} match_similarity REAL — a number, not a string. + * @property {number|null} committed_txn_id + * @property {string|null} error_message + * @property {'cash'|'portfolio'|null} [route] migration 0060 — brokerage routing (ADR-095). + * @property {Date} created_at TIMESTAMPTZ + */ + +// --------------------------------------------------------------------------- +// Asset price history +// --------------------------------------------------------------------------- + +/** + * A row of `asset_price_history` (migration 0001; the FK to `investments` was + * added by 0026 and is dropped again by priceCache's `_dropForeignKey`). + * + * `close_price` is NUMERIC so pg emits it as a string, and `price_date` is a + * DATE so pg emits a local-midnight `Date` — `dateOnlyToTimestampMs` exists + * precisely to unpick that (see its comment: treating it as a string NaN'd out + * every cached read). + * + * @typedef {object} AssetPriceHistoryRow + * @property {number} id SERIAL + * @property {number} investment_id INTEGER NOT NULL + * @property {Date} price_date DATE — a local-midnight `Date`, NOT a 'YYYY-MM-DD' string. + * @property {string} close_price NUMERIC(18,6) — pg emits NUMERIC as a string. + * @property {string} source VARCHAR(50) DEFAULT 'provider' + * @property {Date} fetched_at TIMESTAMPTZ + * @property {Date|null} updated_at TIMESTAMPTZ + */ + +/** + * One point of a price series as the price layer passes it around: an + * epoch-millis timestamp (pinned to UTC noon of the calendar day) and a + * finite, strictly-positive price. Produced by `normalizeHistoryPoints`, which + * drops anything failing those invariants. + * + * @typedef {object} PricePoint + * @property {number} timestampMs + * @property {number} price + */ + +// --------------------------------------------------------------------------- +// Portfolio performance snapshots +// --------------------------------------------------------------------------- + +/** + * A row of `portfolio_performance_snapshots` as returned by `SELECT *` + * (migration 0018; `value_fx_neutral` added by migration 0039). + * + * Every money/percentage column is NUMERIC, so pg emits it as a string — the + * consumers all run them through `toDecimal`/`toNumber`. Every column except + * `value_fx_neutral` is NOT NULL with a DEFAULT. + * + * `value_fx_neutral` is optional AND nullable on purpose: `getSnapshots` uses + * `SELECT *` precisely so the projection still works on a database that has not + * applied 0039 (the property is then absent, not null). + * + * @typedef {object} PortfolioPerformanceSnapshotRow + * @property {number} id SERIAL + * @property {Date} snapshot_date DATE — a local-midnight `Date`, NOT a 'YYYY-MM-DD' string. + * @property {string} invested NUMERIC(18,6) + * @property {string} value NUMERIC(18,6) + * @property {string} stocks_etfs_value NUMERIC(18,6) + * @property {string} crypto_value NUMERIC(18,6) + * @property {string} metals_value NUMERIC(18,6) + * @property {string} cash_value NUMERIC(18,6) + * @property {string} gain_loss NUMERIC(18,6) + * @property {string} return_pct NUMERIC(10,4) + * @property {string} inflation_adjusted_value NUMERIC(18,6) + * @property {string} cumulative_inflation NUMERIC(10,4) DEFAULT 1 + * @property {string} real_return_pct NUMERIC(10,4) + * @property {string} stocks_etfs_invested NUMERIC(18,6) + * @property {string} crypto_invested NUMERIC(18,6) + * @property {string} metals_invested NUMERIC(18,6) + * @property {string} currency VARCHAR(3) DEFAULT 'EUR' + * @property {Date} computed_at TIMESTAMPTZ + * @property {string|null} [value_fx_neutral] NUMERIC(18,2), migration 0039 — absent on un-migrated databases. + */ + +// --------------------------------------------------------------------------- +// Exchange rates +// --------------------------------------------------------------------------- + +/** + * A row of `exchange_rates` (migration 0001; `fetched_at` made NOT NULL with a + * default in migration 0022). + * + * `rate_to_eur` is NUMERIC(20,10) so pg emits it as a string — every consumer + * runs it through `toNumber(toDecimal(...))`. `is_latest` has `DEFAULT false` + * but no NOT NULL, so it is nullable on paper. + * + * Note that most FX queries do NOT select `rate_date` raw: they project + * `to_char(rate_date, 'YYYY-MM-DD') AS rate_date` (or `rate_date::text`) + * precisely to avoid the local-midnight `Date`. Those projections are typed at + * the call site with `Pick<>` plus an explicit `rate_date: string` override + * rather than by loosening this typedef. + * + * @typedef {object} ExchangeRateRow + * @property {number} id SERIAL + * @property {string} currency_code VARCHAR(3) + * @property {string} rate_to_eur NUMERIC(20,10) — pg emits NUMERIC as a string. + * @property {Date} rate_date DATE — a local-midnight `Date`, NOT a 'YYYY-MM-DD' string. + * @property {boolean|null} is_latest BOOLEAN DEFAULT false (no NOT NULL constraint). + * @property {Date} fetched_at TIMESTAMPTZ NOT NULL DEFAULT NOW(). + * @property {Date|null} updated_at TIMESTAMPTZ + */ + +/** + * One entry of the in-memory historical-FX index built by + * `buildHistoricalRateIndex`: a 'YYYY-MM-DD' day and the already-numeric + * `rate_to_eur` for it. + * + * @typedef {object} HistoricalRatePoint + * @property {string} date 'YYYY-MM-DD' + * @property {number} rate + */ + +/** + * The historical-FX index: currency code → date-ascending rate points. + * + * @typedef {Map} HistoricalRateIndex + */ + +/** + * A `{ EUR: 1, USD: x, … }` map of "1 unit of X is this many EUR" multipliers, + * as returned by the ECB / open.er-api fetchers, `loadFromDatabase`, and the + * currency service's cache hierarchy. + * + * @typedef {Record} RateTable + */ + +// --------------------------------------------------------------------------- +// Belgian inflation rates +// --------------------------------------------------------------------------- + +/** + * A row of `belgian_inflation_rates` (migration 0001). + * + * @typedef {object} BelgianInflationRateRow + * @property {number} id SERIAL + * @property {Date} month_date DATE (first-of-month) — a local-midnight `Date`, NOT a 'YYYY-MM-DD' string. + * @property {string} monthly_rate NUMERIC(10,8) — pg emits NUMERIC as a string. + * @property {string} source VARCHAR(50) NOT NULL DEFAULT 'statbel'. + * @property {Date} fetched_at TIMESTAMPTZ NOT NULL DEFAULT NOW(). + * @property {Date|null} updated_at TIMESTAMPTZ + */ + +/** + * The service's normalized shape for one month's inflation rate — used both + * for DB-loaded rows (after `monthKeyFromDatabaseValue`/`Number()`) and rates + * parsed from an external payload (Statbel/Eurostat), which are the same + * shape before being persisted. + * + * @typedef {object} BelgianInflationRate + * @property {string} month 'YYYY-MM'. + * @property {number} monthly_rate Already-numeric fraction (e.g. 0.0025), rounded to `RATE_DECIMALS`. + */ + export {}; diff --git a/apps/node-backend/src/types/thirdPartyModules.d.ts b/apps/node-backend/src/types/thirdPartyModules.d.ts new file mode 100644 index 00000000..efecfa17 --- /dev/null +++ b/apps/node-backend/src/types/thirdPartyModules.d.ts @@ -0,0 +1,28 @@ +// Ambient ("this module exists, no type info") declarations for third-party +// packages that ship no type declarations AND have no `@types/*` package +// installed in this workspace, so a plain `import x from 'pkg'` fails +// noImplicitAny with TS7016 ("Could not find a declaration file for module"). +// +// `pg` and `express` hit this same wall but are only ever referenced in TYPE +// position (`import('pg').PoolClient`, `import('express').Application`), so +// the fix there is a structural JSDoc typedef describing just the slice each +// file uses (see `QueryRunner` in rows.js, `ExpressLayer`/`ExpressApp` in +// services/routeManifest.js) — no ambient declaration needed. +// +// `multer` is different: it is imported as a VALUE (`multer(...)`, +// `multer.memoryStorage()`), and TS7016 fires unconditionally on the import +// statement itself regardless of how the binding is used afterward, so no +// amount of casting at the use site avoids it. The remediation TS's own +// error message suggests is exactly this file: an 'ambient' module +// declaration. `noImplicitAny` still applies everywhere ELSE — this only +// silences the "no declaration file" complaint for the packages listed below, +// each of which then behaves as an untyped (`any`) import at its use sites, +// same as it always has. +// +// tsconfig.check(.strict).json's `include` is `src/**/*.js`, which does not +// pick up `.d.ts` files on its own; a consuming file pulls this in with a +// `/// ` comment, after +// which the ambient declarations below are visible to the whole program +// (ambient declarations are global once included, not per-importer). + +declare module 'multer'; diff --git a/apps/node-backend/tests/accountBalanceSql.test.js b/apps/node-backend/tests/accountBalanceSql.test.js index 34f7ba07..7a4a2c9e 100644 --- a/apps/node-backend/tests/accountBalanceSql.test.js +++ b/apps/node-backend/tests/accountBalanceSql.test.js @@ -12,7 +12,11 @@ */ import { describe, expect, it } from 'vitest'; -import { COMPUTED_BALANCE_LATERAL } from '../src/repositories/accountBalanceSql.js'; +import { + COMPUTED_BALANCE_LATERAL, + computedBalanceByCurrencyLateral, + computedBalanceSeriesCtes, +} from '../src/repositories/accountBalanceSql.js'; describe('COMPUTED_BALANCE_LATERAL', () => { it('exposes balance, anchor_date and post_anchor_count (WP-A1 provenance)', () => { @@ -56,3 +60,101 @@ describe('COMPUTED_BALANCE_LATERAL', () => { expect(COMPUTED_BALANCE_LATERAL.match(/is_active = true/g)?.length).toBe(2); }); }); + +describe('computedBalanceByCurrencyLateral', () => { + const sql = computedBalanceByCurrencyLateral({ account: 'a.id' }); + + it('partitions every part of the computation by currency', () => { + // The defect it fixes is a cross-currency SUM: the currency list, the + // anchor probe and the delta sum must ALL be constrained to one currency, + // or a partition silently absorbs another currency's rows again. + expect(sql).toContain(`GROUP BY COALESCE(t.currency, 'EUR')`); + expect(sql).toContain(`AND COALESCE(t.currency, 'EUR') = ccy.currency`); + expect(sql).toContain(`AND COALESCE(t2.currency, 'EUR') = ccy.currency`); + }); + + it('keeps the anchor+delta structure, per partition', () => { + expect(sql).toContain('t.balance IS NOT NULL'); + expect(sql).toContain('ORDER BY t.date DESC, t.id DESC'); + expect(sql).toContain('(anch.date IS NULL OR (t2.date, t2.id) > (anch.date, anch.id))'); + expect(sql.match(/is_active = true/g)?.length).toBe(3); + }); + + it('is set-returning and aliased, exposing currency and balance', () => { + expect(sql.trim().startsWith('JOIN LATERAL')).toBe(true); + expect(sql).toContain(') bal ON true'); + expect(sql).toContain('AS balance'); + // It is a CURRENT balance: it carries no per-partition FX date, because + // callers must convert it at today's rate (the day the chart ends on), not + // at the partition's last activity. + expect(sql).not.toContain('last_activity'); + expect(computedBalanceByCurrencyLateral({ account: 'a.id', alias: 'x' })).toContain(') x ON true'); + }); +}); + +describe('computedBalanceSeriesCtes', () => { + it('ends in a balance_series CTE over the caller-supplied grid CTEs', () => { + const sql = computedBalanceSeriesCtes(); + expect(sql).toContain('balance_series AS ('); + expect(sql).toContain('JOIN account_list bs_al ON bs_al.account_id = t.account_id'); + // Spliced into an existing WITH chain: no leading WITH, no trailing comma. + expect(sql.trim().startsWith('WITH')).toBe(false); + expect(sql.trim().endsWith(')')).toBe(true); + }); + + it('fills quiet days by expanding spans, never by joining a dense grid', () => { + // A LEFT JOIN of the grid against the (statistics-less) day-end CTE is what + // the planner turned into an O(days²) merge join with the day demoted to a + // filter. Expanding each span with generate_series has no join to mis-plan. + const sql = computedBalanceSeriesCtes(); + expect(sql).toContain("CROSS JOIN LATERAL generate_series(s.from_day, s.thru_day, interval '1 day')"); + expect(sql).toContain('LEAD(day) OVER (PARTITION BY account_id ORDER BY day) - 1'); + expect(sql).not.toContain('LEFT JOIN bs_day_end'); + expect(sql).not.toContain('JOIN days d'); + }); + + it('reads balance as a stamp, never as a population gate — that gate WAS the bug', () => { + // Manual-only accounts must reach the series. `balance IS NOT NULL` may + // appear ONLY in the pre-window anchor probe (picking a stamp); the emitted + // rows are filtered on day boundaries alone. + const sql = computedBalanceSeriesCtes(); + // Exactly two readings of `balance`, both of them "is this row a stamp?": + // the pre-window anchor probe, and the carry-forward CASE. + expect(sql.match(/balance IS NOT NULL/g)?.length).toBe(2); + expect(sql).toContain('AND t.balance IS NOT NULL\n AND t.date < (SELECT first_day FROM bs_span)'); + expect(sql).toContain('MAX(CASE WHEN balance IS NOT NULL'); + expect(sql).toContain('balance - cum]'); + // The only filter on the emitted rows is a day boundary. + expect(sql).toContain('WHERE next_day IS DISTINCT FROM day'); + }); + + it('bounds the walk at the grid and folds earlier activity into one opening row', () => { + const sql = computedBalanceSeriesCtes(); + expect(sql).toContain('AND t.date >= (SELECT first_day FROM bs_span)'); + expect(sql).toContain('AND t.date <= (SELECT last_day FROM bs_span)'); + // The opening row is dated the day before the grid and carries the + // anchor+delta balance as of then, in the `balance` column (i.e. as a + // stamp), so the window pass never has to read pre-window rows. + expect(sql).toContain('(SELECT first_day FROM bs_span) - 1 AS date'); + expect(sql).toContain('COALESCE(anch.balance, 0) + dlt.amount AS balance'); + expect(sql).toContain('GREATEST(date, (SELECT first_day FROM bs_span)) AS day'); + }); + + it('partitions by currency only when asked, and only over active rows', () => { + const cross = computedBalanceSeriesCtes(); + const perCcy = computedBalanceSeriesCtes({ byCurrency: true }); + expect(cross).toContain(`''::text AS currency`); + expect(perCcy).toContain(`COALESCE(t.currency, 'EUR') AS currency`); + // The partition follows the mode; the cross-currency form carries no + // constant column in its sort keys. + expect(perCcy).toContain('WINDOW bs_w AS (PARTITION BY account_id, currency ORDER BY date, id)'); + expect(cross).toContain('WINDOW bs_w AS (PARTITION BY account_id ORDER BY date, id)'); + // …and only the per-currency form confines a partition to one currency. + expect(perCcy).toContain(`AND COALESCE(t.currency, 'EUR') = p.currency`); + expect(cross).not.toContain('= p.currency'); + for (const sql of [cross, perCcy]) { + expect(sql).toContain('WHERE t.is_active = true'); + expect(sql.match(/t\.is_active = true/g)?.length).toBeGreaterThanOrEqual(4); + } + }); +}); diff --git a/apps/node-backend/tests/aliasCategoryResolution.db.test.js b/apps/node-backend/tests/aliasCategoryResolution.db.test.js new file mode 100644 index 00000000..d4c56fc5 --- /dev/null +++ b/apps/node-backend/tests/aliasCategoryResolution.db.test.js @@ -0,0 +1,714 @@ +/** + * Real-Postgres tests for the canonical 3-level effective-category resolution + * on the aggregation surfaces that used to resolve only TWO levels: + * + * • monthly summary — live path (infoRepositoryMonthly) and the + * `mv_monthly_summary` fast path (materializedViewService), + * • sankey flow (services/calculations/aggregation/sankey), + * • recurring detection (services/recurringDetectionService). + * + * …and, added with the follow-up pass, the remaining name-rendering surfaces: + * + * • CSV / NDJSON export (services/transactionExport) and the owed-splits + * export (repositories/splitRepository) — both carried the pre-fix + * `pc`-before-`rc` CASE, so they labelled a row with the PRIMARY's category + * name where the transactions list showed the ALIAS's own, + * • planned transactions (repositories/plannedTransactionRepository, which + * resolved 2 levels, and repositories/infoRepositoryPlanned, which resolved + * 1) — planned_transactions carries its own recipient_id + category_id, so + * the identical 3-level resolution applies there. + * + * "3-level" = own `t.category_id` → recipient's `default_category_id` → + * PRIMARY recipient's `default_category_id`. A transaction recorded under an + * ALIAS recipient whose PRIMARY carries the default category is categorised in + * the transactions list; before this fix these surfaces reported it as + * uncategorised, so the dashboard/sankey/recurring answers disagreed with the + * transactions list for the same row. + * + * This is a DB suite on purpose: the mock suites for these modules feed + * pre-shaped rows into the JS half and cannot observe how Postgres resolves the + * join, which is exactly where the bug lived. + * + * The materialized views are created (and populated) inside the MV tests and + * dropped again before the suite releases the DB lock — every other DB suite + * assumes a freshly-migrated database with no MVs present. + */ + +import { afterAll, afterEach, beforeAll, describe, expect, it } from 'vitest'; +import { + acquireDbSuiteLock, + closeTestPool, + getTestPool, + hasTestDatabase, + releaseDbSuiteLock, +} from './setup/db.js'; +import { getMonthlyFinancialSummary } from '../src/repositories/infoRepositoryMonthly.js'; +import { computeSankeyFlow } from '../src/services/calculations/aggregation/sankey.js'; +import { + detectRecurringPatterns, + __clearRecurringCacheForTests, +} from '../src/services/recurringDetectionService.js'; +import { createMaterializedViews } from '../src/services/materializedViewService.js'; +import transactionRepository from '../src/repositories/transactionRepository.js'; +import { clearMvCache } from '../src/repositories/infoRepositoryHelpers.js'; +import { clearMemoryCache } from '../src/services/currency/currencyConversionService.js'; +import { closePool } from '../src/database/connection.js'; +import { + buildIdListWhere, + streamCsvExport, + streamNdjsonExport, +} from '../src/services/transactionExport.js'; +import splitRepository from '../src/repositories/splitRepository.js'; +import plannedTransactionRepository from '../src/repositories/plannedTransactionRepository.js'; +import { plannedRepository } from '../src/repositories/infoRepositoryPlanned.js'; + +const MANAGED_VIEWS = ['mv_monthly_summary', 'mv_category_totals', 'mv_cashflow_daily']; + +const cat = {}; +const rec = {}; + +/** + * Categories, plus the alias topology under test: + * Electrabel — PRIMARY, default category Bills + * Electrabel Invoicing — ALIAS of Electrabel, NO default category of its own + * Aldi — plain recipient with default category Food + * Misc Payee — no default category anywhere + */ +async function seedBase() { + const pool = getTestPool(); + for (const [key, [general, detail]] of Object.entries({ + Food: ['Food', 'Groceries'], + Bills: ['Bills', 'Utilities'], + })) { + const { rows } = await pool.query( + 'INSERT INTO categories (general, detail) VALUES ($1, $2) RETURNING id', + [general, detail], + ); + cat[key] = rows[0].id; + } + const addRecipient = async (name, { defaultCategoryId = null, primaryId = null } = {}) => { + const { rows } = await pool.query( + `INSERT INTO recipients (name, normalized_name, default_category_id, primary_recipient_id) + VALUES ($1, $2, $3, $4) RETURNING id`, + [name, name.toLowerCase(), defaultCategoryId, primaryId], + ); + return rows[0].id; + }; + rec.electrabel = await addRecipient('Electrabel', { defaultCategoryId: cat.Bills }); + rec.electrabelAlias = await addRecipient('Electrabel Invoicing', { primaryId: rec.electrabel }); + rec.aldi = await addRecipient('Aldi', { defaultCategoryId: cat.Food }); + rec.misc = await addRecipient('Misc Payee'); +} + +/** Pre-create the accounts row (the sync trigger's own INSERT is broken at head). */ +async function ensureAccount(name) { + await getTestPool().query( + `INSERT INTO accounts (name, display_name) VALUES ($1, $1) + ON CONFLICT (lower(btrim(name))) DO NOTHING`, + [name], + ); +} + +/** + * `date` may be a 'YYYY-MM-DD' string or a raw SQL date expression (passed via + * `dateSql`) — the monthly surfaces are windowed on CURRENT_DATE, so their + * fixtures must be anchored to the database's today, not the test author's. + */ +async function insertTxn({ + date = null, + dateSql = null, + amount, + currency = 'EUR', + recipientId, + categoryId = null, + bank = 'MAIN BANK', + isActive = true, + isTransfer = false, +}) { + if (bank) await ensureAccount(bank); + const dateExpr = dateSql ?? '$1'; + const params = dateSql + ? [amount, currency, recipientId, categoryId, bank, isActive, isTransfer] + : [date, amount, currency, recipientId, categoryId, bank, isActive, isTransfer]; + const n = dateSql ? 0 : 1; + const { rows } = await getTestPool().query( + `INSERT INTO transactions (date, amount, currency, recipient_id, category_id, bank_account, is_active, is_transfer) + VALUES (${dateExpr}, $${n + 1}, $${n + 2}, $${n + 3}, $${n + 4}, $${n + 5}, $${n + 6}, $${n + 7}) RETURNING id`, + params, + ); + return rows[0].id; +} + +/** + * seedBase() plus the topology the parent fix pinned in + * transactionRepository.db.test.js ("alias with its own default under a + * differently-defaulted primary"): + * + * Electrabel — PRIMARY, default category Bills:Utilities + * Electrabel Retail — ALIAS of Electrabel, with its OWN default Zzz:Last + * + * A row booked against the alias resolves to Zzz:Last. The pre-fix `pc`-first + * CASE resolved Bills:Utilities instead — this is the only topology in which + * the two orders disagree, since it is the only one where BOTH rc and pc exist + * and differ. 'Zzz'/'Last' sorts after every other fixture category, so an + * ordered assertion can also tell the two resolutions apart. + */ +async function seedAliasWithOwnDefault() { + await seedBase(); + const pool = getTestPool(); + const { rows: catRows } = await pool.query( + `INSERT INTO categories (general, detail) VALUES ('Zzz', 'Last') RETURNING id`, + ); + cat.Zzz = catRows[0].id; + const { rows: recRows } = await pool.query( + `INSERT INTO recipients (name, normalized_name, default_category_id, primary_recipient_id) + VALUES ('Electrabel Retail', 'electrabel retail', $1, $2) RETURNING id`, + [cat.Zzz, rec.electrabel], + ); + rec.electrabelOwnAlias = recRows[0].id; +} + +/** + * The name of the category the transactions list's EFFECTIVE id names for a + * row — the reference every other surface's `category_name` must equal. Read + * back from `categories` rather than hardcoded, so the assertion is literally + * "the id and the name denote the same category". + */ +async function listResolvedCategoryName(txnId) { + const listed = (await transactionRepository.getAll({})).find((t) => t.id === txnId); + const { rows } = await getTestPool().query( + `SELECT general || ':' || detail AS name FROM categories WHERE id = $1`, + [listed.effective_category_id], + ); + return { effectiveCategoryId: listed.effective_category_id, name: rows[0].name }; +} + +/** Minimal `res` double for the streaming export pipeline; collects the chunks. */ +function collectingRes() { + /** @type {string[]} */ + const chunks = []; + return { + chunks, + setHeader() {}, + write(chunk) { chunks.push(chunk); return true; }, + end() {}, + headersSent: false, + }; +} + +/** + * Insert one planned_transactions row directly (not through the repository, so + * repository behaviour is never asserted against itself). `dateSql` is a raw + * SQL date expression: the planned surfaces are windowed on CURRENT_DATE. + */ +async function insertPlanned({ dateSql, amount, recipientId, categoryId = null, memo = null }) { + const { rows } = await getTestPool().query( + `INSERT INTO planned_transactions (planned_date, amount, currency, recipient_id, category_id, memo) + VALUES (${dateSql}, $1, 'EUR', $2, $3, $4) RETURNING id`, + [amount, recipientId, categoryId, memo], + ); + return rows[0].id; +} + +async function dropManagedViews() { + const pool = getTestPool(); + for (const view of MANAGED_VIEWS) { + await pool.query(`DROP MATERIALIZED VIEW IF EXISTS ${view} CASCADE`); + } + clearMvCache(); +} + +describe.skipIf(!hasTestDatabase())('3-level effective-category resolution (real DB)', () => { + beforeAll(async () => { + expect( + process.env.DATABASE_URL, + 'DATABASE_URL must equal TEST_DATABASE_URL for this suite (see scripts/with-test-db.sh)', + ).toBe(process.env.TEST_DATABASE_URL); + await acquireDbSuiteLock(); + }, 180_000); + + afterEach(async () => { + const pool = getTestPool(); + // MVs first: they are the only artefact of this suite that outlives a row + // wipe, and every other DB suite assumes a migrated DB with no MVs. + await dropManagedViews(); + // transaction_splits/split_payments cascade off transactions; + // planned_transactions does NOT cascade off recipients, so it goes first. + await pool.query('DELETE FROM planned_transactions'); + await pool.query('DELETE FROM transactions'); + await pool.query('DELETE FROM accounts'); + await pool.query('DELETE FROM recipients'); + await pool.query('DELETE FROM categories'); + await pool.query('DELETE FROM exchange_rates'); + await pool.query(`DELETE FROM user_settings WHERE key = 'includeTransfers'`); + for (const bag of [cat, rec]) for (const k of Object.keys(bag)) delete bag[k]; + clearMemoryCache(); + __clearRecurringCacheForTests(); + }); + + afterAll(async () => { + await releaseDbSuiteLock(); + await closeTestPool(); + await closePool(); + }); + + // ─────────────────────────────────────────────────────────────────────────── + // Monthly summary — mv_monthly_summary definition + live/MV agreement + // ─────────────────────────────────────────────────────────────────────────── + describe('monthly summary', () => { + /** + * Seed one alias-categorised row and one plainly-categorised row in the + * current month. Returns the ids so callers can assert on them. + */ + async function seedCurrentMonth() { + await seedBase(); + const aliasTxn = await insertTxn({ + dateSql: `(date_trunc('month', CURRENT_DATE) + interval '5 days')::date`, + amount: '-120.00', + recipientId: rec.electrabelAlias, // categorised ONLY via the primary's default + }); + await insertTxn({ + dateSql: `(date_trunc('month', CURRENT_DATE) + interval '6 days')::date`, + amount: '-40.00', + recipientId: rec.aldi, // categorised via its own recipient default + }); + await insertTxn({ + dateSql: `(date_trunc('month', CURRENT_DATE) + interval '7 days')::date`, + amount: '2000.00', + recipientId: rec.misc, // uncategorised at all three levels + }); + return { aliasTxn }; + } + + // The regression pin for the MV definition: mv_monthly_summary used to + // group the alias row under the UNCATEGORISED bucket (category_id NULL, + // category_id_key −1) while the transactions list showed it as Bills. + it('mv_monthly_summary attributes an alias row to its PRIMARY recipient default category', async () => { + const { aliasTxn } = await seedCurrentMonth(); + + // The transactions list is the reference surface for this row… + const listed = (await transactionRepository.getAll({})).find((t) => t.id === aliasTxn); + expect(listed.effective_category_id).toBe(cat.Bills); + expect(listed.category_name).toBe('Bills:Utilities'); + + await createMaterializedViews(); + + const { rows } = await getTestPool().query( + `SELECT category_id, category_id_key, category_name, total_spending, total_income, transaction_count + FROM mv_monthly_summary + WHERE month_start = date_trunc('month', CURRENT_DATE)::date + ORDER BY category_name`, + ); + + // …and the MV now agrees: the −120 sits in Bills, not UNCATEGORISED. + expect( + rows.map((r) => [r.category_name, Number(r.total_spending), Number(r.total_income)]), + ).toEqual([ + ['Bills:Utilities', -120, 0], + ['Food:Groceries', -40, 0], + ['UNCATEGORISED', 0, 2000], // the genuinely uncategorised income row + ]); + const bills = rows.find((r) => r.category_name === 'Bills:Utilities'); + expect(bills.category_id).toBe(cat.Bills); + expect(bills.category_id_key).toBe(cat.Bills); + // Nothing but the real uncategorised row is left in the −1 bucket. + expect(rows.filter((r) => r.category_id_key === -1)).toHaveLength(1); + }); + + // Cross-path guard: the MV fast path and the live query must return the + // same months/summary for the same corpus, so a future divergence in the + // effective-category resolution (or anything else) surfaces here. + it('MV fast path and live path return identical months and summary', async () => { + await seedCurrentMonth(); + // A second month so the comparison spans more than the current one. + await insertTxn({ + dateSql: `(date_trunc('month', CURRENT_DATE) - interval '1 month' + interval '5 days')::date`, + amount: '-33.00', + recipientId: rec.electrabelAlias, + }); + + // No MVs yet → mvAvailable() is false → live query. + const live = await getMonthlyFinancialSummary(); + expect(live.months).toHaveLength(6); + + await createMaterializedViews(); + clearMvCache(); + const mv = await getMonthlyFinancialSummary(); + + expect(mv.months).toEqual(live.months); + expect(mv.summary).toEqual(live.summary); + // Sanity: the alias row's −120 and −33 really are in there. + const current = mv.months[mv.months.length - 1]; + expect(current.total_spending).toBe(-160); + expect(current.total_income).toBe(2000); + expect(mv.months[mv.months.length - 2].total_spending).toBe(-33); + }); + + // The live path's category exclusion resolves 3 levels too: excluding the + // PRIMARY's default category must drop the alias row's amount. + it('live path excludes an alias row by its PRIMARY recipient default category', async () => { + await seedCurrentMonth(); + + const withBills = await getMonthlyFinancialSummary(); + expect(withBills.months[withBills.months.length - 1].total_spending).toBe(-160); + + const withoutBills = await getMonthlyFinancialSummary([cat.Bills]); + expect(withoutBills.months[withoutBills.months.length - 1].total_spending).toBe(-40); + }); + }); + + // ─────────────────────────────────────────────────────────────────────────── + // Sankey + // ─────────────────────────────────────────────────────────────────────────── + describe('computeSankeyFlow', () => { + /** The finding's fixture: €3000 uncategorised income + an alias-Bills and a Food expense. */ + async function seedSankeyYear() { + await seedBase(); + // Uncategorised at all three levels — the row the exclusion clause used to eat. + await insertTxn({ date: '2024-01-15', amount: '3000.00', recipientId: rec.misc }); + await insertTxn({ date: '2024-02-10', amount: '-120.00', recipientId: rec.electrabelAlias }); + await insertTxn({ date: '2024-02-12', amount: '-40.00', recipientId: rec.aldi }); + } + + /** { income, spendingByLabel } for a year's flow graph. */ + async function flow(opts) { + const env = await computeSankeyFlow({ year: 2024, ...opts }); + return { + income: env.data.nodes.find((n) => n.id === '__income__')?.value ?? 0, + spending: Object.fromEntries( + env.data.nodes.filter((n) => n.id.startsWith('cat:')).map((n) => [n.label, n.value]), + ), + }; + } + + // The category JOIN regression pin: the alias row used to land in + // "Uncategorised" because the join resolved only 2 levels. + it('attributes an alias row to its PRIMARY default category', async () => { + await seedSankeyYear(); + + expect(await flow({})).toEqual({ + income: 3000, + spending: { 'Bills: Utilities': 120, 'Food: Groceries': 40 }, // formerly 'Uncategorised' + }); + }); + + // Real pin for the NULL-sentinel half (was characterization only, because + // pre-fix the alias row vanished for the WRONG reason). `!= ALL($n)` with + // no `-1` sentinel is NULL — not true — for a NULL effective category, so + // excluding ANY category silently erased every uncategorised row: this + // exact fixture rendered "Income 0" with €40 still flowing out of it. + // buildExclusionClauses' `COALESCE(..., -1) NOT IN (...)` keeps them. + it('excludes by the PRIMARY default category while keeping uncategorised rows', async () => { + await seedSankeyYear(); + + expect(await flow({ excludedCategoryIds: [cat.Bills] })).toEqual({ + income: 3000, // pre-fix: 0 — the uncategorised income row was dropped + spending: { 'Food: Groceries': 40 }, + }); + }); + + // Recipient exclusion is alias-aware: the bare `t.recipient_id != ALL(...)` + // left rows booked on an ALIAS of the excluded PRIMARY in the graph, while + // the category exclusion beside it already resolved the alias. + it('excludes a PRIMARY recipient together with its aliases, keeping recipient-less rows', async () => { + await seedSankeyYear(); + + expect(await flow({ excludedRecipientIds: [rec.electrabel] })).toEqual({ + income: 3000, + spending: { 'Food: Groceries': 40 }, // pre-fix: Bills 120 survived + }); + }); + + // ADR-083: a savings transfer's two legs inflated BOTH sides of the graph + // (fake income in, fake spending out) — sankey had no is_transfer filter + // at all. It now follows the same runtime setting as its siblings. + it('excludes internal transfers by default and includes them when the setting is on', async () => { + await seedBase(); + await insertTxn({ date: '2024-01-15', amount: '3000.00', recipientId: rec.misc }); + await insertTxn({ date: '2024-02-12', amount: '-40.00', recipientId: rec.aldi }); + await insertTxn({ date: '2024-03-01', amount: '-900.00', recipientId: rec.misc, isTransfer: true }); + await insertTxn({ date: '2024-03-01', amount: '900.00', recipientId: rec.misc, isTransfer: true }); + + expect(await flow({})).toEqual({ + income: 3000, // pre-fix: 3900 + spending: { 'Food: Groceries': 40 }, // pre-fix: + Uncategorised 900 + }); + + await getTestPool().query( + `INSERT INTO user_settings (key, value) VALUES ('includeTransfers', 'true'::jsonb)`, + ); + expect(await flow({})).toEqual({ + income: 3900, + spending: { 'Food: Groceries': 40, Uncategorised: 900 }, + }); + }); + }); + + // ─────────────────────────────────────────────────────────────────────────── + // Recurring detection + // ─────────────────────────────────────────────────────────────────────────── + describe('detectRecurringPatterns', () => { + it('labels an alias-recipient pattern with the PRIMARY default category', async () => { + await seedBase(); + // Three monthly occurrences under the ALIAS, none carrying its own category. + for (const daysAgo of [90, 60, 30]) { + await insertTxn({ + dateSql: `(CURRENT_DATE - interval '${daysAgo} days')::date`, + amount: '-120.00', + recipientId: rec.electrabelAlias, + }); + } + __clearRecurringCacheForTests(); + + const { patterns } = await detectRecurringPatterns(); + const alias = patterns.find((p) => p.recipientId === rec.electrabelAlias); + expect(alias).toBeDefined(); + expect(alias.detectedPattern).toBe('monthly'); + expect(alias.occurrences).toBe(3); + // Formerly null: the 2-level join found no category for the alias. + expect(alias.categoryName).toBe('Bills:Utilities'); + }); + + it('categoryId denotes the same category categoryName names, even for a recipient-default-resolved alias', async () => { + await seedBase(); + // Three monthly occurrences under the ALIAS, none carrying its own + // category.id — the category comes only from the PRIMARY's default. + for (const daysAgo of [90, 60, 30]) { + await insertTxn({ + dateSql: `(CURRENT_DATE - interval '${daysAgo} days')::date`, + amount: '-120.00', + recipientId: rec.electrabelAlias, + }); + } + __clearRecurringCacheForTests(); + + const { patterns } = await detectRecurringPatterns(); + const alias = patterns.find((p) => p.recipientId === rec.electrabelAlias); + expect(alias).toBeDefined(); + // Pre-fix: categoryId was the raw (null) t.category_id while categoryName + // was already resolved through the 3-level COALESCE — so a + // recipient-default-categorised pattern reported categoryId: null beside + // a non-null categoryName, and any consumer keying on the id (e.g. the + // "create planned payment" POST) saw it as uncategorised. + expect(alias.categoryId).not.toBeNull(); + const { rows } = await getTestPool().query( + `SELECT general || ':' || detail AS name FROM categories WHERE id = $1`, + [alias.categoryId], + ); + expect(rows[0]?.name).toBe(alias.categoryName); + expect(alias.categoryId).toBe(cat.Bills); + }); + }); + + // ─────────────────────────────────────────────────────────────────────────── + // CSV / NDJSON export — services/transactionExport + // ─────────────────────────────────────────────────────────────────────────── + describe('transaction export', () => { + /** Category column index in the (balance-less) CSV header. */ + const CSV_CATEGORY_COL = 7; + + it('labels an alias row with the category the transactions list resolves', async () => { + await seedAliasWithOwnDefault(); + const aliasTxn = await insertTxn({ + date: '2024-03-05', + amount: '-11.11', + recipientId: rec.electrabelOwnAlias, + }); + const { effectiveCategoryId, name } = await listResolvedCategoryName(aliasTxn); + expect(effectiveCategoryId).toBe(cat.Zzz); + expect(name).toBe('Zzz:Last'); + + const csvRes = collectingRes(); + await streamCsvExport(csvRes, buildIdListWhere([aliasTxn])); + // [0] is the header line, [1] the single data row. + expect(csvRes.chunks[0].split(',')[CSV_CATEGORY_COL]).toBe('Category'); + // Pre-fix: 'Bills:Utilities' — the PRIMARY's category, not the alias's. + expect(csvRes.chunks[1].trim().split(',')[CSV_CATEGORY_COL]).toBe(name); + + const jsonRes = collectingRes(); + await streamNdjsonExport(jsonRes, buildIdListWhere([aliasTxn])); + expect(JSON.parse(jsonRes.chunks[0]).category).toBe(name); + }); + + it("an own category_id still wins over both recipient defaults", async () => { + await seedAliasWithOwnDefault(); + const ownTxn = await insertTxn({ + date: '2024-03-06', + amount: '-22.22', + recipientId: rec.electrabelOwnAlias, + categoryId: cat.Food, + }); + const { effectiveCategoryId, name } = await listResolvedCategoryName(ownTxn); + expect(effectiveCategoryId).toBe(cat.Food); + + const res = collectingRes(); + await streamCsvExport(res, buildIdListWhere([ownTxn])); + expect(res.chunks[1].trim().split(',')[CSV_CATEGORY_COL]).toBe(name); + }); + + it('still reaches the PRIMARY default when the alias has none of its own', async () => { + await seedBase(); + const aliasTxn = await insertTxn({ + date: '2024-03-07', + amount: '-33.33', + recipientId: rec.electrabelAlias, // no default of its own + }); + const { effectiveCategoryId, name } = await listResolvedCategoryName(aliasTxn); + expect(effectiveCategoryId).toBe(cat.Bills); + + const res = collectingRes(); + await streamCsvExport(res, buildIdListWhere([aliasTxn])); + expect(res.chunks[1].trim().split(',')[CSV_CATEGORY_COL]).toBe(name); + }); + }); + + // ─────────────────────────────────────────────────────────────────────────── + // Owed-splits export — repositories/splitRepository + // ─────────────────────────────────────────────────────────────────────────── + describe('getOwedExportRowsByRecipient', () => { + it('labels an alias row with the category the transactions list resolves', async () => { + await seedAliasWithOwnDefault(); + const aliasTxn = await insertTxn({ + date: '2024-03-05', + amount: '-11.11', + recipientId: rec.electrabelOwnAlias, + }); + // rec.misc owes half of the alias-recipient transaction. + await getTestPool().query( + `INSERT INTO transaction_splits (transaction_id, recipient_id, amount, is_settled) + VALUES ($1, $2, '5.00', false)`, + [aliasTxn, rec.misc], + ); + const { effectiveCategoryId, name } = await listResolvedCategoryName(aliasTxn); + expect(effectiveCategoryId).toBe(cat.Zzz); + + const rows = await splitRepository.getOwedExportRowsByRecipient(rec.misc); + expect(rows).toHaveLength(1); + expect(rows[0].amount).toBe(5); + // Pre-fix: 'Bills:Utilities' — the PRIMARY's category, not the alias's. + expect(rows[0].category_name).toBe(name); + }); + + it('still reaches the PRIMARY default when the alias has none of its own', async () => { + await seedBase(); + const aliasTxn = await insertTxn({ + date: '2024-03-05', + amount: '-11.11', + recipientId: rec.electrabelAlias, + }); + await getTestPool().query( + `INSERT INTO transaction_splits (transaction_id, recipient_id, amount, is_settled) + VALUES ($1, $2, '4.00', false)`, + [aliasTxn, rec.misc], + ); + const { name } = await listResolvedCategoryName(aliasTxn); + + const rows = await splitRepository.getOwedExportRowsByRecipient(rec.misc); + expect(rows[0].category_name).toBe(name); + }); + }); + + // ─────────────────────────────────────────────────────────────────────────── + // Planned transactions — plannedTransactionRepository + infoRepositoryPlanned + // ─────────────────────────────────────────────────────────────────────────── + describe('planned transactions', () => { + // Anchored to the DB's own calendar so it lands inside every window under + // test: next month (getPlannedExpensesNextMonth), ≤90 days out (getDueSoon) + // and ≤3 months out (getForForecast). + const NEXT_MONTH_SQL = `(date_trunc('month', CURRENT_DATE) + interval '1 month' + interval '4 days')::date`; + + async function seedPlanned() { + await seedAliasWithOwnDefault(); + const aliasPlanned = await insertPlanned({ + dateSql: NEXT_MONTH_SQL, + amount: '-77.00', + recipientId: rec.electrabelOwnAlias, // resolves via the ALIAS's own default + memo: 'alias planned', + }); + const inheritedPlanned = await insertPlanned({ + dateSql: NEXT_MONTH_SQL, + amount: '-55.00', + recipientId: rec.electrabelAlias, // resolves via the PRIMARY's default + memo: 'inherited planned', + }); + const ownPlanned = await insertPlanned({ + dateSql: NEXT_MONTH_SQL, + amount: '-11.00', + recipientId: rec.electrabelOwnAlias, + categoryId: cat.Food, // own category_id wins over both defaults + memo: 'own planned', + }); + return { aliasPlanned, inheritedPlanned, ownPlanned }; + } + + it('getAll / getById / getDueSoon / getForForecast resolve all three levels', async () => { + const { aliasPlanned, inheritedPlanned, ownPlanned } = await seedPlanned(); + // Pre-fix this repository resolved 2 levels (c → rc, no `pc` branch and + // no `pr` join), so `inheritedPlanned` came back category_name: null. + const expected = { + [aliasPlanned]: 'Zzz:Last', + [inheritedPlanned]: 'Bills:Utilities', + [ownPlanned]: 'Food:Groceries', + }; + + const { items } = await plannedTransactionRepository.getAll({}); + expect(Object.fromEntries(items.map((r) => [r.id, r.category_name]))).toEqual(expected); + + for (const [id, name] of Object.entries(expected)) { + const row = await plannedTransactionRepository.getById(Number(id)); + expect(row.category_name).toBe(name); + // Where the row carries its OWN category_id, the id and the displayed + // name must denote the same category; where it does not, the name is + // an inherited display value and category_id is legitimately null. + if (row.category_id != null) expect(row.category_id).toBe(cat.Food); + else expect(name).not.toBe('Food:Groceries'); + } + + const due = await plannedTransactionRepository.getDueSoon(90); + expect(Object.fromEntries(due.map((r) => [r.id, r.category_name]))).toEqual(expected); + + const forecast = await plannedTransactionRepository.getForForecast(3); + expect(Object.fromEntries(forecast.map((r) => [r.id, r.category_name]))).toEqual(expected); + }); + + // The search clause matches the RESOLVED label: 'Utilities' reaches the row + // categorised through the PRIMARY recipient's default (formerly unreachable + // — the clause had no pc term) and only that row (the two siblings display + // 'Zzz:Last' and 'Food:Groceries', even though the same pc is joined for + // them). + it('search matches the label a planned row actually displays', async () => { + const { aliasPlanned, inheritedPlanned, ownPlanned } = await seedPlanned(); + + const utilities = await plannedTransactionRepository.getAll({ search: 'Utilities' }); + expect(utilities.items.map((r) => r.id)).toEqual([inheritedPlanned]); + + const zzz = await plannedTransactionRepository.getAll({ search: 'Zzz:La' }); + expect(zzz.items.map((r) => r.id)).toEqual([aliasPlanned]); + + const food = await plannedTransactionRepository.getAll({ search: 'Groceries' }); + expect(food.items.map((r) => r.id)).toEqual([ownPlanned]); + }); + + it('getPlannedExpensesNextMonth categorises what its sibling repository categorises', async () => { + const { aliasPlanned, inheritedPlanned, ownPlanned } = await seedPlanned(); + + // The sibling repository is the reference surface for these rows… + const { items } = await plannedTransactionRepository.getAll({}); + const reference = Object.fromEntries(items.map((r) => [r.id, r.category_name])); + + const result = await plannedRepository.getPlannedExpensesNextMonth('EUR'); + // Response shape is pinned elsewhere — only the name resolution changes. + expect(Object.keys(result).sort()).toEqual( + ['daily_data', 'month', 'period_end', 'period_start', 'summary', 'year'], + ); + const occurrences = result.daily_data.flatMap((d) => d.transactions); + expect(occurrences).toHaveLength(3); + // Pre-fix: all three were null except `ownPlanned` — this site joined + // neither rc nor pc. + expect(Object.fromEntries(occurrences.map((t) => [t.id, t.category_name]))).toEqual(reference); + expect(reference[aliasPlanned]).toBe('Zzz:Last'); + expect(reference[inheritedPlanned]).toBe('Bills:Utilities'); + expect(reference[ownPlanned]).toBe('Food:Groceries'); + }); + }); +}); diff --git a/apps/node-backend/tests/currencyConversionService.test.js b/apps/node-backend/tests/currencyConversionService.test.js index c8f87627..a666c7b4 100644 --- a/apps/node-backend/tests/currencyConversionService.test.js +++ b/apps/node-backend/tests/currencyConversionService.test.js @@ -261,6 +261,34 @@ describe('Currency Conversion Service', () => { expect(nearestLookups).toHaveLength(1); }); + it('probes a currency with no stored rates ONCE per request, not once per distinct day', async () => { + // A daily series (balance history) hands this function up to 366 distinct + // days. The historical index is empty only when `exchange_rates` holds no + // row for the currency at all — and then the per-date point lookup misses + // on every single day and goes to the network, which offline means + // hundreds of sequential multi-second timeouts. One attempt per currency + // per request is enough; it saves what it fetches for next time. + query.mockResolvedValue({ rows: [] }); + global.fetch = vi.fn().mockResolvedValue({ ok: false, status: 503, text: async () => '' }); + + const rows = Array.from({ length: 60 }, (_, i) => ({ + amount: 1, + currency: 'ZZZ', + day: new Date(Date.UTC(2020, 0, 1 + i)).toISOString().slice(0, 10), + })); + + const converted = await convertRowsToEur(rows, 'EUR', { useHistoricalRatesByDate: true, dateField: 'day' }); + expect(converted).toHaveLength(60); + + const sqlCalls = query.mock.calls.map(([sql]) => String(sql)); + const exactLookups = sqlCalls.filter(sql => sql.includes('WHERE currency_code = $1 AND rate_date = $2::date')); + const nearestLookups = sqlCalls.filter(sql => sql.includes('ORDER BY ABS(rate_date - $2::date) ASC')); + expect(exactLookups).toHaveLength(1); + expect(nearestLookups).toHaveLength(1); + // 60 distinct days must not become 60 provider round-trips either. + expect(global.fetch.mock.calls.length).toBeLessThanOrEqual(2); + }); + /** * Route the mocked DB by SQL shape — the backfill flow now spans the pairs * scan, the repair flag in user_settings, rate lookups/saves and the diff --git a/apps/node-backend/tests/errorHandler.test.js b/apps/node-backend/tests/errorHandler.test.js index dd6436f8..e984c652 100644 --- a/apps/node-backend/tests/errorHandler.test.js +++ b/apps/node-backend/tests/errorHandler.test.js @@ -2,9 +2,13 @@ * errorHandler middleware tests. * * Covers: typed error class defaults, production detail masking, 4xx leakage - * policy (expose message), 5xx leakage policy (mask in production). + * policy (expose message), 5xx leakage policy (mask in production), and the + * forwarded-4xx rule for non-AppError errors that carry their own status + * (body-parser's http-errors) — see THE RULE in src/middleware/errorHandler.js. */ +import express from 'express'; +import supertest from 'supertest'; import { describe, it, expect, vi, beforeEach } from 'vitest'; import { mockLogger } from './helpers/mockLogger.js'; @@ -22,6 +26,8 @@ const { createErrorHandler, } = await import('../src/middleware/errorHandler.js'); +const { createRouteApp } = await import('./helpers/routeApp.js'); + function mockRes() { const res = {}; res.status = vi.fn().mockReturnValue(res); @@ -121,3 +127,171 @@ describe('createErrorHandler', () => { }); }); }); + +/** + * Non-AppError errors carrying their own status. Only 400-499 is forwarded, and + * only an allowlisted body-parser `type` gets its message echoed; everything + * else keeps the sanitized 500 path. + */ +describe('forwarded 4xx from non-AppError errors', () => { + /** Build a plain Error with the http-errors shape body-parser produces. */ + function httpish(message, { status, statusCode, type } = {}) { + const e = /** @type {any} */ (new Error(message)); + if (status !== undefined) e.status = status; + if (statusCode !== undefined) e.statusCode = statusCode; + if (type !== undefined) e.type = type; + return e; + } + + const run = (err, production = false) => { + const res = mockRes(); + createErrorHandler(() => production)(err, req, res, () => {}); + return { status: res.status.mock.calls[0][0], body: res.json.mock.calls[0][0] }; + }; + + it('forwards a trusted body-parser 400 with its own message, even in production', () => { + const err = httpish('Unexpected end of JSON input', { status: 400, type: 'entity.parse.failed' }); + for (const production of [false, true]) { + const { status, body } = run(err, production); + expect(status).toBe(400); + expect(body.error).toEqual({ code: 'VALIDATION_ERROR', message: 'Unexpected end of JSON input' }); + } + }); + + it('forwards a trusted body-parser 413 with its own message, even in production', () => { + const err = httpish('request entity too large', { status: 413, type: 'entity.too.large' }); + for (const production of [false, true]) { + const { status, body } = run(err, production); + expect(status).toBe(413); + expect(body.error).toEqual({ code: 'VALIDATION_ERROR', message: 'request entity too large' }); + } + }); + + it('forwards an untrusted 4xx status but replaces its message with the reason phrase', () => { + // A fabricated non-AppError carrying status 403 and no recognised `type`: + // the status is truthful enough to forward, the wording is not vetted. + const { status, body } = run(httpish('token abc123 rejected by ldap://internal', { status: 403 })); + expect(status).toBe(403); + expect(body.error).toEqual({ code: 'FORBIDDEN', message: 'Forbidden' }); + expect(JSON.stringify(body)).not.toContain('ldap'); + }); + + it('honours statusCode as well as status (loanSchedule.js:70 style throws)', () => { + const { status, body } = run(httpish('Invalid loan configuration: term missing', { statusCode: 400 })); + expect(status).toBe(400); + expect(body.error).toEqual({ code: 'VALIDATION_ERROR', message: 'Bad Request' }); + }); + + it('maps forwarded statuses onto the existing ADR-026 code vocabulary', () => { + const cases = [ + [401, 'UNAUTHORIZED'], [403, 'FORBIDDEN'], [404, 'NOT_FOUND'], + [409, 'CONFLICT'], [429, 'RATE_LIMITED'], + [400, 'VALIDATION_ERROR'], [413, 'VALIDATION_ERROR'], [415, 'VALIDATION_ERROR'], + [451, 'VALIDATION_ERROR'], + ]; + for (const [code, expected] of cases) { + expect(run(httpish('x', { status: code })).body.error.code).toBe(expected); + } + // A 4xx with no reason phrase in the table still gets a generic message. + expect(run(httpish('x', { status: 451 })).body.error.message).toBe('Request rejected'); + }); + + it('ignores 5xx and nonsense statuses — those keep the sanitized 500 path', () => { + for (const bad of [500, 502, 399, 700, 0, -1, NaN, 400.5]) { + const { status, body } = run(httpish('DB password wrong', { status: bad }), true); + expect(status).toBe(500); + expect(body.error).toEqual({ + code: 'INTERNAL_SERVER_ERROR', + message: 'An internal server error occurred. Please try again later.', + }); + } + // Non-numeric statuses (a string, an object) must not be coerced either. + for (const bad of ['400', { valueOf: () => 400 }]) { + expect(run(httpish('DB password wrong', { status: bad }), true).status).toBe(500); + } + }); + + it('a non-AppError with no status is unchanged: 500, sanitized in production only', () => { + expect(run(new Error('DB password wrong'), false).body.error).toEqual({ + code: 'INTERNAL_SERVER_ERROR', message: 'DB password wrong', + }); + expect(run(new Error('DB password wrong'), true).body.error).toEqual({ + code: 'INTERNAL_SERVER_ERROR', + message: 'An internal server error occurred. Please try again later.', + }); + }); + + it('a trusted type without a 4xx status is NOT forwarded', () => { + // body-parser's `stream.encoding.set` is a 500; the type allowlist must not + // be able to pull a 5xx into the exposed-message branch. + const { status, body } = run(httpish('stream encoding should not be set', { status: 500, type: 'entity.too.large' }), true); + expect(status).toBe(500); + expect(body.error.message).toBe('An internal server error occurred. Please try again later.'); + }); +}); + +/** + * Same rule end-to-end over real Express + real body-parser (the harness that + * main.js's data plane is modelled on), so the behaviour is proven against the + * actual http-errors objects rather than hand-built look-alikes. + */ +describe('forwarded 4xx over real Express (supertest)', () => { + /** Router that throws whatever the test asks for, so non-parser paths are covered too. */ + function fixtureRouter() { + const router = express.Router(); + router.post('/echo', (req, res) => res.ok({ got: req.body })); + router.get('/boom', () => { throw new Error('DB password wrong'); }); + router.get('/fake403', () => { + const e = /** @type {any} */ (new Error('token abc123 rejected by ldap://internal')); + e.status = 403; + throw e; + }); + return router; + } + + const agent = (production) => supertest(createRouteApp(fixtureRouter(), { + mountPath: '/api/fixture', + isProduction: () => production, + })); + + const PROD_5XX = 'An internal server error occurred. Please try again later.'; + + it.each([['dev', false], ['production', true]])('malformed JSON → 400 with a useful message (%s)', async (_label, production) => { + const res = await agent(production) + .post('/api/fixture/echo') + .set('Content-Type', 'application/json') + .send('{"amount": '); + + expect(res.status).toBe(400); + expect(res.body.error.code).toBe('VALIDATION_ERROR'); + expect(res.body.error.message).toMatch(/JSON/i); + expect(res.body.error.message).not.toBe(PROD_5XX); + }); + + it.each([['dev', false], ['production', true]])('over-limit body → 413 with its reason visible (%s)', async (_label, production) => { + const res = await agent(production) + .post('/api/fixture/echo') + .send({ memo: 'x'.repeat(1024 * 1024 + 100) }); + + expect(res.status).toBe(413); + expect(res.body.error.code).toBe('VALIDATION_ERROR'); + expect(res.body.error.message).toBe('request entity too large'); + }); + + it('a route-thrown non-AppError with status 403 → 403 with the generic reason phrase', async () => { + const res = await agent(true).get('/api/fixture/fake403'); + expect(res.status).toBe(403); + expect(res.body.error).toEqual({ code: 'FORBIDDEN', message: 'Forbidden' }); + expect(JSON.stringify(res.body)).not.toContain('ldap'); + }); + + it('a route-thrown non-AppError with no status → 500, sanitized in production', async () => { + const dev = await agent(false).get('/api/fixture/boom'); + expect(dev.status).toBe(500); + expect(dev.body.error).toMatchObject({ code: 'INTERNAL_SERVER_ERROR', message: 'DB password wrong' }); + + const prod = await agent(true).get('/api/fixture/boom'); + expect(prod.status).toBe(500); + expect(prod.body.error).toMatchObject({ code: 'INTERNAL_SERVER_ERROR', message: PROD_5XX }); + }); +}); diff --git a/apps/node-backend/tests/helpers/routeApp.js b/apps/node-backend/tests/helpers/routeApp.js new file mode 100644 index 00000000..2b513f44 --- /dev/null +++ b/apps/node-backend/tests/helpers/routeApp.js @@ -0,0 +1,186 @@ +/** + * Real-Express route-test harness (supertest). + * + * Replaces the retired `routeHarness.js`, which mocked `express` itself: + * `Router()` returned a stub that kept only the LAST handler registered per + * verb+path, so every guard registered before it (`validateIdParam`, + * per-route rate limiters, multer, …) was silently dropped, and handlers were + * invoked with hand-built `{ query: {} }` objects that never went through + * Express's parsing, the ADR-026 envelope middleware, or the centralized + * error handler. + * + * This module mounts the REAL router on a throwaway `express()` app wired the + * way `src/main.js` wires the data plane, and hands back a supertest agent. + * Repositories/services are still mocked per suite — only the HTTP edge is real. + * + * ── Fidelity map (what is reproduced, and from where) ────────────────────── + * requestId main.js:82 `req.id` → envelope `meta.requestId` + * + the `X-Request-Id` response header + * requestMetrics main.js:85 passive `res.on('finish')` recorder + * express.json({limit}) main.js:130 body parsing (1 MB limit, as prod) + * csrfGuard main.js:315 mounted on the whole `/api` plane + * mountRouter(app, path, …) main.js:317+ real router, real middleware chain, + * real `req.baseUrl` / `req.route` + * 404 → NotFoundError main.js:395 unmatched paths funnel through the + * error handler so the envelope is + * uniform + * createErrorHandler(...) main.js:401 typed errors → `{ ok:false, error }` + * + * ── Deliberately NOT reproduced (and why) ───────────────────────────────── + * CORS reflection main.js:92-127 response headers only; needs the + * real settings allowlist. + * security headers main.js:133-144 response headers only. + * gzip response compression main.js:150-223 wraps `res.write`/`res.end`; + * would obscure streamed-body and + * Content-Length assertions. + * Pass it via `before` if a suite + * needs it. + * request logging main.js:226-229 noise. + * globalRateLimiter and the per-mount limiters + * main.js:307, 323-335 + * module-level counters keyed by + * IP, shared by every request in a + * worker — a suite with more tests + * than the limit would 429 itself. + * Route-level limiters declared + * INSIDE a router (e.g. + * routes/transactions.js:413) are + * still exercised, because the real + * router is mounted. Mount an + * app-level limiter explicitly via + * `before` when that is the thing + * under test. + * static SPA / health / /api root + * main.js:244-390 not reachable from a router. + * + * Usage: + * import { routeAgent } from '../helpers/routeApp.js'; + * // ... vi.mock() the repositories/services this router imports ... + * const { default: router } = await import('../../src/routes/transactions.js'); + * const api = routeAgent(router, { mountPath: '/api/transactions' }); + * + * const res = await api.get('/api/transactions/').expect(200); + * expect(res.body).toEqual({ ok: true, data: {...}, meta: { requestId: expect.any(String) } }); + */ +import express from 'express'; +import supertest from 'supertest'; + +import { requestId } from '../../src/middleware/requestId.js'; +import { requestMetrics } from '../../src/middleware/requestMetrics.js'; +import { wrapResponse } from '../../src/middleware/envelope.js'; +import { createCsrfGuard } from '../../src/middleware/csrfGuard.js'; +import { createErrorHandler, NotFoundError } from '../../src/middleware/errorHandler.js'; + +/** + * @typedef {object} RouteAppOptions + * @property {string} [mountPath='/'] Path the router is mounted at. Use the + * production path (`/api/transactions`, `/api/planned-transactions`, …) so + * `req.baseUrl` and the request paths in the test read like real traffic. + * @property {import('express').RequestHandler[]} [before=[]] Extra middleware + * mounted on `mountPath` BEFORE the router — the slot `main.js` uses for + * per-mount rate limiters and the admin auth guard (main.js:323-335). + * @property {import('express').RequestHandler[]} [after=[]] Extra middleware + * mounted after the router but before the 404 handler. + * @property {boolean} [csrf=true] Mount the CSRF guard (main.js:315). + * supertest sends neither `Origin` nor `Sec-Fetch-Site`, so it is treated as + * a non-browser client and passes; set false only to prove the guard's effect. + * @property {string|string[]} [corsOrigins=[]] Allowlist handed to the CSRF + * guard (main.js:35 passes `settings.api.corsOrigins`). + * @property {string} [jsonLimit='1mb'] Body-size limit (main.js:130). + * @property {() => boolean} [isProduction] Predicate handed to the error + * handler (main.js:401). Defaults to false so 5xx messages stay visible, + * matching a dev/test run. + */ + +/** + * Build a throwaway Express app with `router` mounted the way main.js mounts + * the data plane. + * + * @param {import('express').Router} router + * @param {RouteAppOptions} [options] + * @returns {import('express').Express} + */ +export function createRouteApp(router, options = {}) { + const { + mountPath = '/', + before = [], + after = [], + csrf = true, + corsOrigins = [], + jsonLimit = '1mb', + isProduction = () => false, + } = options; + + const app = express(); + + // main.js:82 — must run first so every later middleware and the envelope see req.id. + app.use(requestId); + // main.js:85 + app.use(requestMetrics); + // main.js:130 + app.use(express.json({ limit: jsonLimit })); + // main.js:315 — CSRF backstop across the whole /api data plane. + if (csrf) app.use(createCsrfGuard(() => corsOrigins)); + // main.js:232 — attaches res.ok(data, meta?) before any router runs. + app.use(wrapResponse); + + // main.js:317+ — mountRouter(app, path, ...perMountMiddleware, router) + app.use(mountPath, ...before, router); + for (const mw of after) app.use(mountPath, mw); + + // main.js:395 — unmatched paths funnel through the error handler. + app.use((req, _res, next) => { + next(new NotFoundError(`Not Found: ${req.method} ${req.path}`)); + }); + // main.js:401 + app.use(createErrorHandler(isProduction)); + + return app; +} + +/** + * `createRouteApp` + a supertest agent bound to it. + * + * @param {import('express').Router} router + * @param {RouteAppOptions} [options] + * @returns {import('supertest').SuperTest} + */ +export function routeAgent(router, options = {}) { + return supertest(createRouteApp(router, options)); +} + +/** + * Matcher for the ADR-026 success envelope: `{ ok: true, data, meta }`, where + * `meta.requestId` is injected by `wrapResponse` from `req.id` (envelope.js:31). + * Use with `expect(res.body).toEqual(okEnvelope({...}))`. + * + * @param {any} data + * @param {Record} [extraMeta] + */ +export function okEnvelope(data, extraMeta = {}) { + return { + ok: true, + data, + meta: { requestId: expect.any(String), ...extraMeta }, + }; +} + +/** + * Matcher for the ADR-026 failure envelope emitted by `createErrorHandler` + * (errorHandler.js:240-244). `details` is only present on typed AppErrors that + * carry it, so it is opt-in here. + * + * @param {{ code?: any, message?: any, details?: any }} [error] + */ +export function errEnvelope(error = {}) { + const shape = { + code: error.code ?? expect.any(String), + message: error.message ?? expect.any(String), + }; + if (error.details !== undefined) shape.details = error.details; + return { + ok: false, + error: shape, + meta: { requestId: expect.any(String) }, + }; +} diff --git a/apps/node-backend/tests/helpers/routeHarness.js b/apps/node-backend/tests/helpers/routeHarness.js deleted file mode 100644 index e0511534..00000000 --- a/apps/node-backend/tests/helpers/routeHarness.js +++ /dev/null @@ -1,67 +0,0 @@ -/** - * Shared route-test harness. - * - * Backend route tests mock `express` so that `Router()` returns a stub whose - * verb methods record the (last) registered handler under a `${method}:${path}` - * key. This module centralizes that scaffold plus the `res` stub used to invoke - * the recorded handlers. - * - * Usage: - * import { createMockRouter, createMockResponse } from '../helpers/routeHarness.js'; - * const { router: mockRouter, handlers: routeHandlers } = createMockRouter(); - * vi.mock('express', () => ({ - * default: { Router: () => mockRouter }, - * Router: () => mockRouter, - * })); - * // ... await import('../../src/routes/.js'); - * const res = createMockResponse(); - * await routeHandlers['get:/'](req, res); - */ -import { vi } from 'vitest'; - -/** - * Create an express Router stub. Verb methods (get/post/put/patch/delete) - * record the final argument (the route handler) under `${method}:${path}`. - * `use` records middleware into a `use` array while remaining a spy. - * - * @returns {{ router: object, handlers: Record }} - */ -export function createMockRouter() { - const handlers = {}; - const record = (method) => - vi.fn((path, ...args) => { - handlers[`${method}:${path}`] = args[args.length - 1]; - }); - const router = { - get: record('get'), - post: record('post'), - put: record('put'), - patch: record('patch'), - delete: record('delete'), - use: vi.fn((...args) => { - handlers.use = handlers.use || []; - handlers.use.push(args[args.length - 1]); - }), - }; - return { router, handlers }; -} - -/** - * Create a stub Express response. Always provides `json`/`status`/`send` spies, - * a chainable `status`, and the `ok(data, meta)` envelope helper. Additional - * response members (setHeader, end, write, headersSent, ...) can be supplied via - * `extra`. - * - * @param {Record} [extra] Extra response members to merge in. - * @returns {object} - */ -export function createMockResponse(extra = {}) { - const res = { json: vi.fn(), status: vi.fn(), send: vi.fn(), ...extra }; - res.status.mockReturnValue(res); - res.ok = (data, meta) => { - const body = { ok: true, data }; - if (meta) body.meta = meta; - return res.json(body); - }; - return res; -} diff --git a/apps/node-backend/tests/importPipeline.stage.test.js b/apps/node-backend/tests/importPipeline.stage.test.js index 97b575c1..317fb33f 100644 --- a/apps/node-backend/tests/importPipeline.stage.test.js +++ b/apps/node-backend/tests/importPipeline.stage.test.js @@ -20,10 +20,47 @@ vi.mock('../src/services/importPipeline/adapters/generic.js', () => ({ default: { name: 'generic', parseWithConfig: (...args) => genericParseWithConfig(...args) }, })); -import { stageBatch } from '../src/services/importPipeline/stage.js'; +import { stageBatch, createBatch } from '../src/services/importPipeline/stage.js'; +import { createBatch as createPortfolioBatch } from '../src/services/portfolioImportPipeline/stage.js'; +import { query } from '../src/database/connection.js'; const CONFIG = { dateColumn: 'D', recipientColumn: 'R', amountColumn: 'A' }; +/** + * The single boundary where a batch id enters the application. `import_batches.id` + * is BIGSERIAL and node-postgres emits BIGINT as a STRING, so without this + * normalization POST /api/import/csv answered `batch_id: "12"` while the + * review-commit route (routes/importRoutes.js:570), which reads the id back off + * the URL through `coercedIdSchema`, answered `batch_id: 12`. + */ +describe('createBatch normalizes the BIGSERIAL id to a number', () => { + beforeEach(() => vi.clearAllMocks()); + + it('returns a NUMBER even though pg hands back a string', async () => { + query.mockResolvedValue({ rows: [{ id: '12' }] }); + + const id = await createBatch({ adapterName: 'vision' }); + + expect(id).toBe(12); + expect(typeof id).toBe('number'); + }); + + it('does the same in the portfolio pipeline, so both agree on the wire', async () => { + query.mockResolvedValue({ rows: [{ id: '12' }] }); + + const id = await createPortfolioBatch({ adapterName: 'generic' }); + + expect(id).toBe(12); + expect(typeof id).toBe('number'); + }); + + it('is exact for ids up to Number.MAX_SAFE_INTEGER (the documented ceiling)', async () => { + query.mockResolvedValue({ rows: [{ id: String(Number.MAX_SAFE_INTEGER) }] }); + + expect(await createBatch({ adapterName: 'vision' })).toBe(Number.MAX_SAFE_INTEGER); + }); +}); + describe('stageBatch adapter resolution', () => { beforeEach(() => { vi.clearAllMocks(); diff --git a/apps/node-backend/tests/infoRepoBanks.db.test.js b/apps/node-backend/tests/infoRepoBanks.db.test.js new file mode 100644 index 00000000..21bfee8f --- /dev/null +++ b/apps/node-backend/tests/infoRepoBanks.db.test.js @@ -0,0 +1,549 @@ +/** + * Real-Postgres tests for infoRepositoryBanks (`getBankBalances`). + * + * DB-backed complement to infoRepoBanks.test.js (which stays: it runs without a + * DB). That mock suite choreographs `query` resolutions in order and asserts SQL + * substrings — it can prove the two statements are *fired* in parallel and that + * they *mention* `in_net_worth`, but it feeds the JS half pre-shaped rows and so + * never observes what Postgres actually returns. Everything asserted here comes + * out of the real schema: the shared anchor+delta lateral + * (COMPUTED_BALANCE_LATERAL) resolving against real NUMERIC/DATE columns, the + * `accounts` population gates (type/in_net_worth/transaction_count), the 12-month + * daily history LATERAL, and FX conversion off seeded `exchange_rates` rows. + * + * Determinism: both statements are anchored on `CURRENT_DATE`, so fixtures are + * dated by SQL expressions relative to it (never by a literal calendar date) and + * the expectations are computed from the same anchors the queries use. + * + * The materialized views are not involved here (this repository has no MV fast + * path); the currency memory cache is cleared around every test so no test sees + * another's seeded rates. + */ + +import { afterAll, afterEach, beforeAll, describe, expect, it } from 'vitest'; +import { + acquireDbSuiteLock, + closeTestPool, + getTestPool, + hasTestDatabase, + releaseDbSuiteLock, +} from './setup/db.js'; +import { banksRepository } from '../src/repositories/infoRepositoryBanks.js'; +import { clearMvCache } from '../src/repositories/infoRepositoryHelpers.js'; +import { clearMemoryCache } from '../src/services/currency/currencyConversionService.js'; +import { closePool } from '../src/database/connection.js'; + +const rec = {}; + +/** 'YYYY-MM-DD' for `CURRENT_DATE + ` (empty string = today). */ +async function ymdFromToday(sqlInterval = '') { + const expr = sqlInterval ? `CURRENT_DATE ${sqlInterval}` : 'CURRENT_DATE'; + const { rows } = await getTestPool().query(`SELECT to_char((${expr})::date, 'YYYY-MM-DD') AS d`); + return rows[0].d; +} + +async function seedRecipient() { + const { rows } = await getTestPool().query( + `INSERT INTO recipients (name, normalized_name) VALUES ('Misc Payee', 'misc payee') RETURNING id`, + ); + rec.misc = rows[0].id; +} + +/** + * Create an accounts row with explicit population attributes. Accounts are + * pre-created (rather than left to the dual-write trigger) because this suite + * is about the population GATES — type, in_net_worth, statement_balance — which + * the trigger's onboarding INSERT does not set. + */ +async function addAccount(name, { + displayName = null, + type = 'checking', + inNetWorth = true, + currency = 'EUR', + statementBalance = null, + statementBalanceDate = null, +} = {}) { + const { rows } = await getTestPool().query( + `INSERT INTO accounts (name, display_name, type, in_net_worth, currency, + statement_balance, statement_balance_date) + VALUES ($1, $2, $3::account_type, $4, $5, $6, $7) RETURNING id`, + [name, displayName, type, inNetWorth, currency, statementBalance, statementBalanceDate], + ); + return rows[0].id; +} + +/** + * Insert one transaction via plain SQL. `date` is a SQL expression evaluated + * server-side so fixtures stay anchored to the same CURRENT_DATE the queries + * under test use. `balance` NULL means "not stamped by a bank import" — the + * distinction the anchor+delta lateral exists for. + */ +async function insertTxn({ + dateExpr, + amount, + currency = 'EUR', + bank, + balance = null, + isActive = true, +}) { + const { rows } = await getTestPool().query( + `INSERT INTO transactions (date, amount, currency, recipient_id, bank_account, balance, is_active) + VALUES ((${dateExpr})::date, $1, $2, $3, $4, $5, $6) RETURNING id`, + [amount, currency, rec.misc, bank, balance, isActive], + ); + return rows[0].id; +} + +/** Seed one exchange_rates row so conversion resolves from the DB, never the network. */ +async function insertRate(code, dateExpr, rate, isLatest = true) { + await getTestPool().query( + `INSERT INTO exchange_rates (currency_code, rate_date, rate_to_eur, is_latest) + VALUES ($1, (${dateExpr})::date, $2, $3)`, + [code, rate, isLatest], + ); +} + +describe.skipIf(!hasTestDatabase())('repositories/infoRepositoryBanks (real DB)', () => { + beforeAll(async () => { + expect( + process.env.DATABASE_URL, + 'DATABASE_URL must equal TEST_DATABASE_URL for this suite (see scripts/with-test-db.sh)', + ).toBe(process.env.TEST_DATABASE_URL); + // DB suites share one database across parallel vitest workers — serialize. + await acquireDbSuiteLock(); + }, 180_000); + + afterEach(async () => { + const pool = getTestPool(); + await pool.query('DELETE FROM transactions'); + await pool.query('DELETE FROM accounts'); + await pool.query('DELETE FROM recipients'); + await pool.query('DELETE FROM exchange_rates'); + for (const k of Object.keys(rec)) delete rec[k]; + clearMemoryCache(); + clearMvCache(); + }); + + afterAll(async () => { + await releaseDbSuiteLock(); + await closeTestPool(); + await closePool(); + }); + + it('returns the empty shape on an empty ledger', async () => { + expect(await banksRepository.getBankBalances()).toEqual({ + accounts: [], + total_net_position: 0, + history: {}, + total_history: [], + }); + }); + + // ─────────────────────────────────────────────────────────────────────────── + // Current balances — anchor+delta lateral and the population gates + // ─────────────────────────────────────────────────────────────────────────── + describe('current balances', () => { + it('computes anchor+delta for stamped accounts and Σ(amount) for manual-only ones', async () => { + await seedRecipient(); + await addAccount('AAA MANUAL'); + await addAccount('BBB STAMPED', { + displayName: 'KBC Zichtrekening', + statementBalance: '960.00', + statementBalanceDate: await ymdFromToday(), + }); + + // Manual-only: nothing stamped anywhere → Σ(amount) = 70. + await insertTxn({ dateExpr: "CURRENT_DATE - interval '20 days'", amount: '-10.00', bank: 'AAA MANUAL' }); + await insertTxn({ dateExpr: "CURRENT_DATE - interval '15 days'", amount: '-20.00', bank: 'AAA MANUAL' }); + await insertTxn({ dateExpr: "CURRENT_DATE - interval '2 days'", amount: '100.00', bank: 'AAA MANUAL' }); + // Stamped at day-10 (balance 1000 embeds the opening balance), then one + // unstamped manual row after the anchor → 1000 + (−25) = 975. + await insertTxn({ dateExpr: "CURRENT_DATE - interval '10 days'", amount: '-50.00', bank: 'BBB STAMPED', balance: '1000.00' }); + await insertTxn({ dateExpr: "CURRENT_DATE - interval '5 days'", amount: '-25.00', bank: 'BBB STAMPED' }); + + const r = await banksRepository.getBankBalances(); + + expect(r.accounts.map((a) => a.bank_account)).toEqual(['AAA MANUAL', 'BBB STAMPED']); // ORDER BY a.name + expect(r.accounts[0]).toMatchObject({ + bank_account: 'AAA MANUAL', + display_name: 'AAA MANUAL', // display_name NULL → falls back to name + balance: 70, + transaction_count: 3, + post_anchor_count: 3, // no anchor → "sum of 3 entries" + }); + expect(r.accounts[0].anchor_date).toBeUndefined(); + expect(r.accounts[0].drift).toBeUndefined(); // no statement balance stored + expect(r.accounts[1]).toMatchObject({ + bank_account: 'BBB STAMPED', + display_name: 'KBC Zichtrekening', + balance: 975, + transaction_count: 2, + anchor_date: await ymdFromToday("- interval '10 days'"), + post_anchor_count: 1, // "as of {anchor} statement + 1 entry since" + drift: -15, // statement_balance 960 − computed 975, native currency + }); + expect(r.total_net_position).toBe(1045); + }); + + it('applies the population gates: liability, tracking-only and no-activity accounts stay out', async () => { + await seedRecipient(); + await addAccount('IN SCOPE'); + await addAccount('MORTGAGE', { type: 'liability' }); + await addAccount('TRACKING ONLY', { inNetWorth: false }); + await addAccount('NO ACTIVITY'); // exists, but zero transactions + + await insertTxn({ dateExpr: 'CURRENT_DATE', amount: '10.00', bank: 'IN SCOPE' }); + await insertTxn({ dateExpr: 'CURRENT_DATE', amount: '-500.00', bank: 'MORTGAGE' }); + await insertTxn({ dateExpr: 'CURRENT_DATE', amount: '20.00', bank: 'TRACKING ONLY' }); + + const r = await banksRepository.getBankBalances(); + expect(r.accounts.map((a) => a.bank_account)).toEqual(['IN SCOPE']); + expect(r.total_net_position).toBe(10); + }); + + it('counts only ACTIVE rows in the balance, the count and the metadata lateral', async () => { + await seedRecipient(); + await addAccount('MIXED'); + await insertTxn({ dateExpr: "CURRENT_DATE - interval '3 days'", amount: '10.00', bank: 'MIXED' }); + await insertTxn({ dateExpr: "CURRENT_DATE - interval '1 day'", amount: '-999.00', bank: 'MIXED', isActive: false }); + + const r = await banksRepository.getBankBalances(); + expect(r.accounts[0]).toMatchObject({ balance: 10, transaction_count: 1 }); + // The inactive row is also invisible to the activity lateral's MIN/MAX. + // DATE columns cross the boundary as calendar-day strings (toWireDate). + expect(r.accounts[0].last_transaction).toBe(await ymdFromToday("- interval '3 days'")); + }); + + it('converts each account at the rate for its most recent activity date', async () => { + await seedRecipient(); + await addAccount('WISE USD', { currency: 'USD' }); + await insertRate('USD', 'CURRENT_DATE', '0.5'); + await insertTxn({ dateExpr: "CURRENT_DATE - interval '1 day'", amount: '80.00', currency: 'USD', bank: 'WISE USD' }); + + const r = await banksRepository.getBankBalances(); + expect(r.accounts[0]).toMatchObject({ bank_account: 'WISE USD', balance: 40 }); + expect(r.total_net_position).toBe(40); + }); + }); + + // ─────────────────────────────────────────────────────────────────────────── + // 12-month daily history + // ─────────────────────────────────────────────────────────────────────────── + describe('history', () => { + it('emits one point per day from the first activity to today, anchoring each day on the latest stamp ≤ that day', async () => { + await seedRecipient(); + await addAccount('HIST BANK'); + await insertTxn({ dateExpr: "CURRENT_DATE - interval '4 days'", amount: '-50.00', bank: 'HIST BANK', balance: '1000.00' }); + // A later UNSTAMPED row moves the series from the day it posts: the walk + // resolves each day with the same anchor+delta definition the headline + // uses (anchor = latest stamp ≤ day, plus every active row after it), + // so manual/trade activity is no longer invisible until the next import. + await insertTxn({ dateExpr: "CURRENT_DATE - interval '2 days'", amount: '-25.00', bank: 'HIST BANK' }); + await insertTxn({ dateExpr: "CURRENT_DATE - interval '1 day'", amount: '-5.00', bank: 'HIST BANK', balance: '900.00' }); + + const r = await banksRepository.getBankBalances(); + const days = [ + await ymdFromToday("- interval '4 days'"), + await ymdFromToday("- interval '3 days'"), + await ymdFromToday("- interval '2 days'"), + await ymdFromToday("- interval '1 day'"), + await ymdFromToday(), + ]; + expect(r.history['HIST BANK']).toEqual([ + { date: days[0], balance: 1000 }, + { date: days[1], balance: 1000 }, // forward-filled: no row that day + { date: days[2], balance: 975 }, // 1000 stamp − the unstamped 25 + { date: days[3], balance: 900 }, // re-anchored on the new stamp + { date: days[4], balance: 900 }, + ]); + expect(r.total_history).toEqual(r.history['HIST BANK']); + // The invariant both history findings demand: the headline IS the last + // chart point, never a step above it. + expect(r.total_history[r.total_history.length - 1].balance).toBe(r.total_net_position); + }); + + it('fills the whole 12-month window from a stamp that predates it', async () => { + await seedRecipient(); + await addAccount('OLD STAMP'); + await insertTxn({ dateExpr: "CURRENT_DATE - interval '18 months'", amount: '-1.00', bank: 'OLD STAMP', balance: '500.00' }); + + const r = await banksRepository.getBankBalances(); + const points = r.history['OLD STAMP']; + // generate_series(CURRENT_DATE - 12 months, CURRENT_DATE, 1 day) inclusive. + const { rows } = await getTestPool().query( + `SELECT (CURRENT_DATE - (CURRENT_DATE - interval '12 months')::date + 1) AS n`, + ); + expect(points).toHaveLength(Number(rows[0].n)); + expect(points[0].date).toBe(await ymdFromToday("- interval '12 months'")); + expect(points[points.length - 1].date).toBe(await ymdFromToday()); + expect(points.every((p) => p.balance === 500)).toBe(true); + }); + + it('opens the window at the balance carried in from BEFORE it, stamps and unstamped rows alike', async () => { + // The series is computed over the whole ledger and then clamped onto the + // grid: activity older than the 12-month window must arrive as the + // window's opening balance, not as a missing/zero point. + await seedRecipient(); + await addAccount('OLD ACTIVITY'); + await insertTxn({ dateExpr: "CURRENT_DATE - interval '18 months'", amount: '-1.00', bank: 'OLD ACTIVITY', balance: '500.00' }); + await insertTxn({ dateExpr: "CURRENT_DATE - interval '15 months'", amount: '-100.00', bank: 'OLD ACTIVITY' }); + + const r = await banksRepository.getBankBalances(); + const points = r.history['OLD ACTIVITY']; + expect(points[0].date).toBe(await ymdFromToday("- interval '12 months'")); + expect(points.every((p) => p.balance === 400)).toBe(true); + expect(r.total_net_position).toBe(400); + }); + + it('orders WITHIN a day: an unstamped row after that day’s stamp moves the same day’s point', async () => { + // Same-date rows tie-break on id, exactly as the current-balance anchor + // does — the stamp anchors and the later row is a delta on top of it. + await seedRecipient(); + await addAccount('SAME DAY'); + await insertTxn({ dateExpr: "CURRENT_DATE - interval '1 day'", amount: '-10.00', bank: 'SAME DAY', balance: '1000.00' }); + await insertTxn({ dateExpr: "CURRENT_DATE - interval '1 day'", amount: '-25.00', bank: 'SAME DAY' }); + + const r = await banksRepository.getBankBalances(); + expect(r.history['SAME DAY']).toEqual([ + { date: await ymdFromToday("- interval '1 day'"), balance: 975 }, + { date: await ymdFromToday(), balance: 975 }, + ]); + expect(r.total_net_position).toBe(975); + }); + + it('ignores inactive rows in the series, as the current balance does', async () => { + await seedRecipient(); + await addAccount('HIST ACTIVE'); + await insertTxn({ dateExpr: "CURRENT_DATE - interval '2 days'", amount: '100.00', bank: 'HIST ACTIVE' }); + await insertTxn({ dateExpr: "CURRENT_DATE - interval '1 day'", amount: '-999.00', bank: 'HIST ACTIVE', isActive: false }); + + const r = await banksRepository.getBankBalances(); + expect(r.history['HIST ACTIVE'].every((p) => p.balance === 100)).toBe(true); + expect(r.total_net_position).toBe(100); + }); + + it('sums per-day across accounts into total_history, honouring the same population gates', async () => { + await seedRecipient(); + await addAccount('A BANK'); + await addAccount('B BANK'); + await addAccount('C TRACKING', { inNetWorth: false }); + await insertTxn({ dateExpr: "CURRENT_DATE - interval '2 days'", amount: '-1.00', bank: 'A BANK', balance: '100.00' }); + await insertTxn({ dateExpr: "CURRENT_DATE - interval '1 day'", amount: '-1.00', bank: 'B BANK', balance: '50.00' }); + await insertTxn({ dateExpr: "CURRENT_DATE - interval '2 days'", amount: '-1.00', bank: 'C TRACKING', balance: '9999.00' }); + + const r = await banksRepository.getBankBalances(); + const d2 = await ymdFromToday("- interval '2 days'"); + const d1 = await ymdFromToday("- interval '1 day'"); + const d0 = await ymdFromToday(); + expect(Object.keys(r.history).sort()).toEqual(['A BANK', 'B BANK']); // tracking-only excluded + expect(r.total_history).toEqual([ + { date: d2, balance: 100 }, + { date: d1, balance: 150 }, + { date: d0, balance: 150 }, + ]); + }); + }); + + // ─────────────────────────────────────────────────────────────────────────── + // Formerly pinned discrepancies — now corrective + // ─────────────────────────────────────────────────────────────────────────── + describe('wire-date convention', () => { + // Was PIN 1. `anchor_date` is emitted as a 'YYYY-MM-DD' string (the lateral + // uses to_char) while `first_transaction` / `last_transaction` were passed + // through RAW from pg. pg reads a DATE column as a JS Date at *local* + // midnight, and lib/dateFormat.js documents the project rule: DATE values + // at an emit boundary go through toWireDate(), because JSON-serializing the + // raw Date emits an ISO timestamp of the PREVIOUS day on any server east of + // UTC (Brussels: all day, every day). The mock suite fed these fields as + // strings, so only a real-DB test can see the type. + it('emits first_transaction/last_transaction as calendar-day strings, not raw pg Dates', async () => { + await seedRecipient(); + await addAccount('DATE SHAPE'); + await insertTxn({ dateExpr: "CURRENT_DATE - interval '3 days'", amount: '-1.00', bank: 'DATE SHAPE' }); + await insertTxn({ dateExpr: "CURRENT_DATE - interval '1 day'", amount: '-2.00', bank: 'DATE SHAPE' }); + + const account = (await banksRepository.getBankBalances()).accounts[0]; + expect(account.first_transaction).toBe(await ymdFromToday("- interval '3 days'")); + expect(account.last_transaction).toBe(await ymdFromToday("- interval '1 day'")); + // The whole object survives JSON round-tripping as calendar days — the + // actual failure mode was an ISO timestamp landing a day early. + const wire = JSON.parse(JSON.stringify(account)); + expect(wire.first_transaction).toBe(account.first_transaction); + expect(wire.last_transaction).toBe(account.last_transaction); + // …matching the provenance field on the SAME object, already a string. + expect(typeof account.anchor_date === 'undefined' || typeof account.anchor_date === 'string').toBe(true); + }); + }); + + describe('multi-currency accounts', () => { + // Was PIN 2. The unpartitioned lateral did `SUM(t2.amount)` across + // currencies and the caller converted that total at the ONE rate belonging + // to the most recent row's currency. The balance computation is now + // partitioned by currency and each partition converted at its own rate. + it('converts each currency partition at its own rate instead of summing raw amounts', async () => { + await seedRecipient(); + await addAccount('MULTI CCY'); + await insertRate('USD', 'CURRENT_DATE', '0.5'); + await insertTxn({ dateExpr: "CURRENT_DATE - interval '2 days'", amount: '100.00', currency: 'EUR', bank: 'MULTI CCY' }); + await insertTxn({ dateExpr: "CURRENT_DATE - interval '1 day'", amount: '100.00', currency: 'USD', bank: 'MULTI CCY' }); + + const r = await banksRepository.getBankBalances(); + expect(r.accounts[0].balance).toBe(150); // 100 EUR + (100 USD × 0.5) — NOT (100+100) × 0.5 + expect(r.total_net_position).toBe(150); + // …and the chart agrees with the headline it sits under. + expect(r.total_history[r.total_history.length - 1].balance).toBe(150); + }); + + it('anchors each currency on ITS OWN latest stamp — a EUR statement never absorbs USD activity', async () => { + // Composition semantics: a stamped `balance` is the statement figure for + // the currency of the row carrying it, so it anchors that partition only. + // EUR: stamp 1000 (day-10) − 25 (day-3) = 975 EUR + // USD: nothing stamped → Σ = 100 USD @0.5 = 50 EUR + await seedRecipient(); + await addAccount('WISE MULTI', { currency: 'EUR' }); + await insertRate('USD', 'CURRENT_DATE', '0.5'); + await insertTxn({ dateExpr: "CURRENT_DATE - interval '10 days'", amount: '-50.00', currency: 'EUR', bank: 'WISE MULTI', balance: '1000.00' }); + await insertTxn({ dateExpr: "CURRENT_DATE - interval '5 days'", amount: '100.00', currency: 'USD', bank: 'WISE MULTI' }); + await insertTxn({ dateExpr: "CURRENT_DATE - interval '3 days'", amount: '-25.00', currency: 'EUR', bank: 'WISE MULTI' }); + + const r = await banksRepository.getBankBalances(); + // The old cross-currency form gave 1000 + (100 − 25) = 1075 at one rate. + expect(r.accounts[0].balance).toBe(1025); + expect(r.total_net_position).toBe(1025); + expect(r.total_history[r.total_history.length - 1].balance).toBe(1025); + }); + + it('revalues the headline at TODAY rate, not at the last statement date, over a moving curve', async () => { + // The archetypal imported foreign-currency account: fully stamped, last + // statement a month old, rate moved since. Keying the headline's FX on + // the account's last activity (as it briefly did) pinned it to the old + // rate while the chart — which revalues day by day — ended at the new + // one: 500 under a chart ending at 900. Both must read 900. + // NOTE: a single seeded rate row makes this test vacuous — the curve has + // to move for the two conventions to disagree. + await seedRecipient(); + await addAccount('WISE USD', { currency: 'USD' }); + await insertRate('USD', "CURRENT_DATE - interval '30 days'", '0.5', false); + await insertRate('USD', 'CURRENT_DATE', '0.9'); + await insertTxn({ + dateExpr: "CURRENT_DATE - interval '30 days'", + amount: '-10.00', currency: 'USD', bank: 'WISE USD', balance: '1000.00', + }); + + const r = await banksRepository.getBankBalances(); + expect(r.accounts[0].balance).toBe(900); // 1000 USD × today's 0.9 + expect(r.total_net_position).toBe(900); + + const points = r.history['WISE USD']; + expect(points[points.length - 1]).toEqual({ date: await ymdFromToday(), balance: 900 }); + // …and the day of the statement is still valued at the rate of that day, + // so the series genuinely moves with FX (otherwise the check above is + // satisfied by a flat curve). + const statementDay = await ymdFromToday("- interval '30 days'"); + expect(points.find((p) => p.date === statementDay)).toEqual({ date: statementDay, balance: 500 }); + }); + + it('holds headline == last chart point for a MIXED-currency account on a moving curve', async () => { + await seedRecipient(); + await addAccount('MULTI MOVING', { currency: 'EUR' }); + await insertRate('USD', "CURRENT_DATE - interval '20 days'", '0.5', false); + await insertRate('USD', 'CURRENT_DATE', '0.8'); + await insertTxn({ dateExpr: "CURRENT_DATE - interval '20 days'", amount: '100.00', currency: 'EUR', bank: 'MULTI MOVING' }); + await insertTxn({ dateExpr: "CURRENT_DATE - interval '20 days'", amount: '200.00', currency: 'USD', bank: 'MULTI MOVING' }); + + const r = await banksRepository.getBankBalances(); + expect(r.accounts[0].balance).toBe(260); // 100 EUR + 200 USD × 0.8 + const points = r.history['MULTI MOVING']; + expect(points[points.length - 1].balance).toBe(260); + expect(points[0]).toEqual({ // 20 days ago, at that day's 0.5 + date: await ymdFromToday("- interval '20 days'"), + balance: 200, + }); + expect(r.total_history[r.total_history.length - 1].balance).toBe(r.total_net_position); + }); + + it('leaves a single-currency account byte-identical to the unpartitioned computation', async () => { + // The overwhelmingly common case: one partition, whose anchor is the + // account's latest stamped row and whose delta is every row after it. + await seedRecipient(); + await addAccount('ONE CCY USD', { currency: 'USD' }); + await insertRate('USD', 'CURRENT_DATE', '0.5'); + await insertTxn({ dateExpr: "CURRENT_DATE - interval '10 days'", amount: '-50.00', currency: 'USD', bank: 'ONE CCY USD', balance: '1000.00' }); + await insertTxn({ dateExpr: "CURRENT_DATE - interval '5 days'", amount: '-100.00', currency: 'USD', bank: 'ONE CCY USD' }); + + const r = await banksRepository.getBankBalances(); + expect(r.accounts[0]).toMatchObject({ + balance: 450, // (1000 − 100) USD × 0.5 + anchor_date: await ymdFromToday("- interval '10 days'"), + post_anchor_count: 1, + }); + expect(r.total_net_position).toBe(450); + }); + }); + + describe('history/headline agreement', () => { + // Was PIN 3. The current-balance query dropped its `balance IS NOT NULL` + // gate (WP-A1) so manual-only accounts reach the headline, but the history + // query kept filtering `WHERE lb.balance IS NOT NULL` — so an all-manual + // ledger rendered a non-zero total_net_position above an EMPTY chart, and a + // mixed one rendered a today-point that silently disagreed with it. + it('includes manual-only accounts in history, so today equals total_net_position (mixed ledger)', async () => { + await seedRecipient(); + await addAccount('MANUAL ONLY'); + await addAccount('STAMPED'); + await insertTxn({ dateExpr: "CURRENT_DATE - interval '1 day'", amount: '70.00', bank: 'MANUAL ONLY' }); + await insertTxn({ dateExpr: "CURRENT_DATE - interval '1 day'", amount: '-1.00', bank: 'STAMPED', balance: '1000.00' }); + + const r = await banksRepository.getBankBalances(); + expect(r.total_net_position).toBe(1070); + expect(Object.keys(r.history).sort()).toEqual(['MANUAL ONLY', 'STAMPED']); + expect(r.history['MANUAL ONLY']).toEqual([ + { date: await ymdFromToday("- interval '1 day'"), balance: 70 }, + { date: await ymdFromToday(), balance: 70 }, + ]); + expect(r.total_history[r.total_history.length - 1].balance).toBe(r.total_net_position); + }); + + // KNOWN DIVERGENCE (deliberate, and the only one left): the headline is the + // unbounded computed balance — the same figure the accounts hub shows, which + // it must not drift from — while the chart stops at today. A FUTURE-dated + // row therefore counts in the headline before it reaches the chart. Bounding + // the headline at today instead would fix the chart at the cost of the hub + // agreement; pinned here so the trade-off stays visible. + it('counts a future-dated row in the headline but not yet in the chart', async () => { + await seedRecipient(); + await addAccount('AHEAD'); + await insertTxn({ dateExpr: "CURRENT_DATE - interval '1 day'", amount: '100.00', bank: 'AHEAD' }); + await insertTxn({ dateExpr: "CURRENT_DATE + interval '3 days'", amount: '40.00', bank: 'AHEAD' }); + + const r = await banksRepository.getBankBalances(); + expect(r.total_net_position).toBe(140); + expect(r.total_history[r.total_history.length - 1]).toEqual({ + date: await ymdFromToday(), + balance: 100, + }); + }); + + it('charts an all-manual ledger instead of leaving the headline over an empty chart', async () => { + await seedRecipient(); + await addAccount('CASH'); + await addAccount('WALLET'); + await insertTxn({ dateExpr: "CURRENT_DATE - interval '3 days'", amount: '200.00', bank: 'CASH' }); + await insertTxn({ dateExpr: "CURRENT_DATE - interval '2 days'", amount: '-50.00', bank: 'CASH' }); + await insertTxn({ dateExpr: "CURRENT_DATE - interval '1 day'", amount: '30.00', bank: 'WALLET' }); + + const r = await banksRepository.getBankBalances(); + expect(r.total_net_position).toBe(180); + // A day BEFORE an account's first row yields no point for it (its first + // known balance is never carried backwards), so the running total starts + // at first activity and steps only where transactions actually land. + expect(r.total_history).toEqual([ + { date: await ymdFromToday("- interval '3 days'"), balance: 200 }, + { date: await ymdFromToday("- interval '2 days'"), balance: 150 }, + { date: await ymdFromToday("- interval '1 day'"), balance: 180 }, + { date: await ymdFromToday(), balance: 180 }, + ]); + expect(r.total_history[r.total_history.length - 1].balance).toBe(r.total_net_position); + }); + }); +}); diff --git a/apps/node-backend/tests/infoRepoForecast.db.test.js b/apps/node-backend/tests/infoRepoForecast.db.test.js new file mode 100644 index 00000000..38e4bcae --- /dev/null +++ b/apps/node-backend/tests/infoRepoForecast.db.test.js @@ -0,0 +1,556 @@ +/** + * Real-Postgres tests for infoRepositoryForecast — `getCashflowComparison`, + * `getCashflowForecastData`, `getCashflowForecastDataRolling` and + * `getCashflowForecastDataByCategory`. + * + * DB-backed complement to infoRepoForecast.test.js (which stays: it runs without + * a DB, and its argument-validation cases are pure JS with no DB reachable). + * The mock suite pins SQL text — `interval '12 months'`, `planned_date > + * CURRENT_DATE`, param numbering — and hand-feeds already-converted rows into + * the JS reducers. It therefore proves nothing about which rows Postgres puts in + * (or leaves out of) each window, which is where every behaviour below lives: + * the four/three/two parallel window predicates, the planned-transaction + * overlays and their `is_executed` gate, the 3-level effective-category JOIN, + * and per-date FX off seeded `exchange_rates` rows. + * + * Determinism: every window is anchored on `CURRENT_DATE`, so no fixture uses a + * literal calendar date — dates are SQL expressions relative to the same anchors + * the queries use, and day-of-month expectations are derived from the response's + * own `current_day` / `days_in_month`. Day 5 and day 10 exist in every month, so + * the past-month averages are exact regardless of when the suite runs. + */ + +import { afterAll, afterEach, beforeAll, describe, expect, it } from 'vitest'; +import { + acquireDbSuiteLock, + closeTestPool, + getTestPool, + hasTestDatabase, + releaseDbSuiteLock, +} from './setup/db.js'; +import { + getCashflowComparison, + getCashflowForecastData, + getCashflowForecastDataRolling, + getCashflowForecastDataByCategory, +} from '../src/repositories/infoRepositoryForecast.js'; +import { getAverageVsCurrentSpending } from '../src/repositories/infoRepositoryAverageVsCurrent.js'; +import { clearMvCache } from '../src/repositories/infoRepositoryHelpers.js'; +import { clearMemoryCache } from '../src/services/currency/currencyConversionService.js'; +import { closePool } from '../src/database/connection.js'; + +const cat = {}; +const rec = {}; + +/** Day N of the month `monthsBack` months before the current one, as 'YYYY-MM-DD'. */ +const monthDay = (monthsBack, day) => + `date_trunc('month', CURRENT_DATE) - interval '${monthsBack} months' + interval '${day - 1} days'`; + +/** 'YYYY-MM-DD' for an arbitrary date expression. */ +async function ymd(dateExpr) { + const { rows } = await getTestPool().query(`SELECT to_char((${dateExpr})::date, 'YYYY-MM-DD') AS d`); + return rows[0].d; +} + +/** + * Categories and the alias topology: `Electrabel` (primary, default category + * Bills) with alias `Electrabel Invoicing` carrying no category of its own — + * the 3-level effective-category case the by-category forecast must resolve. + */ +async function seedBase() { + const pool = getTestPool(); + for (const [key, [general, detail]] of Object.entries({ + Food: ['Food', 'Groceries'], + Bills: ['Bills', 'Utilities'], + })) { + const { rows } = await pool.query( + 'INSERT INTO categories (general, detail) VALUES ($1, $2) RETURNING id', + [general, detail], + ); + cat[key] = rows[0].id; + } + const addRecipient = async (name, { defaultCategoryId = null, primaryId = null } = {}) => { + const { rows } = await pool.query( + `INSERT INTO recipients (name, normalized_name, default_category_id, primary_recipient_id) + VALUES ($1, $2, $3, $4) RETURNING id`, + [name, name.toLowerCase(), defaultCategoryId, primaryId], + ); + return rows[0].id; + }; + rec.electrabel = await addRecipient('Electrabel', { defaultCategoryId: cat.Bills }); + rec.electrabelAlias = await addRecipient('Electrabel Invoicing', { primaryId: rec.electrabel }); + rec.misc = await addRecipient('Misc Payee'); +} + +/** + * Ensure an accounts row exists for a label (the transactions dual-write trigger + * resolves `bank_account` → `account_id`; pre-creating keeps fixture setup + * independent of the trigger). + */ +async function ensureAccount(name) { + await getTestPool().query( + `INSERT INTO accounts (name, display_name) VALUES ($1, $1) + ON CONFLICT (lower(btrim(name))) DO NOTHING`, + [name], + ); +} + +async function insertTxn({ + dateExpr, + amount, + currency = 'EUR', + recipientId = null, + categoryId = null, + bank = 'MAIN BANK', + isActive = true, + isTransfer = false, +}) { + if (bank) await ensureAccount(bank); + const { rows } = await getTestPool().query( + `INSERT INTO transactions (date, amount, currency, recipient_id, category_id, bank_account, is_active, is_transfer) + VALUES ((${dateExpr})::date, $1, $2, $3, $4, $5, $6, $7) RETURNING id`, + [amount, currency, recipientId ?? rec.misc, categoryId, bank, isActive, isTransfer], + ); + return rows[0].id; +} + +async function insertPlanned({ + dateExpr, + amount, + currency = 'EUR', + isActive = true, + isExecuted = false, +}) { + await getTestPool().query( + `INSERT INTO planned_transactions (planned_date, amount, currency, is_active, is_executed) + VALUES ((${dateExpr})::date, $1, $2, $3, $4)`, + [amount, currency, isActive, isExecuted], + ); +} + +/** Seed one exchange_rates row so conversion resolves from the DB, never the network. */ +async function insertRate(code, dateExpr, rate, isLatest = true) { + await getTestPool().query( + `INSERT INTO exchange_rates (currency_code, rate_date, rate_to_eur, is_latest) + VALUES ($1, (${dateExpr})::date, $2, $3)`, + [code, rate, isLatest], + ); +} + +describe.skipIf(!hasTestDatabase())('repositories/infoRepositoryForecast (real DB)', () => { + beforeAll(async () => { + expect( + process.env.DATABASE_URL, + 'DATABASE_URL must equal TEST_DATABASE_URL for this suite (see scripts/with-test-db.sh)', + ).toBe(process.env.TEST_DATABASE_URL); + // DB suites share one database across parallel vitest workers — serialize. + await acquireDbSuiteLock(); + }, 180_000); + + afterEach(async () => { + const pool = getTestPool(); + await pool.query('DELETE FROM planned_transactions'); + await pool.query('DELETE FROM transactions'); + await pool.query('DELETE FROM accounts'); + await pool.query('DELETE FROM recipients'); + await pool.query('DELETE FROM categories'); + await pool.query('DELETE FROM exchange_rates'); + await pool.query(`DELETE FROM user_settings WHERE key = 'includeTransfers'`); + for (const bag of [cat, rec]) for (const k of Object.keys(bag)) delete bag[k]; + clearMemoryCache(); + clearMvCache(); + }); + + afterAll(async () => { + await releaseDbSuiteLock(); + await closeTestPool(); + await closePool(); + }); + + // ─────────────────────────────────────────────────────────────────────────── + // getCashflowComparison + // ─────────────────────────────────────────────────────────────────────────── + describe('getCashflowComparison', () => { + it('averages the running cumulative net across the elapsed months of history', async () => { + await seedBase(); + // Two past months inside the 24-month window; day 5 and day 10 exist in + // every month, so these day buckets are calendar-independent. + await insertTxn({ dateExpr: monthDay(1, 5), amount: '100.00' }); + await insertTxn({ dateExpr: monthDay(1, 10), amount: '-30.00' }); + await insertTxn({ dateExpr: monthDay(2, 5), amount: '60.00' }); + // Outside the 24-month window — must not enter the average. + await insertTxn({ dateExpr: monthDay(25, 5), amount: '9999.00' }); + // Inactive rows never count. + await insertTxn({ dateExpr: monthDay(1, 5), amount: '-777.00', isActive: false }); + + const r = await getCashflowComparison([], [], 'EUR'); + const byDay = Object.fromEntries(r.without_planned.map((d) => [d.day, d])); + + // Divisor is the elapsed months since this ledger's earliest in-window + // month (2 months back → 2), not the full 24-month span. The 25-months-back + // row is outside the window, so it does not stretch the span either. + expect(byDay[4].average).toBe(0); + expect(byDay[5].average).toBe(80); // (100 + 60) / 2 + expect(byDay[9].average).toBe(80); // cumulative carries forward + expect(byDay[10].average).toBe(65); // ((100 − 30) + 60) / 2 + expect(byDay[r.days_in_month].average).toBe(65); + }); + + it('accumulates the current month up to today and leaves later days null', async () => { + await seedBase(); + await insertTxn({ dateExpr: monthDay(0, 1), amount: '50.00' }); + await insertTxn({ dateExpr: 'CURRENT_DATE', amount: '-10.00' }); + // A future-dated row inside this month is invisible (t.date <= CURRENT_DATE). + await insertTxn({ dateExpr: "date_trunc('month', CURRENT_DATE) + interval '1 month' - interval '1 day'", amount: '-500.00' }); + + const r = await getCashflowComparison([], [], 'EUR'); + const byDay = Object.fromEntries(r.without_planned.map((d) => [d.day, d])); + + expect(byDay[1].current).not.toBeNull(); + // Cumulative through today covers both current-month rows (they collapse + // onto one day when the suite runs on the 1st). + expect(byDay[r.current_day].current).toBe(40); + if (r.current_day < r.days_in_month) { + expect(byDay[r.current_day + 1].current).toBeNull(); + expect(byDay[r.days_in_month].current).toBeNull(); + } + expect(r.without_planned).toHaveLength(r.days_in_month); + expect(r.with_planned).toHaveLength(r.days_in_month); + }); + + it('overlays unexecuted planned transactions, in both the current month and the historical average', async () => { + await seedBase(); + await insertTxn({ dateExpr: monthDay(0, 1), amount: '50.00' }); + await insertTxn({ dateExpr: 'CURRENT_DATE', amount: '-10.00' }); + await insertPlanned({ dateExpr: monthDay(0, 1), amount: '-20.00' }); + // Executed / inactive planned rows must not double-count against the real + // transactions they already produced. + await insertPlanned({ dateExpr: monthDay(0, 1), amount: '-4000.00', isExecuted: true }); + await insertPlanned({ dateExpr: monthDay(0, 1), amount: '-5000.00', isActive: false }); + // One historical planned month → its own cumulative average. + await insertPlanned({ dateExpr: monthDay(1, 5), amount: '-40.00' }); + + const r = await getCashflowComparison([], [], 'EUR'); + const plain = Object.fromEntries(r.without_planned.map((d) => [d.day, d])); + const planned = Object.fromEntries(r.with_planned.map((d) => [d.day, d])); + + expect(planned[r.current_day].current).toBe(plain[r.current_day].current - 20); + expect(planned[4].average).toBe(plain[4].average); + expect(planned[5].average).toBe(plain[5].average - 40); + }); + + it('applies alias-aware category and recipient exclusions to the live rows', async () => { + await seedBase(); + await insertTxn({ dateExpr: monthDay(1, 5), amount: '100.00', recipientId: rec.misc, categoryId: cat.Food }); + // Categorised only via the PRIMARY of its alias (3rd resolution level). + await insertTxn({ dateExpr: monthDay(1, 5), amount: '200.00', recipientId: rec.electrabelAlias }); + + const all = await getCashflowComparison([], [], 'EUR'); + expect(all.without_planned.find((d) => d.day === 5).average).toBe(300); + + // Excluding Bills drops the alias row even though it carries no category_id. + const exclCat = await getCashflowComparison([cat.Bills], [], 'EUR'); + expect(exclCat.without_planned.find((d) => d.day === 5).average).toBe(100); + + // Excluding the PRIMARY recipient also drops rows recorded under its alias. + const exclRec = await getCashflowComparison([], [rec.electrabel], 'EUR'); + expect(exclRec.without_planned.find((d) => d.day === 5).average).toBe(100); + }); + + it('converts each day at its own stored rate', async () => { + await seedBase(); + await insertRate('USD', monthDay(1, 5), '0.5', false); + await insertRate('USD', monthDay(2, 5), '0.25', false); + await insertTxn({ dateExpr: monthDay(1, 5), amount: '100.00', currency: 'USD' }); + await insertTxn({ dateExpr: monthDay(2, 5), amount: '100.00', currency: 'USD' }); + + const r = await getCashflowComparison([], [], 'EUR'); + // Same nominal amount, different months, different rates: (50 + 25) / 2. + expect(r.without_planned.find((d) => d.day === 5).average).toBe(37.5); + }); + }); + + // ─────────────────────────────────────────────────────────────────────────── + // getCashflowForecastData / …Rolling + // ─────────────────────────────────────────────────────────────────────────── + describe('getCashflowForecastData', () => { + it('splits history / currentActual / plannedCurrent / plannedHist on the real window boundaries', async () => { + await seedBase(); + await insertTxn({ dateExpr: monthDay(1, 5), amount: '100.00' }); + await insertTxn({ dateExpr: monthDay(1, 5), amount: '-40.00' }); // same day → summed + await insertTxn({ dateExpr: monthDay(2, 5), amount: '10.00' }); + await insertTxn({ dateExpr: monthDay(4, 5), amount: '9999.00' }); // outside a 3-month history + await insertTxn({ dateExpr: 'CURRENT_DATE', amount: '-7.00' }); + await insertPlanned({ dateExpr: monthDay(0, 1), amount: '-20.00' }); + await insertPlanned({ dateExpr: monthDay(1, 5), amount: '-30.00' }); + + const r = await getCashflowForecastData(3, [], [], 'EUR'); + + expect(r.historyMonths).toBe(3); + expect(r.history).toEqual([ + { date: await ymd(monthDay(2, 5)), net: 10 }, + { date: await ymd(monthDay(1, 5)), net: 60 }, // 100 − 40, ascending by date + ]); + expect(r.currentActual).toEqual([{ date: await ymd('CURRENT_DATE'), net: -7 }]); + expect(r.plannedCurrent).toEqual([{ date: await ymd(monthDay(0, 1)), net: -20 }]); + expect(r.plannedHist).toEqual([{ date: await ymd(monthDay(1, 5)), net: -30 }]); + }); + }); + + describe('getCashflowForecastDataRolling', () => { + it('keeps history strictly before the rolling window and only future planned rows', async () => { + await seedBase(); + await insertTxn({ dateExpr: "CURRENT_DATE - interval '40 days'", amount: '100.00' }); + // Exactly on the boundary: history is `< CURRENT_DATE - daysBack`, current + // is `>= CURRENT_DATE - daysBack`, so this row belongs to currentActual. + await insertTxn({ dateExpr: "CURRENT_DATE - interval '30 days'", amount: '5.00' }); + await insertTxn({ dateExpr: 'CURRENT_DATE', amount: '-3.00' }); + await insertPlanned({ dateExpr: 'CURRENT_DATE', amount: '-99.00' }); // not > CURRENT_DATE + await insertPlanned({ dateExpr: "CURRENT_DATE + interval '10 days'", amount: '-50.00' }); + await insertPlanned({ dateExpr: "CURRENT_DATE + interval '90 days'", amount: '-11.00' }); // beyond daysForward + + const r = await getCashflowForecastDataRolling(12, 30, 60, [], [], 'EUR'); + + expect(r.history).toEqual([{ date: await ymd("CURRENT_DATE - interval '40 days'"), net: 100 }]); + expect(r.currentActual).toEqual([ + { date: await ymd("CURRENT_DATE - interval '30 days'"), net: 5 }, + { date: await ymd('CURRENT_DATE'), net: -3 }, + ]); + expect(r.plannedCurrent).toEqual([ + { date: await ymd("CURRENT_DATE + interval '10 days'"), net: -50 }, + ]); + }); + }); + + // ─────────────────────────────────────────────────────────────────────────── + // getCashflowForecastDataByCategory + // ─────────────────────────────────────────────────────────────────────────── + describe('getCashflowForecastDataByCategory', () => { + it('resolves the effective category over all three levels and labels the rest Uncategorized', async () => { + await seedBase(); + await insertTxn({ dateExpr: monthDay(1, 5), amount: '-50.00', recipientId: rec.misc, categoryId: cat.Food }); + await insertTxn({ dateExpr: monthDay(1, 5), amount: '-25.00', recipientId: rec.misc, categoryId: cat.Food }); + // Own category NULL, alias's own default NULL → resolved via the PRIMARY. + await insertTxn({ dateExpr: monthDay(1, 5), amount: '-1000.00', recipientId: rec.electrabelAlias }); + // Uncategorised at all three levels. + await insertTxn({ dateExpr: monthDay(1, 5), amount: '-10.00', recipientId: rec.misc }); + await insertTxn({ dateExpr: 'CURRENT_DATE', amount: '-5.00', recipientId: rec.misc, categoryId: cat.Food }); + + const r = await getCashflowForecastDataByCategory(3, [], [], 'EUR'); + const day = await ymd(monthDay(1, 5)); + + expect(r.historyByCategory).toEqual( + expect.arrayContaining([ + { date: day, category_id: cat.Food, general: 'Food', detail: 'Groceries', net: -75 }, + { date: day, category_id: cat.Bills, general: 'Bills', detail: 'Utilities', net: -1000 }, + { date: day, category_id: null, general: 'Uncategorized', detail: 'Uncategorized', net: -10 }, + ]), + ); + expect(r.historyByCategory).toHaveLength(3); + expect(r.currentActualByCategory).toEqual([ + { date: await ymd('CURRENT_DATE'), category_id: cat.Food, general: 'Food', detail: 'Groceries', net: -5 }, + ]); + }); + }); + + // ─────────────────────────────────────────────────────────────────────────── + // ADR-083 transfer exclusion (was a PIN: the module counted transfer legs) + // ─────────────────────────────────────────────────────────────────────────── + describe('ADR-083 internal-transfer exclusion', () => { + // Was pinned as a discrepancy: every query in infoRepositoryForecast.js + // filtered on `t.is_active = true` only, with no getIncludeTransfers() call + // anywhere in the module, while the sibling surfaces on the SAME dashboard + // (infoRepositoryAverageVsCurrent.js:22-23, infoRepositoryMonthly.js:38/194) + // honoured the setting. On this fixture the cash-flow chart read −1000 and + // the avg-vs-current card read 100. Now all three agree. + it('excludes transfer legs from the comparison, matching getAverageVsCurrentSpending', async () => { + await seedBase(); + await insertTxn({ dateExpr: 'CURRENT_DATE', amount: '-100.00', isTransfer: false }); + await insertTxn({ dateExpr: 'CURRENT_DATE', amount: '-900.00', isTransfer: true }); + + const comparison = await getCashflowComparison([], [], 'EUR'); + const current = comparison.without_planned.find((d) => d.day === comparison.current_day); + expect(current.current).toBe(-100); // transfer leg excluded + + const avg = await getAverageVsCurrentSpending('EUR'); + expect(avg.current_month.total_spending).toBe(100); // same fixture, same answer + + // The forecast pipeline inherits the exclusion. + const forecast = await getCashflowForecastData(3, [], [], 'EUR'); + expect(forecast.currentActual).toEqual([{ date: await ymd('CURRENT_DATE'), net: -100 }]); + }); + + it('excludes transfer legs from the rolling and by-category surfaces too', async () => { + await seedBase(); + await insertTxn({ dateExpr: 'CURRENT_DATE', amount: '-100.00', recipientId: rec.misc, categoryId: cat.Food }); + await insertTxn({ dateExpr: 'CURRENT_DATE', amount: '-900.00', recipientId: rec.misc, categoryId: cat.Food, isTransfer: true }); + // History side of both windows. + await insertTxn({ dateExpr: monthDay(1, 5), amount: '50.00' }); + await insertTxn({ dateExpr: monthDay(1, 5), amount: '-700.00', isTransfer: true }); + + const rolling = await getCashflowForecastDataRolling(12, 30, 60, [], [], 'EUR'); + expect(rolling.currentActual).toEqual([{ date: await ymd('CURRENT_DATE'), net: -100 }]); + expect(rolling.history).toEqual([{ date: await ymd(monthDay(1, 5)), net: 50 }]); + + const byCat = await getCashflowForecastDataByCategory(3, [], [], 'EUR'); + expect(byCat.currentActualByCategory).toEqual([ + { date: await ymd('CURRENT_DATE'), category_id: cat.Food, general: 'Food', detail: 'Groceries', net: -100 }, + ]); + // The transfer leg would otherwise invent −900 of "Food" spending. + expect(byCat.historyByCategory).toEqual([ + { date: await ymd(monthDay(1, 5)), category_id: null, general: 'Uncategorized', detail: 'Uncategorized', net: 50 }, + ]); + }); + + // It is a runtime setting, not a hardcoded filter: opting in must bring the + // legs back, on every surface. + it('counts transfer legs again when includeTransfers is switched on', async () => { + await seedBase(); + await getTestPool().query( + `INSERT INTO user_settings (key, value) VALUES ('includeTransfers', 'true'::jsonb) + ON CONFLICT (key) DO UPDATE SET value = EXCLUDED.value`, + ); + await insertTxn({ dateExpr: 'CURRENT_DATE', amount: '-100.00' }); + await insertTxn({ dateExpr: 'CURRENT_DATE', amount: '-900.00', isTransfer: true }); + + const comparison = await getCashflowComparison([], [], 'EUR'); + const current = comparison.without_planned.find((d) => d.day === comparison.current_day); + expect(current.current).toBe(-1000); + + const forecast = await getCashflowForecastData(3, [], [], 'EUR'); + expect(forecast.currentActual).toEqual([{ date: await ymd('CURRENT_DATE'), net: -1000 }]); + + const rolling = await getCashflowForecastDataRolling(12, 30, 60, [], [], 'EUR'); + expect(rolling.currentActual).toEqual([{ date: await ymd('CURRENT_DATE'), net: -1000 }]); + + const byCat = await getCashflowForecastDataByCategory(3, [], [], 'EUR'); + expect(byCat.currentActualByCategory).toEqual([ + { date: await ymd('CURRENT_DATE'), category_id: null, general: 'Uncategorized', detail: 'Uncategorized', net: -1000 }, + ]); + }); + }); + + // ─────────────────────────────────────────────────────────────────────────── + // Historical-average denominator (was a PIN: divided by populated months) + // ─────────────────────────────────────────────────────────────────────────── + describe('historical average denominator', () => { + // Was pinned as a discrepancy: the divisor was the number of months that + // HAPPEN to carry rows, so one 240 month in a 24-month window reported an + // "average" of 240 — a sparse ledger drew an average line as tall as its + // single busiest month. An elapsed month with no rows is a real zero. + it('divides a sparse window by every elapsed month, not by the populated ones', async () => { + await seedBase(); + // Oldest month inside the 24-month window → the ledger's observed span is + // the full window, so the divisor is 24. + await insertTxn({ dateExpr: monthDay(24, 5), amount: '240.00' }); + + const r = await getCashflowComparison([], [], 'EUR'); + expect(r.without_planned.find((d) => d.day === 5).average).toBe(10); // 240 / 24 + }); + + // …and the counterweight: dividing by the whole window unconditionally + // would deflate a young ledger. Months BEFORE the first entry are not + // observations (the app did not exist for that user), so they must not be + // charged as zeros. Fresh-install shape: history starts 3 months ago. + it('does not charge phantom zeros for months before the ledger started', async () => { + await seedBase(); + // Ledger opens 3 months back; month −2 is genuinely empty, month −1 has data. + await insertTxn({ dateExpr: monthDay(3, 5), amount: '30.00' }); + await insertTxn({ dateExpr: monthDay(1, 5), amount: '30.00' }); + + const r = await getCashflowComparison([], [], 'EUR'); + // 60 / 3 elapsed months. Old behaviour (populated months only) gave 30; + // dividing by the 24-month window would give 2.5. + expect(r.without_planned.find((d) => d.day === 5).average).toBe(20); + }); + + it('counts every elapsed month once history is contiguous', async () => { + await seedBase(); + for (const back of [3, 2, 1]) { + await insertTxn({ dateExpr: monthDay(back, 5), amount: '30.00' }); + } + + const r = await getCashflowComparison([], [], 'EUR'); + expect(r.without_planned.find((d) => d.day === 5).average).toBe(30); // 90 / 3 + }); + + // One divisor for both historical series: a month with transactions but no + // planned rows is a month in which the user planned nothing (a real zero), + // so the planned overlay must not be re-based onto its own shorter span. + // Orientation A — ledger OLDER than the plan. + it('shares the ledger-wide divisor with the planned-history overlay', async () => { + await seedBase(); + await insertTxn({ dateExpr: monthDay(3, 5), amount: '30.00' }); + await insertPlanned({ dateExpr: monthDay(1, 5), amount: '-60.00' }); + + const r = await getCashflowComparison([], [], 'EUR'); + const plain = r.without_planned.find((d) => d.day === 5); + const planned = r.with_planned.find((d) => d.day === 5); + + expect(plain.average).toBe(10); // 30 / 3 + // −60 / 3, not −60 / 1: the planned series rides the same 3-month span. + expect(planned.average).toBe(-10); + }); + + // Orientation B — plan OLDER than the ledger, the orientation where a + // union-of-keys divisor breaks. A recurring plan the auto-linker never + // matched keeps its original past planned_date forever, so a one-month-old + // ledger can easily carry an un-executed row dated 24 months back. That row + // must not stretch the divisor: it deflated the transactions average 24x + // (30 → 1.25), the exact "short history" failure the divisor exists to stop. + it('does not let a stale planned row stretch the divisor backwards', async () => { + await seedBase(); + await insertTxn({ dateExpr: monthDay(1, 5), amount: '30.00' }); + await insertPlanned({ dateExpr: monthDay(24, 5), amount: '-60.00' }); + + const r = await getCashflowComparison([], [], 'EUR'); + // Ledger is one complete month old → divisor 1, so the transactions + // average is untouched by the plan's age. + expect(r.without_planned.find((d) => d.day === 5).average).toBe(30); + // The stale plan still contributes its numerator on the with-planned + // line (it is inside the window); it just cannot move the denominator. + expect(r.with_planned.find((d) => d.day === 5).average).toBe(-30); + }); + + // The divisor answers "when did this ledger start", which is a property of + // the ledger — not of the current view. Deriving it from the filtered rows + // let an exclusion that empties the oldest months re-base it, so toggling a + // category exclusion moved the average line by a factor unrelated to the + // rows it removed. + it('keeps the divisor when a category exclusion empties the oldest month', async () => { + await seedBase(); + // Oldest month is Food-only; excluding Food empties it entirely. + await insertTxn({ dateExpr: monthDay(3, 5), amount: '30.00', recipientId: rec.misc, categoryId: cat.Food }); + await insertTxn({ dateExpr: monthDay(1, 5), amount: '30.00', recipientId: rec.misc }); + + const all = await getCashflowComparison([], [], 'EUR'); + expect(all.without_planned.find((d) => d.day === 5).average).toBe(20); // 60 / 3 + + const exclFood = await getCashflowComparison([cat.Food], [], 'EUR'); + // Numerator drops to 30, denominator stays 3 → 10. If the divisor came + // from the filtered rows it would re-base to 1 and report 30. + expect(exclFood.without_planned.find((d) => d.day === 5).average).toBe(10); + }); + + // Same argument for the ADR-083 filter itself: excluding transfer legs must + // change the numerator only. + it('keeps the divisor when the transfer filter empties the oldest month', async () => { + await seedBase(); + await insertTxn({ dateExpr: monthDay(3, 5), amount: '30.00', isTransfer: true }); + await insertTxn({ dateExpr: monthDay(1, 5), amount: '30.00' }); + + const r = await getCashflowComparison([], [], 'EUR'); + expect(r.without_planned.find((d) => d.day === 5).average).toBe(10); // 30 / 3, not 30 / 1 + }); + + // Soft-deleted rows are not history, so they do not establish the start. + it('ignores inactive rows when locating the ledger start', async () => { + await seedBase(); + await insertTxn({ dateExpr: monthDay(12, 5), amount: '-5000.00', isActive: false }); + await insertTxn({ dateExpr: monthDay(2, 5), amount: '30.00' }); + + const r = await getCashflowComparison([], [], 'EUR'); + expect(r.without_planned.find((d) => d.day === 5).average).toBe(15); // 30 / 2 + }); + }); +}); diff --git a/apps/node-backend/tests/infoRepoForecast.test.js b/apps/node-backend/tests/infoRepoForecast.test.js index 2dcab478..07a0ba96 100644 --- a/apps/node-backend/tests/infoRepoForecast.test.js +++ b/apps/node-backend/tests/infoRepoForecast.test.js @@ -4,16 +4,24 @@ vi.mock('../src/database/connection.js', () => ({ query: vi.fn(), })); +// getIncludeTransfers() reads `user_settings` (ADR-083). Stub it so the module +// under test does not spend a `query` mock call on the settings lookup — the +// call-count/param assertions below are about the cash-flow SQL only. Its +// behaviour is exercised for real in infoRepoForecast.db.test.js. vi.mock('../src/repositories/infoRepositoryHelpers.js', async () => { const actual = await vi.importActual('../src/repositories/infoRepositoryHelpers.js'); return { ...actual, batchConvertGroupsWithHistoricalRateFallback: vi.fn(), + getIncludeTransfers: vi.fn().mockResolvedValue(false), }; }); import { query } from '../src/database/connection.js'; -import { batchConvertGroupsWithHistoricalRateFallback } from '../src/repositories/infoRepositoryHelpers.js'; +import { + batchConvertGroupsWithHistoricalRateFallback, + getIncludeTransfers, +} from '../src/repositories/infoRepositoryHelpers.js'; import { getCashflowComparison, getCashflowForecastData, @@ -24,22 +32,37 @@ import { ValidationError } from '../src/middleware/errorHandler.js'; beforeEach(() => { vi.clearAllMocks(); + // clearAllMocks wipes the factory's mockResolvedValue — restore the default. + getIncludeTransfers.mockResolvedValue(false); vi.useFakeTimers(); vi.setSystemTime(new Date('2025-04-15T12:00:00Z')); }); afterEach(() => vi.useRealTimers()); +/** Matches the unfiltered ledger-start probe (last query of getCashflowComparison). */ +const isLedgerStartSql = (sql) => /MIN\(t\.date\)/.test(sql); + +/** + * query() stub. The ledger-start probe answers with `firstDate` (a 'YYYY-MM-DD' + * string or null); every other query returns no rows. Needed because the probe + * — not the result rows — is what sets the historical-average divisor. + */ +function stubQueries(firstDate = null) { + query.mockImplementation(async (sql) => + isLedgerStartSql(sql) ? { rows: [{ first_date: firstDate }] } : { rows: [] }); +} + describe('getCashflowComparison', () => { function setupEmpty() { - query.mockResolvedValue({ rows: [] }); + stubQueries(null); batchConvertGroupsWithHistoricalRateFallback.mockResolvedValue([[], [], [], []]); } - it('runs four parallel queries (past, current, planned current, planned hist)', async () => { + it('runs five parallel queries (past, current, planned current, planned hist, ledger start)', async () => { setupEmpty(); await getCashflowComparison([], [], 'EUR'); - expect(query).toHaveBeenCalledTimes(4); + expect(query).toHaveBeenCalledTimes(5); expect(query.mock.calls[0][0]).toContain("date >= date_trunc('month', CURRENT_DATE) - interval '24 months'"); expect(query.mock.calls[0][0]).toContain("date < date_trunc('month', CURRENT_DATE)"); expect(query.mock.calls[1][0]).toContain('CURRENT_DATE'); @@ -49,6 +72,9 @@ describe('getCashflowComparison', () => { // executed non-recurring row double-counts against its real transaction. expect(query.mock.calls[2][0]).toContain('is_executed = false'); expect(query.mock.calls[3][0]).toContain('is_executed = false'); + // The ledger-start probe is appended LAST so the four data queries keep + // their call order, and carries no filters of any kind (see D2 below). + expect(query.mock.calls[4][0]).toContain('MIN(t.date)'); }); it('returns days_in_month, current_day, month, year aligned to the system clock', async () => { @@ -72,7 +98,7 @@ describe('getCashflowComparison', () => { }); it('builds cumulative averages from past month data', async () => { - query.mockResolvedValue({ rows: [] }); + stubQueries('2025-01-05'); // ledger starts 2025-01 batchConvertGroupsWithHistoricalRateFallback.mockResolvedValueOnce([ [ { day_of_month: 5, month_key: '2025-01', amount_eur: 100 }, @@ -88,10 +114,14 @@ describe('getCashflowComparison', () => { ]); const r = await getCashflowComparison([], [], 'EUR'); - // Day 5: avg of (100 from Jan, 60 from Feb)/2 = 80 - expect(r.without_planned.find((d) => d.day === 5).average).toBe(80); - // Day 10: ((100-30) + 60)/2 = 65 - expect(r.without_planned.find((d) => d.day === 10).average).toBe(65); + // Divisor is elapsed months since the ledger STARTED (the unfiltered probe + // says 2025-01), NOT the count of months carrying rows: "today" is + // 2025-04-15, so Jan/Feb/Mar are all elapsed and March's silence is a real + // zero → 3. + // Day 5: (100 from Jan + 60 from Feb) / 3 = 53.33 + expect(r.without_planned.find((d) => d.day === 5).average).toBe(53.33); + // Day 10: ((100-30) + 60) / 3 = 43.33 + expect(r.without_planned.find((d) => d.day === 10).average).toBe(43.33); // Current day 1: 50; day 3 cumulative: 40 expect(r.without_planned.find((d) => d.day === 1).current).toBe(50); expect(r.without_planned.find((d) => d.day === 3).current).toBe(40); @@ -264,3 +294,87 @@ describe('getCashflowForecastDataByCategory', () => { expect(params).toEqual([10, 11, 22]); }); }); + +// ADR-083: every `transactions` query in this module must carry the transfer +// predicate unless the user opted in, exactly like the sibling surfaces +// (infoRepositoryAverageVsCurrent.js, infoRepositoryMonthly.js). The previous +// version of this suite asserted the substrings it expected to be PRESENT and +// so never noticed the absent one; these cases assert both directions. +describe('ADR-083 transfer exclusion', () => { + /** + * SQL of every DATA query issued against `transactions`. Excludes the + * ledger-start probe, which is unfiltered by design (asserted separately). + */ + const transactionSqls = () => + query.mock.calls + .map((c) => c[0]) + .filter((sql) => /FROM transactions\b/.test(sql) && !isLedgerStartSql(sql)); + + const cases = [ + ['getCashflowComparison', () => getCashflowComparison([], [], 'EUR'), 4, 2], + ['getCashflowForecastData', () => getCashflowForecastData(12, [], [], 'EUR'), 4, 2], + ['getCashflowForecastDataRolling', () => getCashflowForecastDataRolling(12, 30, 60), 3, 2], + ['getCashflowForecastDataByCategory', () => getCashflowForecastDataByCategory(6), 2, 2], + ]; + + for (const [name, call, queryCount, txnQueryCount] of cases) { + it(`${name} excludes transfers by default`, async () => { + query.mockResolvedValue({ rows: [] }); + batchConvertGroupsWithHistoricalRateFallback.mockResolvedValue( + Array.from({ length: queryCount }, () => []), + ); + + await call(); + const sqls = transactionSqls(); + expect(sqls).toHaveLength(txnQueryCount); + for (const sql of sqls) expect(sql).toContain('AND t.is_transfer = false'); + }); + + it(`${name} keeps transfers when includeTransfers is on`, async () => { + getIncludeTransfers.mockResolvedValue(true); + query.mockResolvedValue({ rows: [] }); + batchConvertGroupsWithHistoricalRateFallback.mockResolvedValue( + Array.from({ length: queryCount }, () => []), + ); + + await call(); + for (const sql of transactionSqls()) expect(sql).not.toContain('is_transfer'); + }); + } + + // planned_transactions has no `is_transfer` column (ADR-083 flagged + // `transactions` only), so the planned overlays deliberately carry no + // predicate. Pinned so a future "consistency" edit does not add one and + // break the query with a 42703 undefined_column. + it('never puts a transfer predicate on the planned_transactions overlays', async () => { + query.mockResolvedValue({ rows: [] }); + batchConvertGroupsWithHistoricalRateFallback.mockResolvedValue([[], [], [], []]); + + await getCashflowComparison([], [], 'EUR'); + await getCashflowForecastData(12, [], [], 'EUR'); + const plannedSqls = query.mock.calls + .map((c) => c[0]) + .filter((sql) => /FROM planned_transactions\b/.test(sql)); + expect(plannedSqls).toHaveLength(4); + for (const sql of plannedSqls) expect(sql).not.toContain('is_transfer'); + }); + + // The ledger-start probe decides the historical-average divisor. If any + // filter reached it, excluding a category (or excluding transfers) could + // empty the oldest months and silently re-base the divisor — the average + // line would move for reasons unrelated to the excluded rows. + it('never filters the ledger-start probe', async () => { + stubQueries('2024-01-01'); + batchConvertGroupsWithHistoricalRateFallback.mockResolvedValue([[], [], [], []]); + + await getCashflowComparison([1, 2], [9], 'EUR'); + const [probeSql, probeParams] = query.mock.calls.find((c) => isLedgerStartSql(c[0])); + expect(probeSql).not.toContain('is_transfer'); + expect(probeSql).not.toContain('NOT IN'); + expect(probeSql).not.toContain('LEFT JOIN'); + expect(probeSql).not.toContain('planned_transactions'); + expect(probeParams).toBeUndefined(); + // Still window-clamped: a row older than the lookback cannot lengthen it. + expect(probeSql).toContain("interval '24 months'"); + }); +}); diff --git a/apps/node-backend/tests/infoRepoMonthly.db.test.js b/apps/node-backend/tests/infoRepoMonthly.db.test.js new file mode 100644 index 00000000..ae0ecae3 --- /dev/null +++ b/apps/node-backend/tests/infoRepoMonthly.db.test.js @@ -0,0 +1,359 @@ +/** + * Real-Postgres tests for infoRepositoryMonthly (`getMonthlyFinancialSummary`). + * + * DB-backed complement to infoRepoMonthly.test.js (which stays: it runs without + * a DB). The mock suite stubs `mvAvailable` and `getIncludeTransfers` outright + * and matches SQL substrings (`FROM mv_monthly_summary`, `generate_series`, + * `NOT IN ($1, $2)`), so it verifies which branch the JS *chose* — never what + * either branch computes. Here both paths run for real: the live path against + * the 6-month `generate_series` × `daily` join, and the materialized-view fast + * path against an actually-created `mv_monthly_summary` (created inside the MV + * tests and dropped again, since every other DB suite assumes a freshly-migrated + * database with no MVs). + * + * The headline property is the one the module's own comments claim and no mock + * could ever check: toggling an exclusion switches paths, so the two paths must + * agree on the same corpus. + * + * Determinism: the window is anchored on `CURRENT_DATE`, so fixtures are dated + * by SQL expressions relative to it. Day 5/10/15 exist in every month. + */ + +import { afterAll, afterEach, beforeAll, describe, expect, it } from 'vitest'; +import { + acquireDbSuiteLock, + closeTestPool, + getTestPool, + hasTestDatabase, + releaseDbSuiteLock, +} from './setup/db.js'; +import { getMonthlyFinancialSummary } from '../src/repositories/infoRepositoryMonthly.js'; +import { createMaterializedViews } from '../src/services/materializedViewService.js'; +import { clearMvCache } from '../src/repositories/infoRepositoryHelpers.js'; +import { clearMemoryCache } from '../src/services/currency/currencyConversionService.js'; +import { closePool } from '../src/database/connection.js'; + +const MANAGED_VIEWS = ['mv_monthly_summary', 'mv_category_totals', 'mv_cashflow_daily']; + +const cat = {}; +const rec = {}; + +/** Day N of the month `monthsBack` months before the current one. */ +const monthDay = (monthsBack, day) => + `date_trunc('month', CURRENT_DATE) - interval '${monthsBack} months' + interval '${day - 1} days'`; + +/** 'YYYY-MM' key for the month `monthsBack` months before the current one. */ +async function monthKey(monthsBack) { + const { rows } = await getTestPool().query( + `SELECT to_char(date_trunc('month', CURRENT_DATE) - interval '${monthsBack} months', 'YYYY-MM') AS k`, + ); + return rows[0].k; +} + +/** 'YYYY-MM-DD' for an arbitrary date expression. */ +async function ymd(dateExpr) { + const { rows } = await getTestPool().query(`SELECT to_char((${dateExpr})::date, 'YYYY-MM-DD') AS d`); + return rows[0].d; +} + +/** A month row keyed 'YYYY-MM', for readable assertions. */ +const byMonth = (result) => + Object.fromEntries(result.months.map((m) => [`${m.year}-${String(m.month).padStart(2, '0')}`, m])); + +async function seedBase() { + const pool = getTestPool(); + for (const [key, [general, detail]] of Object.entries({ + Food: ['Food', 'Groceries'], + Bills: ['Bills', 'Utilities'], + })) { + const { rows } = await pool.query( + 'INSERT INTO categories (general, detail) VALUES ($1, $2) RETURNING id', + [general, detail], + ); + cat[key] = rows[0].id; + } + const addRecipient = async (name, { defaultCategoryId = null, primaryId = null } = {}) => { + const { rows } = await pool.query( + `INSERT INTO recipients (name, normalized_name, default_category_id, primary_recipient_id) + VALUES ($1, $2, $3, $4) RETURNING id`, + [name, name.toLowerCase(), defaultCategoryId, primaryId], + ); + return rows[0].id; + }; + rec.electrabel = await addRecipient('Electrabel', { defaultCategoryId: cat.Bills }); + rec.electrabelAlias = await addRecipient('Electrabel Invoicing', { primaryId: rec.electrabel }); + rec.misc = await addRecipient('Misc Payee'); +} + +async function ensureAccount(name) { + await getTestPool().query( + `INSERT INTO accounts (name, display_name) VALUES ($1, $1) + ON CONFLICT (lower(btrim(name))) DO NOTHING`, + [name], + ); +} + +async function insertTxn({ + dateExpr, + amount, + currency = 'EUR', + recipientId = null, + categoryId = null, + bank = 'MAIN BANK', + isActive = true, + isTransfer = false, +}) { + if (bank) await ensureAccount(bank); + const { rows } = await getTestPool().query( + `INSERT INTO transactions (date, amount, currency, recipient_id, category_id, bank_account, is_active, is_transfer) + VALUES ((${dateExpr})::date, $1, $2, $3, $4, $5, $6, $7) RETURNING id`, + [amount, currency, recipientId ?? rec.misc, categoryId, bank, isActive, isTransfer], + ); + return rows[0].id; +} + +async function dropManagedViews() { + const pool = getTestPool(); + for (const view of MANAGED_VIEWS) { + await pool.query(`DROP MATERIALIZED VIEW IF EXISTS ${view} CASCADE`); + } + clearMvCache(); +} + +describe.skipIf(!hasTestDatabase())('repositories/infoRepositoryMonthly (real DB)', () => { + beforeAll(async () => { + expect( + process.env.DATABASE_URL, + 'DATABASE_URL must equal TEST_DATABASE_URL for this suite (see scripts/with-test-db.sh)', + ).toBe(process.env.TEST_DATABASE_URL); + // DB suites share one database across parallel vitest workers — serialize. + await acquireDbSuiteLock(); + }, 180_000); + + afterEach(async () => { + const pool = getTestPool(); + // MVs first: they are the only artefact of this suite that outlives a row + // wipe, and every other DB suite assumes a migrated DB with no MVs. + await dropManagedViews(); + await pool.query('DELETE FROM transactions'); + await pool.query('DELETE FROM accounts'); + await pool.query('DELETE FROM recipients'); + await pool.query('DELETE FROM categories'); + await pool.query('DELETE FROM exchange_rates'); + await pool.query(`DELETE FROM user_settings WHERE key = 'includeTransfers'`); + for (const bag of [cat, rec]) for (const k of Object.keys(bag)) delete bag[k]; + clearMemoryCache(); + }); + + afterAll(async () => { + await releaseDbSuiteLock(); + await closeTestPool(); + await closePool(); + }); + + // ─────────────────────────────────────────────────────────────────────────── + // Live path + // ─────────────────────────────────────────────────────────────────────────── + describe('live path', () => { + it('returns exactly the 6-month window, zero-filling months with no rows', async () => { + await seedBase(); + await insertTxn({ dateExpr: monthDay(2, 10), amount: '-25.00' }); + + const r = await getMonthlyFinancialSummary([], 'EUR', [], false); + expect(r.months).toHaveLength(6); + const expectedKeys = []; + for (let i = 5; i >= 0; i -= 1) expectedKeys.push(await monthKey(i)); + expect(r.months.map((m) => `${m.year}-${String(m.month).padStart(2, '0')}`)).toEqual(expectedKeys); + + const months = byMonth(r); + expect(months[await monthKey(2)]).toMatchObject({ + total_spending: -25, total_income: 0, net_amount: -25, transaction_count: 1, + }); + expect(months[await monthKey(3)]).toMatchObject({ + total_spending: 0, total_income: 0, net_amount: 0, transaction_count: 0, + }); + }); + + it('emits period_start/period_end as calendar-day strings covering the whole month', async () => { + await seedBase(); + const r = await getMonthlyFinancialSummary([], 'EUR', [], false); + const current = byMonth(r)[await monthKey(0)]; + expect(current.period_start).toBe(await ymd(`date_trunc('month', CURRENT_DATE)`)); + expect(current.period_end).toBe( + await ymd(`date_trunc('month', CURRENT_DATE) + interval '1 month' - interval '1 day'`), + ); + expect(typeof current.period_start).toBe('string'); + }); + + it('splits income from spending per month and rolls the window into summary', async () => { + await seedBase(); + await insertTxn({ dateExpr: monthDay(1, 5), amount: '2000.00' }); + await insertTxn({ dateExpr: monthDay(1, 5), amount: '-300.00' }); // same day, same currency + await insertTxn({ dateExpr: monthDay(1, 15), amount: '-200.00' }); + await insertTxn({ dateExpr: monthDay(0, 10), amount: '-50.00' }); + await insertTxn({ dateExpr: monthDay(0, 10), amount: '-9999.00', isActive: false }); + // Outside the window entirely. + await insertTxn({ dateExpr: monthDay(9, 10), amount: '-1234.00' }); + + const r = await getMonthlyFinancialSummary([], 'EUR', [], false); + const months = byMonth(r); + expect(months[await monthKey(1)]).toMatchObject({ + total_income: 2000, total_spending: -500, net_amount: 1500, transaction_count: 3, + }); + expect(months[await monthKey(0)]).toMatchObject({ + total_income: 0, total_spending: -50, net_amount: -50, transaction_count: 1, + }); + expect(r.summary).toMatchObject({ + total_income: 2000, total_spending: -550, net_amount: 1450, transaction_count: 4, + }); + expect(r.summary.period_start).toBe(await ymd(`date_trunc('month', CURRENT_DATE) - interval '5 months'`)); + }); + + it('excludes internal transfers by default and includes them when the setting is on', async () => { + await seedBase(); + await insertTxn({ dateExpr: monthDay(1, 5), amount: '-100.00' }); + await insertTxn({ dateExpr: monthDay(1, 5), amount: '-900.00', isTransfer: true }); + + const off = await getMonthlyFinancialSummary([], 'EUR', [], false); + expect(byMonth(off)[await monthKey(1)]).toMatchObject({ total_spending: -100, transaction_count: 1 }); + + await getTestPool().query( + `INSERT INTO user_settings (key, value) VALUES ('includeTransfers', 'true'::jsonb)`, + ); + const on = await getMonthlyFinancialSummary([], 'EUR', [], false); + expect(byMonth(on)[await monthKey(1)]).toMatchObject({ total_spending: -1000, transaction_count: 2 }); + }); + + it('applies alias-aware category and recipient exclusions to real rows', async () => { + await seedBase(); + await insertTxn({ dateExpr: monthDay(1, 5), amount: '-100.00', recipientId: rec.misc, categoryId: cat.Food }); + // Categorised only via the PRIMARY of its alias (3rd resolution level). + await insertTxn({ dateExpr: monthDay(1, 5), amount: '-200.00', recipientId: rec.electrabelAlias }); + // Uncategorised at all three levels — must SURVIVE any exclusion (the -1 + // sentinel in buildExclusionClauses exists exactly for this). + await insertTxn({ dateExpr: monthDay(1, 5), amount: '-7.00', recipientId: rec.misc }); + + const all = await getMonthlyFinancialSummary([], 'EUR', [], false); + expect(byMonth(all)[await monthKey(1)].total_spending).toBe(-307); + + const exclBills = await getMonthlyFinancialSummary([cat.Bills], 'EUR', [], false); + expect(byMonth(exclBills)[await monthKey(1)]).toMatchObject({ total_spending: -107, transaction_count: 2 }); + + const exclPrimary = await getMonthlyFinancialSummary([], 'EUR', [rec.electrabel], false); + expect(byMonth(exclPrimary)[await monthKey(1)]).toMatchObject({ total_spending: -107, transaction_count: 2 }); + }); + + it('extends the series back to the earliest active transaction when allTime is set', async () => { + await seedBase(); + await insertTxn({ dateExpr: monthDay(8, 10), amount: '-10.00' }); + await insertTxn({ dateExpr: monthDay(0, 10), amount: '-20.00' }); + + const r = await getMonthlyFinancialSummary([], 'EUR', [], true); + expect(r.months).toHaveLength(9); // month-8 … month-0 inclusive + expect(byMonth(r)[await monthKey(8)]).toMatchObject({ total_spending: -10 }); + expect(byMonth(r)[await monthKey(7)]).toMatchObject({ total_spending: 0, transaction_count: 0 }); + }); + + it('converts each day at its own stored rate', async () => { + await seedBase(); + await getTestPool().query( + `INSERT INTO exchange_rates (currency_code, rate_date, rate_to_eur, is_latest) + VALUES ('USD', (${monthDay(1, 5)})::date, 0.5, false), + ('USD', (${monthDay(2, 5)})::date, 0.25, false)`, + ); + await insertTxn({ dateExpr: monthDay(1, 5), amount: '-100.00', currency: 'USD' }); + await insertTxn({ dateExpr: monthDay(2, 5), amount: '-100.00', currency: 'USD' }); + + const r = await getMonthlyFinancialSummary([], 'EUR', [], false); + expect(byMonth(r)[await monthKey(1)].total_spending).toBe(-50); + expect(byMonth(r)[await monthKey(2)].total_spending).toBe(-25); + }); + }); + + // ─────────────────────────────────────────────────────────────────────────── + // Materialized-view fast path + // ─────────────────────────────────────────────────────────────────────────── + describe('materialized-view fast path', () => { + /** The corpus used for both path-agreement tests. */ + async function seedCorpus() { + await seedBase(); + await insertTxn({ dateExpr: monthDay(1, 5), amount: '2000.00', recipientId: rec.misc }); + await insertTxn({ dateExpr: monthDay(1, 5), amount: '-300.00', recipientId: rec.misc, categoryId: cat.Food }); + await insertTxn({ dateExpr: monthDay(1, 15), amount: '-200.00', recipientId: rec.electrabelAlias }); + await insertTxn({ dateExpr: monthDay(3, 10), amount: '-40.00', recipientId: rec.misc }); + await insertTxn({ dateExpr: monthDay(0, 10), amount: '-50.00', recipientId: rec.misc }); + await insertTxn({ dateExpr: monthDay(0, 11), amount: '-900.00', recipientId: rec.misc, isTransfer: true }); + } + + it('produces the SAME months and summary as the live path on one corpus', async () => { + await seedCorpus(); + // No MVs yet → live path. + const live = await getMonthlyFinancialSummary([], 'EUR', [], false); + + await createMaterializedViews(); + clearMvCache(); + const mv = await getMonthlyFinancialSummary([], 'EUR', [], false); + + // The module's stated invariant: toggling an exclusion flips the code + // path, so the dashboard's month set and figures must not move with it. + expect(mv.months).toEqual(live.months); + expect(mv.summary).toEqual(live.summary); + }); + + it('really serves from the MV: a post-creation insert stays invisible until REFRESH', async () => { + // Path proof for the agreement test above — without it, an MV that failed + // to populate would make "MV path == live path" vacuously true. The MV is + // a snapshot, so a row inserted after CREATE is absent until refreshed. + await seedCorpus(); + await createMaterializedViews(); + clearMvCache(); + + const before = await getMonthlyFinancialSummary([], 'EUR', [], false); + await insertTxn({ dateExpr: monthDay(0, 12), amount: '-77.00', recipientId: rec.misc }); + const after = await getMonthlyFinancialSummary([], 'EUR', [], false); + expect(after.months).toEqual(before.months); // stale MV — the new row is unseen + + await getTestPool().query('REFRESH MATERIALIZED VIEW mv_monthly_summary'); + const refreshed = await getMonthlyFinancialSummary([], 'EUR', [], false); + expect(byMonth(refreshed)[await monthKey(0)].total_spending) + .toBe(byMonth(before)[await monthKey(0)].total_spending - 77); + }); + + it('falls back to the live path when the MV holds a non-target currency', async () => { + await seedBase(); + await getTestPool().query( + `INSERT INTO exchange_rates (currency_code, rate_date, rate_to_eur, is_latest) + VALUES ('USD', (${monthDay(1, 5)})::date, 0.5, true)`, + ); + await insertTxn({ dateExpr: monthDay(1, 5), amount: '-100.00', currency: 'USD' }); + await insertTxn({ dateExpr: monthDay(1, 6), amount: '-10.00', currency: 'EUR' }); + + const live = await getMonthlyFinancialSummary([], 'EUR', [], false); + await createMaterializedViews(); + clearMvCache(); + const withMv = await getMonthlyFinancialSummary([], 'EUR', [], false); + + // Mixed currencies ⇒ the homogeneity probe rejects the MV, so the + // per-(date,currency) live conversion is used and the answer is unchanged. + expect(withMv.months).toEqual(live.months); + expect(byMonth(withMv)[await monthKey(1)].total_spending).toBe(-60); + }); + + it('never takes the MV path when an exclusion or allTime is requested', async () => { + await seedCorpus(); + await createMaterializedViews(); + clearMvCache(); + + // allTime reaches back further than the MV path's fixed 6-month window, + // proving the live query ran. + await insertTxn({ dateExpr: monthDay(8, 10), amount: '-11.00', recipientId: rec.misc }); + const allTime = await getMonthlyFinancialSummary([], 'EUR', [], true); + expect(allTime.months.length).toBeGreaterThan(6); + expect(byMonth(allTime)[await monthKey(8)].total_spending).toBe(-11); + + // An exclusion likewise forces the live path — and is honoured. + const excluded = await getMonthlyFinancialSummary([cat.Food], 'EUR', [], false); + expect(byMonth(excluded)[await monthKey(1)].total_spending).toBe(-200); + }); + }); +}); diff --git a/apps/node-backend/tests/infoRepository.db.test.js b/apps/node-backend/tests/infoRepository.db.test.js new file mode 100644 index 00000000..357fa926 --- /dev/null +++ b/apps/node-backend/tests/infoRepository.db.test.js @@ -0,0 +1,699 @@ +/** + * Real-Postgres tests for the infoRepository barrel — the sub-repositories that + * only reach a consumer through it and have no dedicated DB suite of their own: + * `getNetWorthFromSnapshots` (infoRepositoryNetWorth), `getPlannedExpensesNextMonth` + * (infoRepositoryPlanned) and `getAverageVsCurrentSpending` + * (infoRepositoryAverageVsCurrent), plus the barrel's own assembly. + * + * DB-backed complement to infoRepository.test.js (which stays: it runs without a + * DB). That mock suite dispatches on SQL substrings — `if (sql.includes('WITH + * anchor'))`, `if (sql.includes('balance_series'))` — to decide which canned rows + * to hand back, so it asserts the shape of the JS reducers and the presence of + * SQL fragments, never what the statements return. Everything here runs against + * the migrated schema: the daily balance-series walk and the shared anchor+delta + * lateral resolving over real NUMERIC/DATE columns, `portfolio_performance_snapshots` + * joined by day, the transaction-flow fallback, the planned-transaction recurrence + * expansion off real DATE values, and the 6-month/current-month aggregation windows. + * + * The other barrel members already have dedicated DB suites — statistics + * (infoRepoStatistics.db.test.js), banks (infoRepoBanks.db.test.js), monthly + * (infoRepoMonthly.db.test.js), forecast (infoRepoForecast.db.test.js) and + * recipients (infoRepositoryRecipients.db.test.js) — so they are only smoke-checked + * here for barrel wiring. + * + * Determinism: net-worth day series and the planned-expense window are anchored + * on the APP-TIMEZONE today (ADR-009), so those fixtures are dated with the very + * same `todayAppDateString()` helper the repository uses rather than SQL's + * `CURRENT_DATE` — the two can disagree for the last hours of a UTC day. + */ + +import { afterAll, afterEach, beforeAll, describe, expect, it } from 'vitest'; +import { + acquireDbSuiteLock, + closeTestPool, + getTestPool, + hasTestDatabase, + releaseDbSuiteLock, +} from './setup/db.js'; +import infoRepository from '../src/repositories/infoRepository.js'; +import { clearMvCache } from '../src/repositories/infoRepositoryHelpers.js'; +import { clearMemoryCache } from '../src/services/currency/currencyConversionService.js'; +import { closePool } from '../src/database/connection.js'; +import { todayAppDateString, addDaysYmd, firstOfMonthYmd } from '../src/lib/timezone.js'; + +const cat = {}; +const rec = {}; + +const TODAY = () => todayAppDateString(); + +async function seedBase() { + const pool = getTestPool(); + for (const [key, [general, detail]] of Object.entries({ + Food: ['Food', 'Groceries'], + Rent: ['Home', 'Rent'], + })) { + const { rows } = await pool.query( + 'INSERT INTO categories (general, detail) VALUES ($1, $2) RETURNING id', + [general, detail], + ); + cat[key] = rows[0].id; + } + const { rows } = await pool.query( + `INSERT INTO recipients (name, normalized_name) VALUES ('Misc Payee', 'misc payee') RETURNING id`, + ); + rec.misc = rows[0].id; +} + +/** Create an accounts row with explicit net-worth attributes. */ +async function addAccount(name, { type = 'checking', inNetWorth = true, currency = 'EUR' } = {}) { + const { rows } = await getTestPool().query( + `INSERT INTO accounts (name, display_name, type, in_net_worth, currency) + VALUES ($1, $1, $2::account_type, $3, $4) RETURNING id`, + [name, type, inNetWorth, currency], + ); + return rows[0].id; +} + +async function insertTxn({ + date, + amount, + currency = 'EUR', + bank = 'MAIN BANK', + balance = null, + categoryId = null, + isActive = true, + isTransfer = false, +}) { + if (bank) { + await getTestPool().query( + `INSERT INTO accounts (name, display_name) VALUES ($1, $1) + ON CONFLICT (lower(btrim(name))) DO NOTHING`, + [bank], + ); + } + const { rows } = await getTestPool().query( + `INSERT INTO transactions (date, amount, currency, recipient_id, category_id, bank_account, balance, is_active, is_transfer) + VALUES ($1::date, $2, $3, $4, $5, $6, $7, $8, $9) RETURNING id`, + [date, amount, currency, rec.misc, categoryId, bank, balance, isActive, isTransfer], + ); + return rows[0].id; +} + +async function insertSnapshot(date, value, currency = 'EUR') { + await getTestPool().query( + `INSERT INTO portfolio_performance_snapshots (snapshot_date, value, currency) + VALUES ($1::date, $2, $3)`, + [date, value, currency], + ); +} + +async function insertPlanned({ + date, + amount, + currency = 'EUR', + categoryId = null, + recurring = false, + pattern = null, + isActive = true, + isExecuted = false, +}) { + const { rows } = await getTestPool().query( + `INSERT INTO planned_transactions + (planned_date, amount, currency, recipient_id, category_id, is_recurring, recurrence_pattern, is_active, is_executed) + VALUES ($1::date, $2, $3, $4, $5, $6, $7, $8, $9) RETURNING id`, + [date, amount, currency, rec.misc, categoryId, recurring, pattern, isActive, isExecuted], + ); + return rows[0].id; +} + +describe.skipIf(!hasTestDatabase())('repositories/infoRepository barrel (real DB)', () => { + beforeAll(async () => { + expect( + process.env.DATABASE_URL, + 'DATABASE_URL must equal TEST_DATABASE_URL for this suite (see scripts/with-test-db.sh)', + ).toBe(process.env.TEST_DATABASE_URL); + // DB suites share one database across parallel vitest workers — serialize. + await acquireDbSuiteLock(); + }, 180_000); + + afterEach(async () => { + const pool = getTestPool(); + await pool.query('DELETE FROM planned_transactions'); + await pool.query('DELETE FROM transactions'); + await pool.query('DELETE FROM portfolio_performance_snapshots'); + await pool.query('DELETE FROM accounts'); + await pool.query('DELETE FROM recipients'); + await pool.query('DELETE FROM categories'); + await pool.query('DELETE FROM exchange_rates'); + await pool.query(`DELETE FROM user_settings WHERE key = 'includeTransfers'`); + for (const bag of [cat, rec]) for (const k of Object.keys(bag)) delete bag[k]; + clearMemoryCache(); + clearMvCache(); + }); + + afterAll(async () => { + await releaseDbSuiteLock(); + await closeTestPool(); + await closePool(); + }); + + // ─────────────────────────────────────────────────────────────────────────── + // Barrel assembly + // ─────────────────────────────────────────────────────────────────────────── + it('assembles every sub-repository method onto one object, all callable against the DB', async () => { + // The exact assembled surface. Note what is NOT here: the module's own + // header comment (infoRepository.js:9-10) advertises `getStatistics` and + // `getTransactionSummary` on infoRepositoryStatistics, and neither exists — + // that sub-module exports getCategoryBreakdown/getBanks/getTransactionCount/ + // getCategoryPivot only. Pinning the key set keeps the drift visible. + const expected = [ + 'getAverageVsCurrentSpending', + 'getBankBalances', + 'getBanks', + 'getCashflowComparison', + 'getCashflowForecastData', + 'getCashflowForecastDataByCategory', + 'getCashflowForecastDataRolling', + 'getCategoryBreakdown', + 'getCategoryPivot', + // Not a sub-repository query: the ADR-083 toggle read, exposed on the + // barrel because the forecast cache key needs it before deciding whether + // to call any of the queries around it. + 'getIncludeTransfers', + 'getMonthlyFinancialSummary', + 'getNetWorthFromSnapshots', + 'getPlannedExpensesNextMonth', + 'getRecipientByYear', + 'getRecipientInsights', + 'getRecipientPivot', + 'getTransactionCount', + ]; + expect(Object.keys(infoRepository).sort()).toEqual(expected); + for (const name of expected) { + expect(typeof infoRepository[name], `${name} missing from the barrel`).toBe('function'); + } + // Spread order matters: statisticsRepository and the sub-repositories must + // not shadow one another. Smoke-call the ones with dedicated DB suites + // through the barrel to prove the wiring, not the behaviour. + await seedBase(); + expect(await infoRepository.getBanks()).toEqual([]); + expect(await infoRepository.getTransactionCount()).toBe(0); + expect(await infoRepository.getBankBalances()).toMatchObject({ accounts: [], total_net_position: 0 }); + expect((await infoRepository.getMonthlyFinancialSummary([])).months).toHaveLength(6); + expect(await infoRepository.getRecipientInsights('EUR')).toEqual({ topMerchants: [], monthOverMonth: [] }); + }); + + // ─────────────────────────────────────────────────────────────────────────── + // getNetWorthFromSnapshots + // ─────────────────────────────────────────────────────────────────────────── + describe('getNetWorthFromSnapshots', () => { + it('returns the empty shape when there is no source record at all', async () => { + expect(await infoRepository.getNetWorthFromSnapshots()).toEqual({ + current: { liquid: 0, liabilities: 0, investments: 0, netWorth: 0 }, + monthlyChange: 0, + monthlyChangePercent: 0, + snapshots: [], + }); + }); + + it('walks stamped balances daily, forward-fills investments and splits liabilities out of liquid', async () => { + await seedBase(); + const today = TODAY(); + const d2 = addDaysYmd(today, -2); + const d1 = addDaysYmd(today, -1); + await addAccount('KBC'); + await addAccount('MORTGAGE', { type: 'liability' }); + + await insertTxn({ date: d2, amount: '-10.00', bank: 'KBC', balance: '4500.00' }); + await insertTxn({ date: today, amount: '-10.00', bank: 'KBC', balance: '5000.00' }); + await insertTxn({ date: d2, amount: '-1.00', bank: 'MORTGAGE', balance: '-300.00' }); + // Snapshots exist only on two of the three days — the missing day must + // carry the previous value forward, not collapse to liquid-only. + await insertSnapshot(d2, '3235'); + await insertSnapshot(today, '4470'); + + const r = await infoRepository.getNetWorthFromSnapshots(); + expect(r.snapshots.map((s) => s.date)).toEqual([d2, d1, today]); + expect(r.snapshots[0]).toMatchObject({ liquid: 4500, liabilities: -300, investments: 3235, netWorth: 7435 }); + expect(r.snapshots[1]).toMatchObject({ liquid: 4500, liabilities: -300, investments: 3235 }); // forward-filled + expect(r.current).toEqual({ liquid: 5000, liabilities: -300, investments: 4470, netWorth: 9170 }); + }); + + it('resolves EVERY point with the anchor+delta definition, current and historical alike', async () => { + await seedBase(); + const today = TODAY(); + const d1 = addDaysYmd(today, -1); + await addAccount('KBC'); + await addAccount('CASH'); // manual-only: never stamped + + await insertTxn({ date: d1, amount: '-10.00', bank: 'KBC', balance: '5000.00' }); + await insertTxn({ date: today, amount: '150.00', bank: 'KBC' }); // unstamped, after the anchor + await insertTxn({ date: d1, amount: '200.00', bank: 'CASH' }); // never stamped at all + + const r = await infoRepository.getNetWorthFromSnapshots(); + // History on d1: KBC's stamp 5000 + CASH's unstamped 200 — the manual-only + // account is no longer invisible until the final point. + expect(r.snapshots[0]).toMatchObject({ date: d1, liquid: 5200 }); + // Current: 5000 + 150 (post-anchor) + 200 (manual-only account) = 5350, + // i.e. exactly the 150 that actually posted today above the d1 point. + expect(r.current.liquid).toBe(5350); + expect(r.current.netWorth).toBe(5350); + expect(r.snapshots[r.snapshots.length - 1].liquid).toBe(5350); + }); + + it('falls back to the cumulative transaction flow when nothing is stamped anywhere', async () => { + await seedBase(); + const today = TODAY(); + const d1 = addDaysYmd(today, -1); + // No accounts row and no balance stamp → the stamped walk is empty. + await insertTxn({ date: d1, amount: '1200.00', bank: null }); + await insertTxn({ date: today, amount: '300.00', bank: null }); + + const r = await infoRepository.getNetWorthFromSnapshots(); + expect(r.snapshots.map((s) => [s.date, s.liquid])).toEqual([ + [d1, 1200], + [today, 1500], // cumulative, not per-day + ]); + expect(r.current.liquid).toBe(1500); + }); + + it('overlays the live portfolio value on the latest point only', async () => { + await seedBase(); + const today = TODAY(); + const d1 = addDaysYmd(today, -1); + await addAccount('KBC'); + await insertTxn({ date: d1, amount: '-1.00', bank: 'KBC', balance: '1000.00' }); + await insertSnapshot(d1, '500'); + await insertSnapshot(today, '600'); + + const r = await infoRepository.getNetWorthFromSnapshots('EUR', { liveInvestments: 5123.45 }); + expect(r.current.investments).toBe(5123.45); + expect(r.current.netWorth).toBe(6123.45); + expect(r.snapshots[0].investments).toBe(500); // earlier points untouched + expect(r.snapshots[r.snapshots.length - 1].investments).toBe(5123.45); + }); + + it('measures monthlyChange against the last point before the current month', async () => { + await seedBase(); + const today = TODAY(); + const monthStart = firstOfMonthYmd(today); + const dayBefore = addDaysYmd(monthStart, -1); + await addAccount('KBC'); + await insertTxn({ date: dayBefore, amount: '-1.00', bank: 'KBC', balance: '1000.00' }); + await insertTxn({ date: today, amount: '-1.00', bank: 'KBC', balance: '1100.00' }); + + const r = await infoRepository.getNetWorthFromSnapshots(); + expect(r.current.netWorth).toBe(1100); + expect(r.monthlyChange).toBe(100); + expect(r.monthlyChangePercent).toBe(10); + }); + + it('excludes tracking-only accounts from both the history walk and the current point', async () => { + await seedBase(); + const today = TODAY(); + await addAccount('KBC'); + await addAccount('TRACKING', { inNetWorth: false }); + await insertTxn({ date: today, amount: '-1.00', bank: 'KBC', balance: '1000.00' }); + await insertTxn({ date: today, amount: '-1.00', bank: 'TRACKING', balance: '9999.00' }); + + const r = await infoRepository.getNetWorthFromSnapshots(); + expect(r.current.liquid).toBe(1000); + }); + + // The transaction-flow fallback fires whenever the balance walk returns no + // rows — which is not only the un-migrated ledger it was written for, but + // ALSO a ledger where every account with activity is in_net_worth=false. + // It used to sum ALL active transactions with no account / in_net_worth + // predicate, so that ledger reported the tracking-only accounts' running + // total as net worth (measured: liquid −143.25 where 0 is correct). + it('reports 0 for a ledger whose only active accounts are tracking-only', async () => { + await seedBase(); + const today = TODAY(); + const d1 = addDaysYmd(today, -1); + await addAccount('TRACKING', { inNetWorth: false }); + // Unstamped on purpose: the walk is empty either way (no in-net-worth + // account owns a row), so the fallback is what answers. + await insertTxn({ date: d1, amount: '-133.25', bank: 'TRACKING' }); + await insertTxn({ date: today, amount: '-10.00', bank: 'TRACKING' }); + + const r = await infoRepository.getNetWorthFromSnapshots(); + expect(r.current).toEqual({ liquid: 0, liabilities: 0, investments: 0, netWorth: 0 }); + expect(r.snapshots.map((s) => s.liquid)).toEqual([0, 0]); + }); + + // The other half of the same predicate: the fallback must keep counting + // rows that are NOT positively attributed to a tracking-only account. A + // bare inner join to `accounts` would have dropped exactly these — the + // un-migrated ledger the fallback exists to serve. + it('still sums un-migrated rows whose bank_account has no accounts row', async () => { + await seedBase(); + const today = TODAY(); + const d1 = addDaysYmd(today, -1); + // Written with no bank label (account_id NULL), then relabelled: the + // sync trigger only ever resolves an UPDATE against an EXISTING account, + // so the row keeps account_id NULL behind a bank_account string with no + // accounts row — the shape a pre-accounts ledger carries. + await insertTxn({ date: d1, amount: '1200.00', bank: null }); + await insertTxn({ date: today, amount: '300.00', bank: null }); + await getTestPool().query(`UPDATE transactions SET bank_account = 'LEGACY BANK'`); + expect( + (await getTestPool().query(`SELECT count(*)::int AS n FROM transactions WHERE account_id IS NULL`)).rows[0].n, + ).toBe(2); + expect((await getTestPool().query('SELECT count(*)::int AS n FROM accounts')).rows[0].n).toBe(0); + + const r = await infoRepository.getNetWorthFromSnapshots(); + expect(r.snapshots.map((s) => [s.date, s.liquid])).toEqual([[d1, 1200], [today, 1500]]); + expect(r.current.liquid).toBe(1500); + }); + + it('drops only the tracking-attributed rows when un-migrated rows sit alongside them', async () => { + await seedBase(); + const today = TODAY(); + await addAccount('TRACKING', { inNetWorth: false }); + await insertTxn({ date: today, amount: '-133.25', bank: 'TRACKING' }); + await insertTxn({ date: today, amount: '1000.00', bank: null }); // unattributed + + const r = await infoRepository.getNetWorthFromSnapshots(); + expect(r.current.liquid).toBe(1000); // not 866.75 + }); + }); + + // ─────────────────────────────────────────────────────────────────────────── + // getPlannedExpensesNextMonth + // ─────────────────────────────────────────────────────────────────────────── + describe('getPlannedExpensesNextMonth', () => { + /** First / last day of the month after the app-timezone current month. */ + const nextMonthStart = () => firstOfMonthYmd(TODAY(), 1); + const monthAfterStart = () => firstOfMonthYmd(TODAY(), 2); + + it('reports the next-month window and only the rows inside it', async () => { + await seedBase(); + const start = nextMonthStart(); + const after = monthAfterStart(); + const lastDay = addDaysYmd(after, -1); + const midNext = addDaysYmd(start, 4); + + await insertPlanned({ date: midNext, amount: '-20.00', categoryId: cat.Rent }); + await insertPlanned({ date: midNext, amount: '50.00' }); + await insertPlanned({ date: TODAY(), amount: '-999.00' }); // current month → out + await insertPlanned({ date: after, amount: '-888.00' }); // month after → out + + const r = await infoRepository.getPlannedExpensesNextMonth('EUR'); + expect(r.period_start).toBe(start); + expect(r.period_end).toBe(lastDay); + // The month/year fields must name the same month period_start does. + expect(`${r.year}-${String(r.month).padStart(2, '0')}`).toBe(start.slice(0, 7)); + expect(r.daily_data).toHaveLength(1); + expect(r.daily_data[0]).toMatchObject({ date: midNext, total_income: 50, total_expenses: -20 }); + // Same-day rows come back in whatever order `ORDER BY pt.planned_date` + // leaves them (no tiebreaker in the SQL), so assert set-wise. + expect(r.daily_data[0].transactions).toHaveLength(2); + expect(r.daily_data[0].transactions).toEqual( + expect.arrayContaining([ + expect.objectContaining({ amount: -20, category_name: 'Home:Rent', is_recurring: false }), + expect.objectContaining({ amount: 50, category_name: null }), + ]), + ); + expect(r.summary).toMatchObject({ + total_income: 50, total_expenses: -20, net_amount: 30, transaction_count: 2, + }); + }); + + it('ignores executed and inactive planned rows', async () => { + await seedBase(); + const midNext = addDaysYmd(nextMonthStart(), 4); + await insertPlanned({ date: midNext, amount: '-10.00' }); + await insertPlanned({ date: midNext, amount: '-4000.00', isExecuted: true }); + await insertPlanned({ date: midNext, amount: '-5000.00', isActive: false }); + + const r = await infoRepository.getPlannedExpensesNextMonth('EUR'); + expect(r.summary).toMatchObject({ total_expenses: -10, transaction_count: 1 }); + }); + + it('expands a recurring row dated in the CURRENT month into its next-month occurrences', async () => { + await seedBase(); + const start = nextMonthStart(); + // Weekly, anchored on the 1st of the CURRENT month: the row itself never + // falls inside next month, so only expansion can surface it. + await insertPlanned({ + date: firstOfMonthYmd(TODAY()), + amount: '-50.00', + recurring: true, + pattern: 'weekly', + }); + + const r = await infoRepository.getPlannedExpensesNextMonth('EUR'); + expect(r.summary.transaction_count).toBeGreaterThanOrEqual(4); + expect(r.summary.total_expenses).toBe(-50 * r.summary.transaction_count); + for (const d of r.daily_data) { + expect(d.date >= start && d.date < monthAfterStart()).toBe(true); + // Every occurrence is 7 days after the previous one. + expect((new Date(`${d.date}T00:00:00Z`) - new Date(`${firstOfMonthYmd(TODAY())}T00:00:00Z`)) % (7 * 86_400_000)).toBe(0); + } + }); + + it('fast-forwards a stale daily recurrence instead of dropping it (120-hop cap)', async () => { + await seedBase(); + const start = nextMonthStart(); + const after = monthAfterStart(); + // Anchored ~8 months back: a flat 120-hop walk would run out before + // reaching next month and the row would silently vanish. + await insertPlanned({ + date: firstOfMonthYmd(TODAY(), -8), + amount: '-5.00', + recurring: true, + pattern: 'daily', + }); + + const r = await infoRepository.getPlannedExpensesNextMonth('EUR'); + const daysInNextMonth = Math.round( + (new Date(`${after}T00:00:00Z`) - new Date(`${start}T00:00:00Z`)) / 86_400_000, + ); + expect(r.summary.transaction_count).toBe(daysInNextMonth); + expect(r.daily_data[0].date).toBe(start); + expect(r.daily_data[r.daily_data.length - 1].date).toBe(addDaysYmd(after, -1)); + }); + }); + + // ─────────────────────────────────────────────────────────────────────────── + // getAverageVsCurrentSpending + // ─────────────────────────────────────────────────────────────────────────── + describe('getAverageVsCurrentSpending', () => { + /** Day N of the month `monthsBack` months before the current one. */ + const monthDay = (monthsBack, day) => + `date_trunc('month', CURRENT_DATE) - interval '${monthsBack} months' + interval '${day - 1} days'`; + + async function insertTxnAt(dateExpr, amount, opts = {}) { + const { rows } = await getTestPool().query(`SELECT to_char((${dateExpr})::date, 'YYYY-MM-DD') AS d`); + return insertTxn({ date: rows[0].d, amount, ...opts }); + } + + it('divides the 6-month spend by calendar days and projects the current month on elapsed days', async () => { + await seedBase(); + await insertTxnAt(monthDay(1, 10), '-10.00'); + await insertTxnAt(monthDay(2, 10), '-20.00'); + await insertTxnAt(monthDay(2, 10), '5.00'); // income never counts as spending + await insertTxnAt(monthDay(9, 10), '-9999.00'); // outside the 6-month window + await insertTxn({ date: TODAY(), amount: '-5.00' }); + await insertTxn({ date: TODAY(), amount: '1.00' }); + + // Observed window: the ledger's first in-window row is in month -2 (the + // month -9 row is outside the 6-month floor), so the denominators are + // months -2 and -1 — in months for the monthly figure, in calendar days + // for the daily one. + const { rows } = await getTestPool().query(` + SELECT (date_trunc('month', CURRENT_DATE)::date + - (date_trunc('month', CURRENT_DATE) - interval '2 months')::date) AS n + `); + const calendarDays = Number(rows[0].n); + + const r = await infoRepository.getAverageVsCurrentSpending('EUR'); + expect(r.past_6_months.months_counted).toBe(2); + expect(r.past_6_months.avg_monthly_spending).toBe(15); // 30 over 2 elapsed months + // Rounded to cents by the repository, so compare the rounded figure. + expect(r.past_6_months.avg_daily_spending).toBe(Math.round((30 / calendarDays) * 100) / 100); + expect(r.current_month.total_spending).toBe(5); + expect(r.current_month.daily_data).toEqual([{ date: TODAY(), spending: 5, income: 1 }]); + const projected = Math.round(((5 / r.current_month.days_elapsed) * r.current_month.days_in_month) * 100) / 100; + expect(r.comparison.projected_monthly_total).toBe(projected); + expect(r.comparison.variance).toBe(Math.round((projected - 15) * 100) / 100); + }); + + it('excludes internal transfers by default and includes them when the setting is on', async () => { + await seedBase(); + await insertTxn({ date: TODAY(), amount: '-100.00' }); + await insertTxn({ date: TODAY(), amount: '-900.00', isTransfer: true }); + + expect((await infoRepository.getAverageVsCurrentSpending('EUR')).current_month.total_spending).toBe(100); + + await getTestPool().query( + `INSERT INTO user_settings (key, value) VALUES ('includeTransfers', 'true'::jsonb)`, + ); + expect((await infoRepository.getAverageVsCurrentSpending('EUR')).current_month.total_spending).toBe(1000); + }); + + it('ignores inactive rows on both windows', async () => { + await seedBase(); + await insertTxnAt(monthDay(1, 10), '-40.00', { isActive: false }); + await insertTxn({ date: TODAY(), amount: '-30.00', isActive: false }); + + const r = await infoRepository.getAverageVsCurrentSpending('EUR'); + expect(r.past_6_months.months_counted).toBe(1); // the "no data" divisor floor + expect(r.past_6_months.avg_monthly_spending).toBe(0); + expect(r.current_month.total_spending).toBe(0); + expect(r.comparison.pace).toBeNull(); + }); + }); + + // ─────────────────────────────────────────────────────────────────────────── + // PINNED real-DB behaviours (see the suite report — do NOT "fix" here) + // ─────────────────────────────────────────────────────────────────────────── + describe('pinned discrepancies (current real behaviour)', () => { + const monthDay = (monthsBack, day) => + `date_trunc('month', CURRENT_DATE) - interval '${monthsBack} months' + interval '${day - 1} days'`; + + async function insertTxnAt(dateExpr, amount, opts = {}) { + const { rows } = await getTestPool().query(`SELECT to_char((${dateExpr})::date, 'YYYY-MM-DD') AS d`); + return insertTxn({ date: rows[0].d, amount, ...opts }); + } + + // Was PIN 1 — "average monthly spending over the past 6 months" divided by + // the number of months that contain ANY transaction. A `monthlySpending` + // key was seeded for every month with a row (before the `eur < 0` test), + // then the total was divided by `monthKeys.length`: a gap month vanished + // from the divisor (inflating the average) and a month whose only rows were + // INCOME entered it (diluting the average with zero extra spend). The + // `avg_daily_spending` sibling on the same object divided by calendar days + // instead, so the two disagreed by construction. + // + // Both now divide the same numerator by the same observed window — the span + // from the ledger's first in-window transaction through the last complete + // month (infoRepositoryAverageVsCurrent.countObservedMonths, the same + // definition infoRepositoryForecast uses), one expressed in months and one + // in days. + it('divides by elapsed months since the ledger started, counting empty months as zeros', async () => { + await seedBase(); + await insertTxnAt(monthDay(1, 10), '-240.00'); + + // A one-month-old ledger is not charged five phantom zeros: 240, not 40. + const one = await infoRepository.getAverageVsCurrentSpending('EUR'); + expect(one.past_6_months.months_counted).toBe(1); + expect(one.past_6_months.avg_monthly_spending).toBe(240); + + // Pushing history back to month -3 makes months -3 and -2 real zeros: + // the divisor is 3, not the 2 months that happen to carry rows. + await insertTxnAt(monthDay(3, 10), '-60.00'); + const three = await infoRepository.getAverageVsCurrentSpending('EUR'); + expect(three.past_6_months.months_counted).toBe(3); + expect(three.past_6_months.avg_monthly_spending).toBe(100); // 300 / 3, not 300 / 2 = 150 + }); + + it('is unmoved by an income-only month INSIDE an established span (an income row that is the ledger-oldest row still extends the span, matching the forecast doctrine)', async () => { + await seedBase(); + await insertTxnAt(monthDay(3, 10), '-240.00'); + + const before = await infoRepository.getAverageVsCurrentSpending('EUR'); + expect(before.past_6_months.months_counted).toBe(3); + expect(before.past_6_months.avg_monthly_spending).toBe(80); // 240 over months -3..-1 + + // A month holding ONLY income adds no spend and no new observed month + // (month -1 was already inside the span), so the average cannot move. + await insertTxnAt(monthDay(1, 10), '1000.00'); + const after = await infoRepository.getAverageVsCurrentSpending('EUR'); + expect(after.past_6_months.months_counted).toBe(3); + expect(after.past_6_months.avg_monthly_spending).toBe(80); // was 240 → 120 before the fix + }); + + it('keeps avg_monthly_spending and avg_daily_spending on one denominator', async () => { + await seedBase(); + await insertTxnAt(monthDay(2, 10), '-240.00'); + + const { rows } = await getTestPool().query(` + SELECT (date_trunc('month', CURRENT_DATE)::date + - (date_trunc('month', CURRENT_DATE) - interval '2 months')::date) AS n + `); + const observedDays = Number(rows[0].n); + + const r = await infoRepository.getAverageVsCurrentSpending('EUR'); + expect(r.past_6_months.months_counted).toBe(2); + expect(r.past_6_months.avg_monthly_spending).toBe(120); + // Same 240 spread over the same window, in days rather than months — + // previously 240/183 next to 240/2, a factor-3 disagreement. + expect(r.past_6_months.avg_daily_spending).toBe(Math.round((240 / observedDays) * 100) / 100); + }); + + // Was PIN 2 — net worth's stamp-based history walk hid never-stamped + // accounts from every point except the last one, where the current-point + // override added them back in one go: a vertical step no transaction + // explains, which the monthly-change figure then reported as a real gain. + // The transaction-flow fallback only rescued "nothing stamped anywhere"; + // mixing one stamped account with one manual account defeated it. The walk + // now resolves each day with the same unstamped-tolerant anchor+delta + // definition the current point uses. + it('carries manual-only accounts through the whole net-worth history, not just the last point', async () => { + await seedBase(); + const today = TODAY(); + const d1 = addDaysYmd(today, -1); + await addAccount('KBC'); + await addAccount('CASH'); + await insertTxn({ date: d1, amount: '-1.00', bank: 'KBC', balance: '1000.00' }); + await insertTxn({ date: d1, amount: '200.00', bank: 'CASH' }); // never stamped + + const r = await infoRepository.getNetWorthFromSnapshots(); + expect(r.snapshots.map((s) => [s.date, s.liquid])).toEqual([ + [d1, 1200], // CASH's 200 counts from the day it posts… + [today, 1200], // …so the last point is flat, not a step + ]); + expect(r.current.liquid).toBe(1200); + // Nothing moved after d1, so nothing is reported as movement. + expect(r.monthlyChange).toBe(0); + }); + + it('keeps headline == last chart point for a foreign-currency account on a moving curve', async () => { + // Same invariant as the banks widget's: the current point converts at + // today's rate and the walk at each day's, so a rate that moved after the + // last statement must not open a gap between the headline and the chart. + await seedBase(); + const today = TODAY(); + const d30 = addDaysYmd(today, -30); + await addAccount('WISE USD', { currency: 'USD' }); + for (const [date, rate, isLatest] of [[d30, '0.5', false], [today, '0.9', true]]) { + await getTestPool().query( + `INSERT INTO exchange_rates (currency_code, rate_date, rate_to_eur, is_latest) + VALUES ('USD', $1::date, $2, $3)`, + [date, rate, isLatest], + ); + } + await insertTxn({ date: d30, amount: '-10.00', currency: 'USD', bank: 'WISE USD', balance: '1000.00' }); + + const r = await infoRepository.getNetWorthFromSnapshots('EUR'); + expect(r.current.liquid).toBe(900); // 1000 USD × today's 0.9 + const last = r.snapshots[r.snapshots.length - 1]; + expect(last.liquid).toBe(r.current.liquid); + // The curve moves the series, so the equality above is not a flat-curve + // coincidence: the statement day itself is still valued at 0.5. + expect(r.snapshots[0]).toMatchObject({ date: d30, liquid: 500 }); + }); + + it('keeps headline == last chart point on an all-manual net-worth ledger', async () => { + await seedBase(); + const today = TODAY(); + const d2 = addDaysYmd(today, -2); + const d1 = addDaysYmd(today, -1); + await addAccount('CASH'); + await addAccount('WALLET'); + await insertTxn({ date: d2, amount: '200.00', bank: 'CASH' }); + await insertTxn({ date: d1, amount: '-50.00', bank: 'CASH' }); + await insertTxn({ date: d1, amount: '30.00', bank: 'WALLET' }); + + const r = await infoRepository.getNetWorthFromSnapshots(); + expect(r.snapshots.map((s) => [s.date, s.liquid])).toEqual([ + [d2, 200], + [d1, 180], + [today, 180], + ]); + expect(r.current.liquid).toBe(180); + expect(r.snapshots[r.snapshots.length - 1].liquid).toBe(r.current.liquid); + }); + }); +}); diff --git a/apps/node-backend/tests/infoRepository.test.js b/apps/node-backend/tests/infoRepository.test.js index 0709e385..f7686e3b 100644 --- a/apps/node-backend/tests/infoRepository.test.js +++ b/apps/node-backend/tests/infoRepository.test.js @@ -75,7 +75,7 @@ describe('InfoRepository', () => { ], }; } - if (sql.includes('account_list') && sql.includes('LEFT JOIN LATERAL')) { + if (sql.includes('balance_series')) { return { rows: [ { day: '2026-02-01', bank_account: 'Chase', balance: '4500', currency: 'EUR' }, @@ -118,7 +118,7 @@ describe('InfoRepository', () => { ], }; } - if (sql.includes('account_list') && sql.includes('LEFT JOIN LATERAL')) { + if (sql.includes('balance_series')) { return { rows: [ { day: '2026-02-01', bank_account: 'Chase', balance: '4500', currency: 'EUR' }, @@ -152,7 +152,7 @@ describe('InfoRepository', () => { if (sql.includes('portfolio_performance_snapshots') && sql.includes('value AS investments')) { return { rows: [{ day: todayKey, investments: '4470' }] }; } - if (sql.includes('account_list') && sql.includes('LEFT JOIN LATERAL')) { + if (sql.includes('balance_series')) { return { rows: [{ day: todayKey, bank_account: 'Chase', balance: '5000', currency: 'EUR' }] }; } return { rows: [] }; @@ -191,7 +191,7 @@ describe('InfoRepository', () => { if (sql.includes('portfolio_performance_snapshots') && sql.includes('value AS investments')) { return { rows: [] }; } - if (sql.includes('account_list') && sql.includes('LEFT JOIN LATERAL')) { + if (sql.includes('balance_series')) { return { rows: [{ day: todayKey, bank_account: 'Main', balance: '1234.56', currency: 'EUR' }] }; } return { rows: [] }; @@ -214,7 +214,7 @@ describe('InfoRepository', () => { if (sql.includes('portfolio_performance_snapshots') && sql.includes('value AS investments')) { return { rows: [] }; } - if (sql.includes('account_list') && sql.includes('LEFT JOIN LATERAL')) { + if (sql.includes('balance_series')) { return { rows: [ { day: firstDayKey, bank_account: 'A', balance: '1000', currency: 'EUR' }, @@ -234,15 +234,13 @@ describe('InfoRepository', () => { it('should fall back to cumulative transaction flow when no bank balances are available', async () => { const todayKey = todayAppDateString(); query.mockImplementation(async (sql) => { - if (sql.includes('SELECT 1 FROM')) return { rows: [] }; - if (sql.includes('first_data_date')) return { rows: [{ first_data_date: '2026-02-01' }] }; - if (sql.includes('portfolio_performance_snapshots') && sql.includes('value AS investments')) { - return { rows: [{ day: todayKey, investments: '500' }] }; - } - if (sql.includes('account_list') && sql.includes('LEFT JOIN LATERAL')) { - return { rows: [] }; - } - if (sql.includes('COALESCE(SUM(t.amount), 0) AS amount')) { + // Discriminated by 'tx_cumulative', the fallback's own CTE: + // `COALESCE(SUM(t.amount), 0) AS amount` also appears inside the + // shared balance-series CTEs, so it cross-matches the history walk. + // Like the 'WITH anchor' branch below, this MUST come before the + // generic 'SELECT 1 FROM' one — the fallback's tracking-only exclusion + // contains `NOT EXISTS (SELECT 1 FROM accounts a …)`. + if (sql.includes('tx_cumulative')) { return { rows: [ { day: '2026-02-01', currency: 'EUR', value: '1200' }, @@ -250,11 +248,26 @@ describe('InfoRepository', () => { ], }; } + if (sql.includes('SELECT 1 FROM')) return { rows: [] }; + if (sql.includes('first_data_date')) return { rows: [{ first_data_date: '2026-02-01' }] }; + if (sql.includes('portfolio_performance_snapshots') && sql.includes('value AS investments')) { + return { rows: [{ day: todayKey, investments: '500' }] }; + } + if (sql.includes('balance_series')) { + return { rows: [] }; + } return { rows: [] }; }); const result = await infoRepository.getNetWorthFromSnapshots(); + // The fallback query must carry the tracking-only exclusion: a ledger + // whose only active accounts are in_net_worth=false must not have THEIR + // running total reported as net worth. + const fallbackSql = query.mock.calls.map(([s]) => s) + .find((s) => typeof s === 'string' && s.includes('tx_cumulative')); + expect(fallbackSql).toContain('a.in_net_worth = false'); + expect(result.current.liquid).toBe(1500); expect(result.current.investments).toBe(500); expect(result.current.netWorth).toBe(2000); @@ -274,7 +287,7 @@ describe('InfoRepository', () => { if (sql.includes('portfolio_performance_snapshots') && sql.includes('value AS investments')) { return { rows: [] }; } - if (sql.includes('account_list') && sql.includes('LEFT JOIN LATERAL')) { + if (sql.includes('balance_series')) { return { rows: [{ day: todayKey, bank_account: 'FallbackAccount', balance: '99', currency: 'EUR' }] }; } return { rows: [] }; @@ -289,10 +302,11 @@ describe('InfoRepository', () => { // ── WP-A1: current point uses the unified computed-balance definition ── // // The unified current-balance query is discriminated by 'WITH anchor' - // (the shared lateral's CTE) — the stamped history walk has 'account_list' - // instead, so the two mocks can't cross-match. The 'WITH anchor' branch - // MUST come before the generic 'SELECT 1 FROM' branch: the lateral's - // no-stamp fallback contains `NOT EXISTS (SELECT 1 FROM anchor)`. + // (the shared lateral's CTE) — the daily history walk ends in a + // 'balance_series' CTE instead, so the two mocks can't cross-match. The + // 'WITH anchor' branch MUST come before the generic 'SELECT 1 FROM' + // branch: the lateral's no-stamp fallback contains + // `NOT EXISTS (SELECT 1 FROM anchor)`. const mockUnifiedNetWorth = ({ firstDataDate, investmentsRows, walkRows, currentRows }) => { query.mockImplementation(async (sql) => { if (sql.includes('WITH anchor')) return { rows: currentRows }; @@ -301,7 +315,7 @@ describe('InfoRepository', () => { if (sql.includes('portfolio_performance_snapshots') && sql.includes('value AS investments')) { return { rows: investmentsRows }; } - if (sql.includes('account_list') && sql.includes('LEFT JOIN LATERAL')) { + if (sql.includes('balance_series')) { return { rows: walkRows }; } return { rows: [] }; @@ -421,7 +435,7 @@ describe('InfoRepository', () => { if (sql.includes('portfolio_performance_snapshots') && sql.includes('value AS investments')) { return { rows: [] }; } - if (sql.includes('account_list') && sql.includes('LEFT JOIN LATERAL')) { + if (sql.includes('balance_series')) { return { rows: [{ day: todayKey, bank_account: 'Main', balance: '1000', currency: 'EUR' }] }; } return { rows: [] }; @@ -729,6 +743,50 @@ describe('InfoRepository', () => { } }); + // The month window used to be built as LOCAL Dates (`new Date(today.year, + // today.month, 1)`) but read back for the `month`/`year` fields with UTC + // getters. On a server east of UTC — including the default + // APP_TIMEZONE=Europe/Brussels — local midnight of the 1st is still the + // PREVIOUS month in UTC, so the response named month 6 while period_start + // (local extraction) named 2026-07-01, and at a year boundary it named the + // previous YEAR too. Neither CI nor the container can see it: both run + // TZ=UTC, where local and UTC getters agree. + it.each([ + ['mid-year', '2026-06-15T12:00:00Z', 7, 2026, '2026-07-01', '2026-07-31'], + ['across the year boundary', '2026-12-15T12:00:00Z', 1, 2027, '2027-01-01', '2027-01-31'], + ])('names the next month consistently with period_start east of UTC (%s)', async ( + _label, systemTime, month, year, periodStart, periodEnd, + ) => { + convertRowsToEur.mockImplementation(async (rows) => rows.map((row) => ({ + ...row, + amount_eur: Number(row.amount ?? 0), + }))); + + const prevTz = process.env.TZ; + process.env.TZ = 'Asia/Tokyo'; // UTC+9 — local midnight is the day before in UTC + vi.useFakeTimers(); + vi.setSystemTime(new Date(systemTime)); + try { + query.mockResolvedValueOnce({ rows: [] }); + + const result = await infoRepository.getPlannedExpensesNextMonth('EUR'); + + expect(result.period_start).toBe(periodStart); + expect(result.period_end).toBe(periodEnd); + // The headline month/year must name the SAME month period_start does. + expect({ month: result.month, year: result.year }).toEqual({ month, year }); + // …and the SQL window must be bound to that same month. + expect(query.mock.calls[0][1]).toEqual([periodStart, expect.any(String)]); + } finally { + vi.useRealTimers(); + // Restore, not reassign: `process.env.TZ = undefined` writes the literal + // string "undefined", leaving Intl on an invalid zone for the rest of + // the file. + if (prevTz === undefined) delete process.env.TZ; + else process.env.TZ = prevTz; + } + }); + it('expands a recurring planned tx into its next-month occurrences', async () => { convertRowsToEur.mockImplementation(async (rows) => rows.map((row) => ({ ...row, @@ -789,6 +847,63 @@ describe('InfoRepository', () => { } }); + // Recurrence expansion used to walk `Date`s: the pg DATE arrives at + // SERVER-LOCAL midnight, occurrences were rendered with toAppDateString + // (APP_TIMEZONE) and the fast-forward compared that instant against + // Date.UTC() of the window start. East of APP_TIMEZONE (TZ=Asia/Tokyo with + // the default Europe/Brussels) local midnight is still the previous day in + // the app zone, so every occurrence shifted a day back: the weekly row + // below started on 2026-06-30 instead of 2026-07-01 and every July + // occurrence landed 6 days off, and the monthly row shifted onto May 31 and + // then clamped to the 30th for the rest of the year. The expansion is pure + // calendar-string math now, so the occurrence days are identical on every + // host — hence the same expectations for each TZ below. + it.each([ + // [host TZ, cadence, [stored y, m, d], expected occurrences in July 2026] + ...['UTC', 'Asia/Tokyo', 'America/Los_Angeles'].flatMap((tz) => [ + [tz, 'weekly', [2026, 6, 3], ['2026-07-01', '2026-07-08', '2026-07-15', '2026-07-22', '2026-07-29']], + [tz, 'monthly', [2026, 6, 1], ['2026-07-01']], + // Month-end clamping compounds across sequential hops and must stay + // that way: Jan 31 → Feb 28 → Mar 28 → … → Jul 28 (never back to the + // 31st). + [tz, 'monthly', [2026, 1, 31], ['2026-07-28']], + ]), + ])('expands a recurring row on the same days regardless of host TZ (%s, %s)', async ( + tz, pattern, [y, m, d], expected, + ) => { + convertRowsToEur.mockImplementation(async (rows) => rows.map((row) => ({ + ...row, + amount_eur: Number(row.amount ?? 0), + }))); + + const prevTz = process.env.TZ; + process.env.TZ = tz; + vi.useFakeTimers(); + vi.setSystemTime(new Date('2026-06-15T12:00:00Z')); // → July 2026 window + try { + // Built with LOCAL components on purpose: that is exactly how pg hands + // back a DATE column (a Date at server-local midnight). + query.mockResolvedValueOnce({ + rows: [ + { id: 1, planned_date: new Date(y, m - 1, d), amount: '-50', currency: 'EUR', recipient_name: 'Gym', category_name: null, is_recurring: true, recurrence_pattern: pattern }, + ], + }); + + const result = await infoRepository.getPlannedExpensesNextMonth('EUR'); + + expect(result.daily_data.map((day) => day.date)).toEqual(expected); + expect(result.summary.transaction_count).toBe(expected.length); + expect(result.summary.total_expenses).toBe(-50 * expected.length); + } finally { + vi.useRealTimers(); + // Restore, not reassign: `process.env.TZ = undefined` writes the + // literal string "undefined", leaving Intl on an invalid zone for the + // rest of the file. + if (prevTz === undefined) delete process.env.TZ; + else process.env.TZ = prevTz; + } + }); + it('should compute average-vs-current spending projections on calendar-day denominators', async () => { convertRowsToEur.mockImplementation(async (rows) => rows.map((row) => ({ ...row, @@ -807,7 +922,10 @@ describe('InfoRepository', () => { { amount: '-5', currency: 'EUR', date: '2026-03-03' }, { amount: '1', currency: 'EUR', date: '2026-03-03' }, ], - }); + }) + // Third call: the unfiltered ledger-start probe. A first row back at the + // window floor means the full 6 observed months apply. + .mockResolvedValueOnce({ rows: [{ first_date: '2025-09-04' }] }); // Pin the clock so the calendar-day denominators are deterministic. vi.useFakeTimers(); @@ -815,9 +933,12 @@ describe('InfoRepository', () => { try { const result = await infoRepository.getAverageVsCurrentSpending('EUR'); - // 6-month window = 2025-09-01 → 2026-03-01 = 181 calendar days; spend 30. - // Old (buggy) code divided by 2 transaction days → 15. + // Observed window = 2025-09-01 → 2026-03-01 = 6 months = 181 calendar + // days; spend 30. Old (buggy) code divided by 2 transaction days → 15. expect(result.past_6_months.avg_daily_spending).toBeCloseTo(30 / 181, 2); + // The monthly sibling divides the SAME 30 by the SAME window in months. + expect(result.past_6_months.months_counted).toBe(6); + expect(result.past_6_months.avg_monthly_spending).toBeCloseTo(30 / 6, 2); expect(result.current_month.total_spending).toBe(5); // daysElapsed is the calendar day (15), not the 1 day that had a txn. expect(result.current_month.days_elapsed).toBe(15); @@ -844,11 +965,20 @@ describe('InfoRepository', () => { await infoRepository.getAverageVsCurrentSpending('EUR'); - for (const [sql] of query.mock.calls) { + // Calls 0/1 are the two aggregate windows; call 2 is the ledger-start + // probe, which is a MIN(date) scalar and has nothing to group by. + for (const [sql] of query.mock.calls.slice(0, 2)) { expect(sql).toContain('GROUP BY t.date, t.currency'); expect(sql).toContain('SUM(t.amount)'); expect(sql).not.toMatch(/LIMIT/i); } + // The average denominator must be probed UNFILTERED: an exclusion or the + // ADR-083 transfer predicate emptying the oldest months would silently + // re-base the divisor (see infoRepositoryForecast's sqlLedgerStart). + const probeSql = query.mock.calls[2][0]; + expect(probeSql).toContain('MIN(t.date)'); + expect(probeSql).not.toMatch(/is_transfer/); + expect(probeSql).not.toMatch(/LIMIT/i); }); }); }); diff --git a/apps/node-backend/tests/infoRepositoryRecipients.db.test.js b/apps/node-backend/tests/infoRepositoryRecipients.db.test.js new file mode 100644 index 00000000..a71cea06 --- /dev/null +++ b/apps/node-backend/tests/infoRepositoryRecipients.db.test.js @@ -0,0 +1,506 @@ +/** + * Real-Postgres tests for infoRepositoryRecipients — `getRecipientInsights`, + * `getRecipientByYear` and `getRecipientPivot`. + * + * DB-backed complement to infoRepositoryRecipients.test.js (which stays: it runs + * without a DB). That mock suite stubs `convertRowsToEur` and hand-feeds the + * per-currency rows it wants, so it exercises the JS reducers and the SQL text + * only. Every behaviour below instead comes out of the real schema: the alias + * roll-up `COALESCE(pr.id, r.id)` against real `recipients.primary_recipient_id` + * links, the expense/active/transfer predicates, the month-over-month + * like-for-like window arithmetic done in Postgres, `EXTRACT(YEAR …)` bucketing, + * `TO_CHAR` period keys, the alias-member narrowing round-trip, and FX resolved + * from seeded `exchange_rates` rows. + * + * `getRecipientInsights`'s MoM window is anchored on `CURRENT_DATE`, so those + * fixtures are dated by SQL expressions relative to it; the by-year and pivot + * queries have no date window, so those use literal calendar dates. + */ + +import { afterAll, afterEach, beforeAll, describe, expect, it } from 'vitest'; +import { + acquireDbSuiteLock, + closeTestPool, + getTestPool, + hasTestDatabase, + releaseDbSuiteLock, +} from './setup/db.js'; +import { recipientInsightsRepository } from '../src/repositories/infoRepositoryRecipients.js'; +import { clearMvCache } from '../src/repositories/infoRepositoryHelpers.js'; +import { clearMemoryCache } from '../src/services/currency/currencyConversionService.js'; +import { closePool } from '../src/database/connection.js'; + +const cat = {}; +const rec = {}; + +/** + * Categories plus the alias topology: + * Delhaize — PRIMARY + * Delhaize Wilrijk — ALIAS of Delhaize (spend rolls up to the primary) + * Colruyt — plain recipient, default category Food + * Electrabel — plain recipient, default category Bills + */ +async function seedBase() { + const pool = getTestPool(); + for (const [key, [general, detail]] of Object.entries({ + Food: ['Food', 'Groceries'], + Bills: ['Bills', 'Utilities'], + })) { + const { rows } = await pool.query( + 'INSERT INTO categories (general, detail) VALUES ($1, $2) RETURNING id', + [general, detail], + ); + cat[key] = rows[0].id; + } + const addRecipient = async (name, { defaultCategoryId = null, primaryId = null } = {}) => { + const { rows } = await pool.query( + `INSERT INTO recipients (name, normalized_name, default_category_id, primary_recipient_id) + VALUES ($1, $2, $3, $4) RETURNING id`, + [name, name.toLowerCase(), defaultCategoryId, primaryId], + ); + return rows[0].id; + }; + rec.delhaize = await addRecipient('Delhaize'); + rec.delhaizeAlias = await addRecipient('Delhaize Wilrijk', { primaryId: rec.delhaize }); + rec.colruyt = await addRecipient('Colruyt', { defaultCategoryId: cat.Food }); + rec.electrabel = await addRecipient('Electrabel', { defaultCategoryId: cat.Bills }); +} + +async function ensureAccount(name) { + await getTestPool().query( + `INSERT INTO accounts (name, display_name) VALUES ($1, $1) + ON CONFLICT (lower(btrim(name))) DO NOTHING`, + [name], + ); +} + +/** + * Insert one transaction. Exactly one of `date` (literal 'YYYY-MM-DD') or + * `dateExpr` (a SQL expression, for the CURRENT_DATE-anchored MoM window) is + * given. `recipientId: null` writes a recipient-less row on purpose. + */ +async function insertTxn({ + date = null, + dateExpr = null, + amount, + currency = 'EUR', + recipientId, + categoryId = null, + bank = 'MAIN BANK', + isActive = true, + isTransfer = false, +}) { + if (bank) await ensureAccount(bank); + const dateSql = dateExpr ? `(${dateExpr})::date` : '$6::date'; + const params = [amount, currency, recipientId, categoryId, bank, ...(dateExpr ? [] : [date]), isActive, isTransfer]; + const n = dateExpr ? 6 : 7; + const { rows } = await getTestPool().query( + `INSERT INTO transactions (amount, currency, recipient_id, category_id, bank_account, date, is_active, is_transfer) + VALUES ($1, $2, $3, $4, $5, ${dateSql}, $${n}, $${n + 1}) RETURNING id`, + params, + ); + return rows[0].id; +} + +/** Seed one exchange_rates row so conversion resolves from the DB, never the network. */ +async function insertRate(code, date, rate, isLatest = false) { + await getTestPool().query( + `INSERT INTO exchange_rates (currency_code, rate_date, rate_to_eur, is_latest) + VALUES ($1, $2::date, $3, $4)`, + [code, date, rate, isLatest], + ); +} + +const merchant = (result, name) => result.topMerchants.find((m) => m.name === name); +const mom = (result, name) => result.monthOverMonth.find((m) => m.name === name); + +describe.skipIf(!hasTestDatabase())('repositories/infoRepositoryRecipients (real DB)', () => { + beforeAll(async () => { + expect( + process.env.DATABASE_URL, + 'DATABASE_URL must equal TEST_DATABASE_URL for this suite (see scripts/with-test-db.sh)', + ).toBe(process.env.TEST_DATABASE_URL); + // DB suites share one database across parallel vitest workers — serialize. + await acquireDbSuiteLock(); + }, 180_000); + + afterEach(async () => { + const pool = getTestPool(); + await pool.query('DELETE FROM transactions'); + await pool.query('DELETE FROM accounts'); + await pool.query('DELETE FROM recipients'); + await pool.query('DELETE FROM categories'); + await pool.query('DELETE FROM exchange_rates'); + await pool.query(`DELETE FROM user_settings WHERE key = 'includeTransfers'`); + for (const bag of [cat, rec]) for (const k of Object.keys(bag)) delete bag[k]; + clearMemoryCache(); + clearMvCache(); + }); + + afterAll(async () => { + await releaseDbSuiteLock(); + await closeTestPool(); + await closePool(); + }); + + // ─────────────────────────────────────────────────────────────────────────── + // getRecipientInsights — top merchants + // ─────────────────────────────────────────────────────────────────────────── + describe('getRecipientInsights / topMerchants', () => { + it('rolls alias spend up to the primary and reports count, average and seen-dates', async () => { + await seedBase(); + await insertTxn({ date: '2024-01-15', amount: '-100.00', recipientId: rec.delhaize }); + await insertTxn({ date: '2024-03-20', amount: '-50.00', recipientId: rec.delhaizeAlias }); + await insertTxn({ date: '2024-02-01', amount: '-40.00', recipientId: rec.colruyt }); + + const r = await recipientInsightsRepository.getRecipientInsights('EUR'); + expect(r.topMerchants.map((m) => m.name)).toEqual(['Delhaize', 'Colruyt']); // desc by spend + expect(merchant(r, 'Delhaize')).toMatchObject({ + recipientId: rec.delhaize, // the alias id never surfaces + totalSpend: 150, + transactionCount: 2, + avgAmount: 75, + firstSeen: '2024-01-15', + lastSeen: '2024-03-20', // widened by the alias row + }); + expect(typeof merchant(r, 'Delhaize').firstSeen).toBe('string'); // wire dates, not pg Dates + }); + + it('counts only active negative-amount rows and never internal transfers', async () => { + await seedBase(); + await insertTxn({ date: '2024-01-10', amount: '-30.00', recipientId: rec.colruyt }); + await insertTxn({ date: '2024-01-11', amount: '1000.00', recipientId: rec.colruyt }); // income + await insertTxn({ date: '2024-01-12', amount: '-999.00', recipientId: rec.colruyt, isActive: false }); + await insertTxn({ date: '2024-01-13', amount: '-500.00', recipientId: rec.colruyt, isTransfer: true }); + // ADR-083's includeTransfers toggle governs cash-flow aggregates, NOT this + // merchant lens — turning it on must not pull the transfer leg in. + await getTestPool().query( + `INSERT INTO user_settings (key, value) VALUES ('includeTransfers', 'true'::jsonb)`, + ); + + const r = await recipientInsightsRepository.getRecipientInsights('EUR'); + expect(merchant(r, 'Colruyt')).toMatchObject({ totalSpend: 30, transactionCount: 1 }); + }); + + it('merges a recipient spanning two currencies into one row', async () => { + await seedBase(); + await insertRate('USD', '2024-01-15', '0.5', true); + await insertTxn({ date: '2024-01-15', amount: '-100.00', recipientId: rec.colruyt }); + await insertTxn({ date: '2024-01-15', amount: '-100.00', currency: 'USD', recipientId: rec.colruyt }); + + const r = await recipientInsightsRepository.getRecipientInsights('EUR'); + expect(r.topMerchants).toHaveLength(1); + expect(merchant(r, 'Colruyt')).toMatchObject({ totalSpend: 150, transactionCount: 2, avgAmount: 75 }); + }); + + it('applies 3-level category exclusions and alias-aware recipient exclusions', async () => { + await seedBase(); + // Categorised only via its recipient's default (2nd level). + await insertTxn({ date: '2024-01-10', amount: '-40.00', recipientId: rec.colruyt }); + await insertTxn({ date: '2024-01-10', amount: '-60.00', recipientId: rec.electrabel }); + await insertTxn({ date: '2024-01-10', amount: '-10.00', recipientId: rec.delhaizeAlias }); + + const exclFood = await recipientInsightsRepository.getRecipientInsights('EUR', { + excludedCategoryIds: [cat.Food], + }); + expect(exclFood.topMerchants.map((m) => m.name).sort()).toEqual(['Delhaize', 'Electrabel']); + + // Excluding the PRIMARY also removes rows recorded under its alias. + const exclPrimary = await recipientInsightsRepository.getRecipientInsights('EUR', { + excludedRecipientIds: [rec.delhaize], + }); + expect(exclPrimary.topMerchants.map((m) => m.name).sort()).toEqual(['Colruyt', 'Electrabel']); + }); + + // Was PIN 1 — the three recipient surfaces disagreed on FX. getRecipientByYear + // and getRecipientPivot both pass `{ useHistoricalRatesByDate: true, + // dateField: 'date' }`, converting each row at the rate for ITS date; + // top merchants passed no options at all and converted every row at the + // CURRENT `is_latest` rate, so one USD purchase read 90 in "Top merchants" + // and 25 in "Top recipients by year" / the recipient pivot on the same page. + // The top-merchants query now carries `t.date` in its GROUP BY so it can be + // converted per date like its siblings. + it('converts top merchants at each row date historical rate, agreeing with by-year and pivot', async () => { + await seedBase(); + await insertRate('USD', '2024-06-01', '0.25'); + await insertRate('USD', '2026-01-01', '0.90', true); + await insertTxn({ date: '2024-06-01', amount: '-100.00', currency: 'USD', recipientId: rec.colruyt }); + + const insights = await recipientInsightsRepository.getRecipientInsights('EUR'); + const byYear = await recipientInsightsRepository.getRecipientByYear('EUR'); + const pivot = await recipientInsightsRepository.getRecipientPivot([], 'EUR'); + + expect(merchant(insights, 'Colruyt').totalSpend).toBe(25); // 2024-06-01 rate, not the latest 0.90 + expect(byYear.recipientsByYear['2024'][0].totalSpend).toBe(25); + expect(pivot.recipientPivot['2024-06'][0].total).toBe(25); + }); + + // Per-date conversion must not disturb the per-recipient reduction: two + // purchases on DIFFERENT dates at different rates still collapse into one + // merchant row whose count, average and first/last-seen bounds span both. + it('sums a recipient across dates at each date own rate, keeping count and seen-dates', async () => { + await seedBase(); + await insertRate('USD', '2024-06-01', '0.25'); + await insertRate('USD', '2024-09-01', '0.50'); + await insertRate('USD', '2026-01-01', '0.90', true); + await insertTxn({ date: '2024-06-01', amount: '-100.00', currency: 'USD', recipientId: rec.colruyt }); + await insertTxn({ date: '2024-09-01', amount: '-100.00', currency: 'USD', recipientId: rec.colruyt }); + + const r = await recipientInsightsRepository.getRecipientInsights('EUR'); + expect(r.topMerchants).toHaveLength(1); + expect(merchant(r, 'Colruyt')).toMatchObject({ + totalSpend: 75, // 25 + 50, not 2 x 90 at the latest rate + transactionCount: 2, + avgAmount: 37.5, + firstSeen: '2024-06-01', + lastSeen: '2024-09-01', + }); + }); + }); + + // ─────────────────────────────────────────────────────────────────────────── + // getRecipientInsights — month over month + // ─────────────────────────────────────────────────────────────────────────── + describe('getRecipientInsights / monthOverMonth', () => { + it('reports only recipients present in BOTH periods, with the percentage change', async () => { + await seedBase(); + await insertTxn({ dateExpr: `date_trunc('month', CURRENT_DATE)`, amount: '-60.00', recipientId: rec.colruyt }); + await insertTxn({ dateExpr: `date_trunc('month', CURRENT_DATE) - interval '1 month'`, amount: '-40.00', recipientId: rec.colruyt }); + // Current month only → no comparison possible. + await insertTxn({ dateExpr: `date_trunc('month', CURRENT_DATE)`, amount: '-30.00', recipientId: rec.electrabel }); + // Previous month only. + await insertTxn({ dateExpr: `date_trunc('month', CURRENT_DATE) - interval '1 month'`, amount: '-25.00', recipientId: rec.delhaize }); + + const r = await recipientInsightsRepository.getRecipientInsights('EUR'); + expect(r.monthOverMonth).toEqual([ + { + recipientId: rec.colruyt, + name: 'Colruyt', + currentSpend: 60, + previousSpend: 40, + changePercent: 50, + }, + ]); + }); + + it('caps the previous month at the same day-of-month as today (like-for-like)', async () => { + await seedBase(); + const pool = getTestPool(); + const { rows } = await pool.query(` + SELECT to_char( + (date_trunc('month', CURRENT_DATE) - INTERVAL '1 month')::date + + (CURRENT_DATE - DATE_TRUNC('month', CURRENT_DATE)::date), 'YYYY-MM-DD') AS cap, + ((date_trunc('month', CURRENT_DATE) - INTERVAL '1 month')::date + + (CURRENT_DATE - DATE_TRUNC('month', CURRENT_DATE)::date) + 1) + < date_trunc('month', CURRENT_DATE)::date AS beyond_cap_still_prev_month + `); + const { cap, beyond_cap_still_prev_month: beyondIsTestable } = rows[0]; + + await insertTxn({ dateExpr: `date_trunc('month', CURRENT_DATE)`, amount: '-10.00', recipientId: rec.colruyt }); + await insertTxn({ date: cap, amount: '-7.00', recipientId: rec.colruyt }); // exactly on the cap → in + if (beyondIsTestable) { + // Only assertable when cap+1 is still inside the previous month: for a + // short previous month and a late day-of-month the cap lands past its + // end, and then the whole previous month is in-window by construction. + await insertTxn({ + dateExpr: `(date_trunc('month', CURRENT_DATE) - INTERVAL '1 month')::date + + (CURRENT_DATE - DATE_TRUNC('month', CURRENT_DATE)::date) + 1`, + amount: '-1000.00', + recipientId: rec.colruyt, + }); + } + + const r = await recipientInsightsRepository.getRecipientInsights('EUR'); + expect(mom(r, 'Colruyt')).toMatchObject({ currentSpend: 10, previousSpend: 7 }); + }); + + it('ignores months outside the two-period window entirely', async () => { + await seedBase(); + await insertTxn({ dateExpr: `date_trunc('month', CURRENT_DATE)`, amount: '-10.00', recipientId: rec.colruyt }); + await insertTxn({ dateExpr: `date_trunc('month', CURRENT_DATE) - interval '1 month'`, amount: '-10.00', recipientId: rec.colruyt }); + await insertTxn({ dateExpr: `date_trunc('month', CURRENT_DATE) - interval '2 months'`, amount: '-9999.00', recipientId: rec.colruyt }); + + const r = await recipientInsightsRepository.getRecipientInsights('EUR'); + expect(mom(r, 'Colruyt')).toMatchObject({ currentSpend: 10, previousSpend: 10, changePercent: 0 }); + }); + + // Same class as the top-merchants FX fix, one function away: `momConverted` + // passed NO options, so both compared months were converted at the CURRENT + // `is_latest` rate. A rate move between them was therefore invisible — an + // identical 100 USD in each month read 90/90 (changePercent 0) instead of + // 50/25 — and the EUR figures contradicted the historical-rate + // top-merchants / by-year / pivot surfaces on the same page. The query now + // carries `t.date` in its GROUP BY so it can be converted per date too. + it('converts each month at ITS OWN date rate, agreeing with top merchants', async () => { + await seedBase(); + const { rows } = await getTestPool().query(` + SELECT to_char(date_trunc('month', CURRENT_DATE), 'YYYY-MM-DD') AS cur_day, + to_char(date_trunc('month', CURRENT_DATE) - INTERVAL '1 month', 'YYYY-MM-DD') AS prev_day + `); + const { cur_day: curDay, prev_day: prevDay } = rows[0]; + + await insertRate('USD', prevDay, '0.25'); + await insertRate('USD', curDay, '0.50'); + // The latest rate is what the old code used for BOTH months. + await insertRate('USD', '2000-01-01', '0.90', true); + + await insertTxn({ date: prevDay, amount: '-100.00', currency: 'USD', recipientId: rec.colruyt }); + await insertTxn({ date: curDay, amount: '-100.00', currency: 'USD', recipientId: rec.colruyt }); + + const r = await recipientInsightsRepository.getRecipientInsights('EUR'); + expect(mom(r, 'Colruyt')).toMatchObject({ + currentSpend: 50, // curDay rate 0.50, not the latest 0.90 + previousSpend: 25, // prevDay rate 0.25 — the rate move is now visible + changePercent: 100, // was 0: identical USD spend at one shared rate + }); + // Cross-surface: MoM's two months must add up to what top merchants + // reports for the same recipient over the same two transactions. + expect(merchant(r, 'Colruyt').totalSpend).toBe(75); + }); + }); + + // ─────────────────────────────────────────────────────────────────────────── + // getRecipientByYear + // ─────────────────────────────────────────────────────────────────────────── + describe('getRecipientByYear', () => { + it('buckets by calendar year, rolls aliases up and sorts each year by spend', async () => { + await seedBase(); + await insertTxn({ date: '2024-12-31', amount: '-100.00', recipientId: rec.delhaize }); + await insertTxn({ date: '2024-06-01', amount: '-25.00', recipientId: rec.delhaizeAlias }); + await insertTxn({ date: '2024-06-01', amount: '-50.00', recipientId: rec.colruyt }); + await insertTxn({ date: '2025-01-01', amount: '-10.00', recipientId: rec.delhaize }); + + const r = await recipientInsightsRepository.getRecipientByYear('EUR'); + expect(r.recipientsByYear['2024']).toEqual([ + { recipientId: rec.delhaize, name: 'Delhaize', totalSpend: 125, transactionCount: 2 }, + { recipientId: rec.colruyt, name: 'Colruyt', totalSpend: 50, transactionCount: 1 }, + ]); + expect(r.recipientsByYear['2025']).toEqual([ + { recipientId: rec.delhaize, name: 'Delhaize', totalSpend: 10, transactionCount: 1 }, + ]); + // 2024-12-31 lands in 2024, not 2025 — the DATE round-trip does not shift. + expect(Object.keys(r.recipientsByYear).sort()).toEqual(['2024', '2025']); + }); + + it('converts at each row date historical rate, not the latest one', async () => { + await seedBase(); + await insertRate('USD', '2024-06-01', '0.25'); + await insertRate('USD', '2026-01-01', '0.90', true); + await insertTxn({ date: '2024-06-01', amount: '-100.00', currency: 'USD', recipientId: rec.colruyt }); + + const r = await recipientInsightsRepository.getRecipientByYear('EUR'); + expect(r.recipientsByYear['2024'][0].totalSpend).toBe(25); + }); + + it('applies category exclusions and drops invalid recipient ids', async () => { + await seedBase(); + await insertTxn({ date: '2024-01-10', amount: '-40.00', recipientId: rec.colruyt }); // Food via default + await insertTxn({ date: '2024-01-10', amount: '-60.00', recipientId: rec.electrabel }); // Bills via default + + const excl = await recipientInsightsRepository.getRecipientByYear( + 'EUR', + [0, -1, 1.5, 'evil', rec.electrabel], + [cat.Food], + ); + expect(excl.recipientsByYear['2024']).toBeUndefined(); // both rows filtered out + }); + }); + + // ─────────────────────────────────────────────────────────────────────────── + // getRecipientPivot + // ─────────────────────────────────────────────────────────────────────────── + describe('getRecipientPivot', () => { + it('buckets monthly by default and yearly on request, ascending by total', async () => { + await seedBase(); + await insertTxn({ date: '2024-03-05', amount: '-100.00', recipientId: rec.delhaize }); + await insertTxn({ date: '2024-03-06', amount: '-25.00', recipientId: rec.delhaizeAlias }); + await insertTxn({ date: '2024-03-07', amount: '-50.00', recipientId: rec.colruyt }); + await insertTxn({ date: '2024-04-01', amount: '-10.00', recipientId: rec.colruyt }); + + const monthly = await recipientInsightsRepository.getRecipientPivot([], 'EUR'); + expect(Object.keys(monthly.recipientPivot).sort()).toEqual(['2024-03', '2024-04']); + expect(monthly.recipientPivot['2024-03']).toEqual([ + { recipientId: rec.colruyt, name: 'Colruyt', total: 50, transactionCount: 1 }, + { recipientId: rec.delhaize, name: 'Delhaize', total: 125, transactionCount: 2 }, + ]); + + const yearly = await recipientInsightsRepository.getRecipientPivot([], 'EUR', { bucket: 'yearly' }); + expect(Object.keys(yearly.recipientPivot)).toEqual(['2024']); + expect(yearly.recipientPivot['2024']).toEqual([ + { recipientId: rec.colruyt, name: 'Colruyt', total: 60, transactionCount: 2 }, + { recipientId: rec.delhaize, name: 'Delhaize', total: 125, transactionCount: 2 }, + ]); + }); + + it('narrows to the selected recipients, pulling in their alias members', async () => { + await seedBase(); + await insertTxn({ date: '2024-03-05', amount: '-100.00', recipientId: rec.delhaize }); + await insertTxn({ date: '2024-03-06', amount: '-25.00', recipientId: rec.delhaizeAlias }); + await insertTxn({ date: '2024-03-07', amount: '-50.00', recipientId: rec.colruyt }); + + // Selecting the PRIMARY resolves its alias members through the real + // recipients table, so the alias row is included. + const primaryOnly = await recipientInsightsRepository.getRecipientPivot([], 'EUR', { + recipientIds: [rec.delhaize], + }); + expect(primaryOnly.recipientPivot['2024-03']).toEqual([ + { recipientId: rec.delhaize, name: 'Delhaize', total: 125, transactionCount: 2 }, + ]); + + // An id nobody matches short-circuits to an empty pivot. + const none = await recipientInsightsRepository.getRecipientPivot([], 'EUR', { + recipientIds: [2147483646], + }); + expect(none).toEqual({ recipientPivot: {} }); + }); + + it('applies inclusive start/end date filters and recipient exclusions together', async () => { + await seedBase(); + await insertTxn({ date: '2024-02-28', amount: '-1.00', recipientId: rec.colruyt }); + await insertTxn({ date: '2024-03-01', amount: '-10.00', recipientId: rec.colruyt }); + await insertTxn({ date: '2024-03-31', amount: '-20.00', recipientId: rec.colruyt }); + await insertTxn({ date: '2024-04-01', amount: '-2.00', recipientId: rec.colruyt }); + await insertTxn({ date: '2024-03-15', amount: '-99.00', recipientId: rec.delhaizeAlias }); + + const r = await recipientInsightsRepository.getRecipientPivot([rec.delhaize], 'EUR', { + startDate: '2024-03-01', + endDate: '2024-03-31', + }); + expect(r.recipientPivot).toEqual({ + '2024-03': [{ recipientId: rec.colruyt, name: 'Colruyt', total: 30, transactionCount: 2 }], + }); + }); + }); + + // ─────────────────────────────────────────────────────────────────────────── + // PINNED real-DB behaviours (see the suite report — do NOT "fix" here) + // ─────────────────────────────────────────────────────────────────────────── + describe('pinned discrepancies (current real behaviour)', () => { + + // PIN 2 — selecting an ALIAS in the pivot returns a series labelled with + // the PRIMARY but holding only the alias's spend. + // getRecipientPivot's member resolution (infoRepositoryRecipients.js:293-296) + // expands DOWNWARD only — `id = ANY($1) OR primary_recipient_id = ANY($1)` + // — so picking an alias id yields just that alias, while the pivot's + // GROUP BY `COALESCE(pr.id, r.id)` (309-310, 324) then relabels the result + // with the PRIMARY's id and name. The saved-chart user who picks + // "Delhaize Wilrijk" gets a series called "Delhaize" showing 25 of the + // primary's 125 — a plausible-looking number under the wrong label. The + // mock suite stubbed the member-resolution query's rows, so the mismatch + // between what it resolves and what the GROUP BY reports never appeared. + it('PIN: picking an alias in the pivot yields the PRIMARY label with only the alias spend', async () => { + await seedBase(); + await insertTxn({ date: '2024-03-05', amount: '-100.00', recipientId: rec.delhaize }); + await insertTxn({ date: '2024-03-06', amount: '-25.00', recipientId: rec.delhaizeAlias }); + + const aliasPick = await recipientInsightsRepository.getRecipientPivot([], 'EUR', { + recipientIds: [rec.delhaizeAlias], + }); + expect(aliasPick.recipientPivot['2024-03']).toEqual([ + // Labelled as the primary, but only the alias's 25 — not 125, and not + // "Delhaize Wilrijk". + { recipientId: rec.delhaize, name: 'Delhaize', total: 25, transactionCount: 1 }, + ]); + }); + }); +}); diff --git a/apps/node-backend/tests/infoRepositoryRecipients.test.js b/apps/node-backend/tests/infoRepositoryRecipients.test.js index a1eb2ac0..bd502c6e 100644 --- a/apps/node-backend/tests/infoRepositoryRecipients.test.js +++ b/apps/node-backend/tests/infoRepositoryRecipients.test.js @@ -88,6 +88,14 @@ describe('recipientInsightsRepository.getRecipientInsights', () => { // day-of-month so a partial current month isn't compared to a full prior one. const momSql = query.mock.calls[1][0]; expect(momSql).toContain("(CURRENT_DATE - DATE_TRUNC('month', CURRENT_DATE)::date)"); + // MoM converts at HISTORICAL per-date rates like every other recipient + // surface — the no-DB guard for the fix that ended latest-rate conversion + // here (the DB pin proves the numbers; this pins the contract). + expect(momSql).toContain('t.date'); + expect(convertRowsToEur).toHaveBeenNthCalledWith(2, expect.any(Array), 'EUR', { + useHistoricalRatesByDate: true, + dateField: 'date', + }); }); it('caps month-over-month list to top-10 by current spend', async () => { diff --git a/apps/node-backend/tests/main.test.js b/apps/node-backend/tests/main.test.js index 0b7c4b6d..a4c774b6 100644 --- a/apps/node-backend/tests/main.test.js +++ b/apps/node-backend/tests/main.test.js @@ -7,7 +7,6 @@ */ import { describe, it, expect, vi, beforeEach } from 'vitest'; import { mockLogger } from './helpers/mockLogger.js'; -import { createMockRouter } from './helpers/routeHarness.js'; // ── Mock dependencies before importing main ────────────────── @@ -36,13 +35,6 @@ vi.mock('../src/middleware/rateLimiter.js', () => ({ importRateLimiter: (req, res, next) => next(), })); -// Mock all route modules to avoid pulling in full dependency trees -const { router: mockRouter } = createMockRouter(); -vi.mock('express', async () => { - const actual = await vi.importActual('express'); - return { ...actual, default: actual.default, Router: () => mockRouter }; -}); - import { getSettings } from '../src/config/config.js'; import { checkConnection } from '../src/database/connection.js'; import { logger } from '../src/config/logger.js'; diff --git a/apps/node-backend/tests/materializedViewService.test.js b/apps/node-backend/tests/materializedViewService.test.js index c12139a8..e5f9ae49 100644 --- a/apps/node-backend/tests/materializedViewService.test.js +++ b/apps/node-backend/tests/materializedViewService.test.js @@ -170,6 +170,29 @@ describe('materializedViewService', () => { expect(logger.info).toHaveBeenCalledWith('Materialized views ready'); }); + // Guard for the canonical 3-level effective-category resolution (own → + // recipient default → PRIMARY recipient's default). Both category-bearing MVs + // used to resolve only two levels, so a row recorded under an alias whose + // PRIMARY carries the default category was counted as UNCATEGORISED here + // while the transactions list showed it categorised. Changing either + // definition needs a DROP migration (0084 / 0085) — see the DB-backed + // assertion in tests/aliasCategoryResolution.db.test.js. + it('resolves the effective category over three levels in both category-bearing views', async () => { + const { createMaterializedViews, query } = await loadMaterializedViewService(); + query.mockResolvedValue({ rows: [] }); + + await createMaterializedViews(); + + const sqlCalls = query.mock.calls.map(([sql]) => sql); + for (const view of ['mv_monthly_summary', 'mv_category_totals']) { + const ddl = sqlCalls.find((sql) => sql.includes(`CREATE MATERIALIZED VIEW IF NOT EXISTS ${view}`)); + expect(ddl, `${view} DDL`).toContain('LEFT JOIN recipients pr ON r.primary_recipient_id = pr.id'); + expect(ddl, `${view} category join`).toContain( + 'COALESCE(t.category_id, r.default_category_id, pr.default_category_id) = c.id', + ); + } + }); + it('no longer creates the dead mv_bank_balances view (dropped — zero readers)', async () => { const { createMaterializedViews, query } = await loadMaterializedViewService(); query.mockResolvedValue({ rows: [] }); diff --git a/apps/node-backend/tests/rateLimiter.test.js b/apps/node-backend/tests/rateLimiter.test.js index dab5d08c..f337d27f 100644 --- a/apps/node-backend/tests/rateLimiter.test.js +++ b/apps/node-backend/tests/rateLimiter.test.js @@ -1,5 +1,4 @@ import { afterEach, describe, expect, it, vi } from 'vitest'; -import { createMockResponse } from './helpers/routeHarness.js'; vi.mock('../src/config/config.js', () => ({ getSettings: () => ({ isDevelopment: () => false }), @@ -25,7 +24,13 @@ function createRequest({ ip, remoteAddress } = {}) { return req; } -const createResponse = () => createMockResponse({ setHeader: vi.fn() }); +// rateLimiter is unit-tested as a plain middleware function (req, res, next) +// — a minimal res stub is enough; there is no router/HTTP path to exercise. +function createResponse() { + const res = { json: vi.fn(), status: vi.fn(), setHeader: vi.fn() }; + res.status.mockReturnValue(res); + return res; +} describe('rateLimiter middleware', () => { afterEach(() => { diff --git a/apps/node-backend/tests/recurringDetectionService.test.js b/apps/node-backend/tests/recurringDetectionService.test.js index ab2531c7..8be6f5eb 100644 --- a/apps/node-backend/tests/recurringDetectionService.test.js +++ b/apps/node-backend/tests/recurringDetectionService.test.js @@ -22,7 +22,13 @@ const gymRow = (id, date, amount) => ({ bank_account: 'BE00', recipient_id: 42, recipient_name: 'Gym', + // The scan query resolves and emits `effective_category_id` (the same + // 3-level COALESCE `category_name` is resolved from) — kept equal to the + // raw `category_id` here since none of these fixtures exercise a + // recipient-default resolution; see the dedicated test below for the case + // where they diverge. category_id: 7, + effective_category_id: 7, category_name: 'HEALTH:GYM', }); @@ -48,6 +54,7 @@ describe('detectRecurringPatterns', () => { recipient_id: 42, recipient_name: 'Gym', category_id: 7, + effective_category_id: 7, category_name: 'HEALTH:GYM', }, { @@ -60,6 +67,7 @@ describe('detectRecurringPatterns', () => { recipient_id: 42, recipient_name: 'Gym', category_id: 7, + effective_category_id: 7, category_name: 'HEALTH:GYM', }, { @@ -72,6 +80,7 @@ describe('detectRecurringPatterns', () => { recipient_id: 42, recipient_name: 'Gym', category_id: 7, + effective_category_id: 7, category_name: 'HEALTH:GYM', }, ], @@ -100,6 +109,7 @@ describe('detectRecurringPatterns', () => { recipient_id: 42, recipient_name: 'Employer & Landlord', category_id: 7, + effective_category_id: 7, category_name: 'MISC:MISC', }); mockQuery @@ -147,4 +157,62 @@ describe('detectRecurringPatterns', () => { expect(second).toBe(first); expect(mockQuery.mock.calls.length).toBe(callsAfterFirst); }); + + // The scan must resolve the effective category over all THREE levels (own → + // recipient default → PRIMARY recipient's default), matching + // transactionRepository. With the former 2-level resolution a pattern whose + // transactions sit under an alias recipient — categorised only via the + // PRIMARY's default — reported categoryName: null while the transactions + // list showed it categorised. Behavioural coverage lives in + // tests/aliasCategoryResolution.db.test.js. + it('resolves the effective category over three levels in the scan query', async () => { + mockQuery.mockResolvedValueOnce({ rows: [] }); + + await detectRecurringPatterns(); + + const sql = mockQuery.mock.calls[0][0]; + expect(sql).toContain('LEFT JOIN recipients pr ON r.primary_recipient_id = pr.id'); + expect(sql).toContain( + 'LEFT JOIN categories c ON COALESCE(t.category_id, r.default_category_id, pr.default_category_id) = c.id', + ); + // The pattern's emitted categoryId must come from the same COALESCE + // categoryName is resolved from, not the raw t.category_id — otherwise a + // recipient-default-categorised pattern reports categoryId: null beside a + // non-null categoryName. + expect(sql).toContain( + 'COALESCE(t.category_id, r.default_category_id, pr.default_category_id) AS effective_category_id', + ); + }); + + // Pins the categoryId fix directly: a row whose own category_id is null but + // whose recipient/primary default resolves it (effective_category_id: 7) + // must surface as pattern.categoryId === 7, not the raw null. Pre-fix, the + // service read `category_id` off the row and this assertion failed while + // `categoryName` was already correct. + it('emits the resolved effective_category_id as categoryId, not the raw category_id', async () => { + const row = (id, date) => ({ + id, + date: new Date(`${date}T00:00:00.000Z`), + amount: '-120.00', + currency: 'EUR', + memo: `Electrabel ${id}`, + bank_account: 'BE00', + recipient_id: 99, + recipient_name: 'Electrabel Invoicing', + category_id: null, + effective_category_id: 7, + category_name: 'Bills:Utilities', + }); + mockQuery + .mockResolvedValueOnce({ + rows: [row(1, '2026-01-01'), row(2, '2026-02-01'), row(3, '2026-03-01')], + }) + .mockResolvedValueOnce({ rows: [{ exists: true }] }) + .mockResolvedValueOnce({ rows: [] }); + + const result = await detectRecurringPatterns(); + + expect(result.patterns[0].categoryId).toBe(7); + expect(result.patterns[0].categoryName).toBe('Bills:Utilities'); + }); }); diff --git a/apps/node-backend/tests/routes/accounts.test.js b/apps/node-backend/tests/routes/accounts.test.js index c63d7e08..8772ac20 100644 --- a/apps/node-backend/tests/routes/accounts.test.js +++ b/apps/node-backend/tests/routes/accounts.test.js @@ -5,16 +5,12 @@ * (net-worth + bank-balances) via invalidatePortfolioCaches, otherwise the * 5-min netWorthResponseCache keeps serving a stale net worth after an account * is renamed / toggled in_net_worth / archived / merged / reconciled, etc. + * + * Runs against the REAL router mounted on a throwaway Express app (see + * tests/helpers/routeApp.js) — validateIdParam is no longer stubbed. */ import { describe, it, expect, vi, beforeEach } from 'vitest'; -import { createMockRouter, createMockResponse } from '../helpers/routeHarness.js'; - -const { router: mockRouter, handlers: routeHandlers } = createMockRouter(); - -vi.mock('express', () => ({ - default: { Router: () => mockRouter }, - Router: () => mockRouter, -})); +import { routeAgent, okEnvelope, errEnvelope } from '../helpers/routeApp.js'; vi.mock('../../src/services/accountService.js', () => ({ default: { @@ -28,6 +24,7 @@ vi.mock('../../src/services/accountService.js', () => ({ vi.mock('../../src/services/accountMergeService.js', () => ({ mergeAccounts: vi.fn(), + previewMerge: vi.fn(), })); vi.mock('../../src/services/openingBalanceService.js', () => ({ @@ -46,69 +43,57 @@ vi.mock('../../src/services/info/cache.js', () => ({ invalidatePortfolioCaches: vi.fn(), })); -vi.mock('../../src/middleware/validation.js', () => ({ - validateIdParam: (req, res, next) => next(), -})); - import accountService from '../../src/services/accountService.js'; import { mergeAccounts } from '../../src/services/accountMergeService.js'; import { setOpeningBalance } from '../../src/services/openingBalanceService.js'; import { reconcileAccount } from '../../src/services/reconcileService.js'; import { scheduleAggregationRefresh } from '../../src/services/aggregationRefresh.js'; import { invalidatePortfolioCaches } from '../../src/services/info/cache.js'; -import { ValidationError } from '../../src/middleware/errorHandler.js'; -await import('../../src/routes/accounts.js'); + +const { default: accountsRouter } = await import('../../src/routes/accounts.js'); + +const api = routeAgent(accountsRouter, { mountPath: '/api/accounts' }); +const BASE = '/api/accounts'; describe('Account Routes — portfolio cache invalidation', () => { beforeEach(() => vi.clearAllMocks()); it('create busts the portfolio caches', async () => { accountService.create.mockResolvedValue({ id: 1, name: 'Cash' }); - const res = createMockResponse(); - await routeHandlers['post:/']({ body: { name: 'Cash' } }, res); + const res = await api.post(BASE).send({ name: 'Cash' }).expect(201); expect(invalidatePortfolioCaches).toHaveBeenCalledTimes(1); - expect(res.status).toHaveBeenCalledWith(201); + expect(res.body).toEqual(okEnvelope({ id: 1, name: 'Cash', links: [] })); }); it('update busts the portfolio caches (rename / in_net_worth / is_active / statement_balance)', async () => { accountService.update.mockResolvedValue({ id: 1, name: 'Renamed' }); - const res = createMockResponse(); - await routeHandlers['patch:/:id']({ params: { id: '1' }, body: { name: 'Renamed' } }, res); + await api.patch(`${BASE}/1`).send({ name: 'Renamed' }).expect(200); expect(invalidatePortfolioCaches).toHaveBeenCalledTimes(1); }); it('delete busts the portfolio caches and answers 204 with no body', async () => { accountService.remove.mockResolvedValue(undefined); - const res = createMockResponse(); - await routeHandlers['delete:/:id']({ params: { id: '1' } }, res); + const res = await api.delete(`${BASE}/1`).expect(204); expect(invalidatePortfolioCaches).toHaveBeenCalledTimes(1); - expect(res.status).toHaveBeenCalledWith(204); - expect(res.send).toHaveBeenCalledWith(); - expect(res.json).not.toHaveBeenCalled(); + expect(res.text).toBe(''); }); it('merge busts the portfolio caches', async () => { mergeAccounts.mockResolvedValue({ survivor_id: 1, merged: [2] }); - const res = createMockResponse(); - await routeHandlers['post:/:id/merge']({ params: { id: '1' }, body: { source_ids: [2] } }, res); + await api.post(`${BASE}/1/merge`).send({ source_ids: [2] }).expect(200); expect(invalidatePortfolioCaches).toHaveBeenCalledTimes(1); }); it('opening-balance busts the portfolio caches and still refreshes aggregations', async () => { setOpeningBalance.mockResolvedValue({ id: 1 }); - const res = createMockResponse(); - await routeHandlers['post:/:id/opening-balance']( - { params: { id: '1' }, body: { balance: 100, date: '2026-01-01' } }, - res, - ); + await api.post(`${BASE}/1/opening-balance`).send({ balance: 100, date: '2026-01-01' }).expect(200); expect(invalidatePortfolioCaches).toHaveBeenCalledTimes(1); expect(scheduleAggregationRefresh).toHaveBeenCalledTimes(1); }); it('reconcile busts the portfolio caches and still refreshes aggregations', async () => { reconcileAccount.mockResolvedValue({ id: 1 }); - const res = createMockResponse(); - await routeHandlers['post:/:id/reconcile']({ params: { id: '1' }, body: { mode: 'accept' } }, res); + await api.post(`${BASE}/1/reconcile`).send({ mode: 'accept' }).expect(200); expect(invalidatePortfolioCaches).toHaveBeenCalledTimes(1); expect(scheduleAggregationRefresh).toHaveBeenCalledTimes(1); }); @@ -124,46 +109,34 @@ describe('POST /:id/merge — source_ids are rejected, not filtered', () => { it('merges a fully-valid body unchanged (parseInt coercion preserved)', async () => { mergeAccounts.mockResolvedValue({ into: 1, merged: [2, 3] }); - const res = createMockResponse(); - await routeHandlers['post:/:id/merge']({ params: { id: '1' }, body: { source_ids: [2, '3'] } }, res); + const res = await api.post(`${BASE}/1/merge`).send({ source_ids: [2, '3'] }).expect(200); expect(mergeAccounts).toHaveBeenCalledWith(1, [2, 3]); - expect(res.json).toHaveBeenCalledWith({ - ok: true, - data: { into: 1, merged: [2, 3], links: [] }, - }); + expect(res.body).toEqual(okEnvelope({ into: 1, merged: [2, 3], links: [] })); }); it('rejects with 400 when one source id of several is not an integer, merging nothing', async () => { - const res = createMockResponse(); - await expect( - routeHandlers['post:/:id/merge']({ params: { id: '1' }, body: { source_ids: [2, 'abc', 3] } }, res), - ).rejects.toBeInstanceOf(ValidationError); + const res = await api.post(`${BASE}/1/merge`).send({ source_ids: [2, 'abc', 3] }).expect(400); + expect(res.body.error.code).toBe('VALIDATION_ERROR'); expect(mergeAccounts).not.toHaveBeenCalled(); expect(invalidatePortfolioCaches).not.toHaveBeenCalled(); }); it('names each offending entry (index and raw value) in the 400', async () => { - const res = createMockResponse(); - await expect( - routeHandlers['post:/:id/merge']({ params: { id: '1' }, body: { source_ids: [2, 'abc', null] } }, res), - ).rejects.toThrow(/source_ids\[1\] \("abc"\).*source_ids\[2\] \(null\)/s); + const res = await api.post(`${BASE}/1/merge`).send({ source_ids: [2, 'abc', null] }).expect(400); + expect(res.body.error.message).toMatch(/source_ids\[1\] \("abc"\).*source_ids\[2\] \(null\)/s); expect(mergeAccounts).not.toHaveBeenCalled(); }); it('rejects non-integer forms that parseInt cannot coerce (null, objects, empty string)', async () => { for (const bad of [null, {}, '', [], undefined]) { - const res = createMockResponse(); - await expect( - routeHandlers['post:/:id/merge']({ params: { id: '1' }, body: { source_ids: [bad] } }, res), - ).rejects.toBeInstanceOf(ValidationError); + await api.post(`${BASE}/1/merge`).send({ source_ids: [bad] }).expect(400); } expect(mergeAccounts).not.toHaveBeenCalled(); }); it('leaves the non-array / missing source_ids path to the service (empty list)', async () => { mergeAccounts.mockResolvedValue({ into: 1, merged: [] }); - const res = createMockResponse(); - await routeHandlers['post:/:id/merge']({ params: { id: '1' }, body: {} }, res); + await api.post(`${BASE}/1/merge`).send({}).expect(200); expect(mergeAccounts).toHaveBeenCalledWith(1, []); }); }); @@ -175,42 +148,44 @@ describe('Account Routes — GET / pagination is opt-in', () => { // must keep answering the complete list (and must not echo limit/offset). it('returns the full list and no limit/offset when neither param is sent', async () => { accountService.list.mockResolvedValue({ items: [{ id: 1 }, { id: 2 }], total: 2 }); - const res = createMockResponse(); - await routeHandlers['get:/']({ query: {} }, res); + const res = await api.get(BASE).expect(200); expect(accountService.list).toHaveBeenCalledWith({ active: true }); - expect(res.json).toHaveBeenCalledWith({ - ok: true, - data: { items: [{ id: 1 }, { id: 2 }], total: 2, links: [] }, - }); + expect(res.body).toEqual(okEnvelope({ items: [{ id: 1 }, { id: 2 }], total: 2, links: [] })); }); it('treats an empty limit param as absent', async () => { accountService.list.mockResolvedValue({ items: [{ id: 1 }], total: 1 }); - const res = createMockResponse(); - await routeHandlers['get:/']({ query: { limit: '' } }, res); + const res = await api.get(`${BASE}?limit=`).expect(200); expect(accountService.list).toHaveBeenCalledWith({ active: true }); - expect(res.json.mock.calls[0][0].data.limit).toBeUndefined(); + expect(res.body.data.limit).toBeUndefined(); }); it('pages and reports the full total when limit/offset are supplied', async () => { accountService.list.mockResolvedValue({ items: [{ id: 3 }], total: 12 }); - const res = createMockResponse(); - await routeHandlers['get:/']({ query: { active: 'all', limit: '1', offset: '2' } }, res); + const res = await api.get(`${BASE}?active=all&limit=1&offset=2`).expect(200); expect(accountService.list).toHaveBeenCalledWith({ active: null, limit: 1, offset: 2 }); - expect(res.json).toHaveBeenCalledWith({ - ok: true, - data: { items: [{ id: 3 }], total: 12, limit: 1, offset: 2, links: [] }, - }); + expect(res.body).toEqual(okEnvelope({ items: [{ id: 3 }], total: 12, limit: 1, offset: 2, links: [] })); }); it('clamps limit to the per-resource cap', async () => { accountService.list.mockResolvedValue({ items: [], total: 0 }); - const res = createMockResponse(); - await routeHandlers['get:/']({ query: { limit: '99999' } }, res); + await api.get(`${BASE}?limit=99999`).expect(200); expect(accountService.list).toHaveBeenCalledWith({ active: true, limit: 1000, offset: 0 }); }); }); + +describe('GET /:id — real validateIdParam guard', () => { + beforeEach(() => vi.clearAllMocks()); + + it('rejects a non-integer :id with a 400 VALIDATION_ERROR envelope', async () => { + // Previously `vi.mock('.../middleware/validation.js')` replaced + // validateIdParam with a pass-through, so this guard was never tested. + const res = await api.get(`${BASE}/abc`).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); + expect(accountService.get).not.toHaveBeenCalled(); + }); +}); diff --git a/apps/node-backend/tests/routes/admin.test.js b/apps/node-backend/tests/routes/admin.test.js index 65764632..28e50148 100644 --- a/apps/node-backend/tests/routes/admin.test.js +++ b/apps/node-backend/tests/routes/admin.test.js @@ -1,17 +1,24 @@ /** * Admin route tests. * Mirrors: apps/backend/tests/test_admin.py + * + * Runs against the REAL router mounted on a throwaway Express app (see + * tests/helpers/routeApp.js). The production admin mount + * (main.js:324 — `mountRouter(app, '/api/admin', adminRateLimiter, + * adminCsrfGuard, adminAuthMiddleware, adminRouter)`) is reproduced via the + * harness's `before` slot with the REAL `createAdminAuthMiddleware`, driven by + * the same `getSettings().admin.authToken` getter main.js uses — so a + * configured token is now actually enforced in tests, previously impossible + * under the mock-router harness. `adminRateLimiter` (app-level, module-scoped + * counters) and `adminCsrfGuard` (redundant with the harness's own global CSRF + * mount, see routeApp.js fidelity map) are intentionally not added again here. + * `adminMutateLimiter`, declared INSIDE admin.js on individual mutate routes, + * is exercised for free since the real router is mounted. */ import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'; import { mockLogger } from '../helpers/mockLogger.js'; -import { createMockRouter, createMockResponse } from '../helpers/routeHarness.js'; - -const { router: mockRouter, handlers: routeHandlers } = createMockRouter(); - -vi.mock('express', () => ({ - default: { Router: () => mockRouter }, - Router: () => mockRouter, -})); +import { routeAgent, okEnvelope, errEnvelope } from '../helpers/routeApp.js'; +import { createAdminAuthMiddleware } from '../../src/middleware/adminAuth.js'; vi.mock('https', () => ({ default: { @@ -22,12 +29,13 @@ vi.mock('https', () => ({ vi.mock('../../src/database/connection.js', () => ({ checkConnection: vi.fn(), getTableCount: vi.fn(), + getClient: vi.fn(), query: vi.fn(), })); vi.mock('../../src/config/config.js', () => ({ getSettings: vi.fn(() => ({ - admin: { enableResetDb: false }, + admin: { enableResetDb: false, authToken: undefined }, isDevelopment: () => true, })), })); @@ -55,7 +63,18 @@ import { sanitizePersistedKinesisHistory } from '../../src/services/priceProvide import { listProviderHealth } from '../../src/services/providerHealthService.js'; import { getRouteManifest } from '../../src/services/routeManifest.js'; import https from 'https'; -await import('../../src/routes/admin.js'); + +const { default: adminRouter } = await import('../../src/routes/admin.js'); + +// Mirrors main.js:31 exactly — a per-request getter so a test can flip +// settings.admin.authToken between calls and see the guard react. +const adminAuthMiddleware = createAdminAuthMiddleware(() => getSettings().admin.authToken); + +const api = routeAgent(adminRouter, { + mountPath: '/api/admin', + before: [adminAuthMiddleware], +}); +const BASE = '/api/admin'; describe('Admin Routes', () => { const initialAppVersion = process.env.APP_VERSION; @@ -63,6 +82,10 @@ describe('Admin Routes', () => { beforeEach(() => { vi.clearAllMocks(); + getSettings.mockReturnValue({ + admin: { enableResetDb: false, authToken: undefined }, + isDevelopment: () => true, + }); delete process.env.APP_VERSION; delete process.env.APP_IMAGE_TAG; }); @@ -81,16 +104,58 @@ describe('Admin Routes', () => { } }); + // Newly on-path: the mock-router harness never ran any middleware, so the + // auth guard was never actually exercised even though it protects every + // admin request in production. + describe('admin auth guard (main.js:324)', () => { + it('passes through with no configured token (loopback-only trust model)', async () => { + checkConnection.mockResolvedValue(true); + getTableCount.mockResolvedValue(5); + + await api.get(`${BASE}/`).expect(200); + }); + + it('rejects a request with no Authorization header once a token is configured', async () => { + getSettings.mockReturnValue({ + admin: { enableResetDb: false, authToken: 'secret-token' }, + isDevelopment: () => true, + }); + + const res = await api.get(`${BASE}/`).expect(401); + expect(res.body).toEqual(errEnvelope({ code: 'UNAUTHORIZED' })); + expect(checkConnection).not.toHaveBeenCalled(); + }); + + it('rejects a request bearing the wrong token', async () => { + getSettings.mockReturnValue({ + admin: { enableResetDb: false, authToken: 'secret-token' }, + isDevelopment: () => true, + }); + + const res = await api.get(`${BASE}/`).set('Authorization', 'Bearer wrong').expect(401); + expect(res.body).toEqual(errEnvelope({ code: 'UNAUTHORIZED' })); + }); + + it('passes through with the correct bearer token', async () => { + getSettings.mockReturnValue({ + admin: { enableResetDb: false, authToken: 'secret-token' }, + isDevelopment: () => true, + }); + checkConnection.mockResolvedValue(true); + getTableCount.mockResolvedValue(5); + + await api.get(`${BASE}/`).set('Authorization', 'Bearer secret-token').expect(200); + }); + }); + describe('GET /', () => { it('should return status when connected', async () => { checkConnection.mockResolvedValue(true); getTableCount.mockResolvedValue(5); - const req = { query: {} }; - const res = mockResponse(); - await routeHandlers['get:/'](req, res); + const res = await api.get(`${BASE}/`).expect(200); - const result = res.json.mock.calls[0][0].data; + const result = res.body.data; expect(result.is_initialised).toBe(true); expect(result.table_count).toBe(5); expect(result.timestamp).toBeDefined(); @@ -100,30 +165,25 @@ describe('Admin Routes', () => { checkConnection.mockResolvedValue(true); getTableCount.mockResolvedValue(0); - const req = { query: {} }; - const res = mockResponse(); - await routeHandlers['get:/'](req, res); + const res = await api.get(`${BASE}/`).expect(200); - expect(res.json.mock.calls[0][0].data.is_initialised).toBe(false); + expect(res.body.data.is_initialised).toBe(false); }); it('should report uninitialised when disconnected', async () => { checkConnection.mockResolvedValue(false); - const req = { query: {} }; - const res = mockResponse(); - await routeHandlers['get:/'](req, res); + const res = await api.get(`${BASE}/`).expect(200); - expect(res.json.mock.calls[0][0].data.is_initialised).toBe(false); - expect(res.json.mock.calls[0][0].data.table_count).toBe(0); + expect(res.body.data.is_initialised).toBe(false); + expect(res.body.data.table_count).toBe(0); }); it('should propagate errors when connection fails', async () => { checkConnection.mockRejectedValue(new Error('Connection failed')); - const req = { query: {} }; - const res = mockResponse(); - await expect(routeHandlers['get:/'](req, res)).rejects.toThrow('Connection failed'); + const res = await api.get(`${BASE}/`).expect(500); + expect(res.body.error.message).toBe('Connection failed'); }); }); @@ -131,59 +191,50 @@ describe('Admin Routes', () => { it('should verify connection successfully', async () => { checkConnection.mockResolvedValue(true); - const req = { query: {} }; - const res = mockResponse(); - await routeHandlers['post:/database/init'](req, res); + const res = await api.post(`${BASE}/database/init`).expect(201); - expect(res.status).toHaveBeenCalledWith(201); - expect(res.json).toHaveBeenCalledWith(expect.objectContaining({ ok: true })); + expect(res.body).toEqual(expect.objectContaining({ ok: true })); }); it('should throw AppError when cannot connect', async () => { - const { AppError } = await import('../../src/middleware/errorHandler.js'); checkConnection.mockResolvedValue(false); - const req = { query: {} }; - const res = mockResponse(); - await expect(routeHandlers['post:/database/init'](req, res)).rejects.toBeInstanceOf(AppError); + const res = await api.post(`${BASE}/database/init`).expect(500); + expect(res.body).toEqual(errEnvelope({ code: 'APP_ERROR', message: 'Cannot connect to database' })); }); it('should propagate errors when init check throws', async () => { checkConnection.mockRejectedValue(new Error('driver stack trace')); - const req = { query: {} }; - const res = mockResponse(); - await expect(routeHandlers['post:/database/init'](req, res)).rejects.toThrow('driver stack trace'); + const res = await api.post(`${BASE}/database/init`).expect(500); + expect(res.body.error.message).toBe('driver stack trace'); }); }); describe('POST /database/reset', () => { it('should throw NotFoundError when reset disabled', async () => { - const { NotFoundError } = await import('../../src/middleware/errorHandler.js'); getSettings.mockReturnValue({ admin: { enableResetDb: false }, isDevelopment: () => true }); - const req = { query: {} }; - const res = mockResponse(); - await expect(routeHandlers['post:/database/reset'](req, res)).rejects.toBeInstanceOf(NotFoundError); + const res = await api.post(`${BASE}/database/reset`).expect(404); + expect(res.body).toEqual(errEnvelope({ code: 'NOT_FOUND' })); }); it('should throw ValidationError without force parameter', async () => { - const { ValidationError } = await import('../../src/middleware/errorHandler.js'); getSettings.mockReturnValue({ admin: { enableResetDb: true }, isDevelopment: () => true }); - const req = { query: {} }; - const res = mockResponse(); - await expect(routeHandlers['post:/database/reset'](req, res)).rejects.toBeInstanceOf(ValidationError); + const res = await api.post(`${BASE}/database/reset`).expect(400); + expect(res.body).toEqual(errEnvelope({ + code: 'VALIDATION_ERROR', + message: 'Database reset requires force=true parameter', + details: { hint: 'Set force=true query parameter to confirm reset (DESTRUCTIVE)' }, + })); }); it('should accept force=true', async () => { getSettings.mockReturnValue({ admin: { enableResetDb: true }, isDevelopment: () => true }); - const req = { query: { force: 'true' } }; - const res = mockResponse(); - await routeHandlers['post:/database/reset'](req, res); - - expect(res.json).toHaveBeenCalledWith(expect.objectContaining({ ok: true })); + const res = await api.post(`${BASE}/database/reset?force=true`).expect(200); + expect(res.body).toEqual(expect.objectContaining({ ok: true })); }); }); @@ -196,11 +247,9 @@ describe('Admin Routes', () => { html_url: 'https://github.com/EraPartner/Vision/releases/tag/v1.2.3', })); - const req = { query: {} }; - const res = mockResponse(); - await routeHandlers['get:/update/check'](req, res); + const res = await api.get(`${BASE}/update/check`).expect(200); - const payload = res.json.mock.calls[0][0].data; + const payload = res.body.data; expect(payload.latest_version).toBe('v1.2.3'); expect(payload.published_at).toBe('2026-04-01T12:00:00Z'); expect(payload.release_notes).toBe('Release notes'); @@ -216,21 +265,17 @@ describe('Admin Routes', () => { // made the frontend default to 'source' and render a dead Install button. mockGitHubReleaseBody(JSON.stringify({ tag_name: 'v9.9.9' })); - const req = { query: {} }; - const res = mockResponse(); - await routeHandlers['get:/update/check'](req, res); + const res = await api.get(`${BASE}/update/check`).expect(200); - expect(res.json.mock.calls[0][0].data.update_mode).toBe('docker-compose'); + expect(res.body.data.update_mode).toBe('docker-compose'); }); it('should include version metadata in update check response', async () => { mockGitHubReleaseBody(JSON.stringify({ tag_name: 'v2.1.0' })); - const req = { query: {} }; - const res = mockResponse(); - await routeHandlers['get:/update/check'](req, res); + const res = await api.get(`${BASE}/update/check`).expect(200); - const payload = res.json.mock.calls[0][0].data; + const payload = res.body.data; expect(payload.latest_version).toBe('v2.1.0'); expect(payload).toHaveProperty('current_version'); expect(payload).toHaveProperty('up_to_date'); @@ -239,58 +284,42 @@ describe('Admin Routes', () => { it('should return no-release payload when GitHub returns not found', async () => { mockGitHubReleaseBody(JSON.stringify({ message: 'Not Found' })); - const req = { query: {} }; - const res = mockResponse(); - await routeHandlers['get:/update/check'](req, res); - - expect(res.json).toHaveBeenCalledWith({ - ok: true, - data: { - up_to_date: true, - error: 'No published releases found', - latest_version: null, - }, - }); + const res = await api.get(`${BASE}/update/check`).expect(200); + + expect(res.body).toEqual(okEnvelope({ + up_to_date: true, + error: 'No published releases found', + latest_version: null, + })); }); it('should propagate error when release payload is invalid json', async () => { mockGitHubReleaseBody('{ invalid-json'); - const req = { query: {} }; - const res = mockResponse(); - await expect(routeHandlers['get:/update/check'](req, res)).rejects.toThrow(); + const res = await api.get(`${BASE}/update/check`).expect(500); + expect(res.body.error.code).toBe('INTERNAL_SERVER_ERROR'); }); }); describe('POST /update/apply', () => { it('should return update acknowledgement payload', async () => { - const req = { query: {} }; - const res = mockResponse(); - await routeHandlers['post:/update/apply'](req, res); - - expect(res.json).toHaveBeenCalledWith({ - ok: true, - data: { - success: true, - note: 'Updates are applied automatically by the desktop app. If an update is available, use the notification in the Vision app window to download and install it.', - }, - }); + const res = await api.post(`${BASE}/update/apply`).expect(200); + + expect(res.body).toEqual(okEnvelope({ + success: true, + note: 'Updates are applied automatically by the desktop app. If an update is available, use the notification in the Vision app window to download and install it.', + })); }); }); describe('POST /update/apply-and-restart', () => { it('should return backwards compatible update acknowledgement payload', async () => { - const req = { query: {} }; - const res = mockResponse(); - await routeHandlers['post:/update/apply-and-restart'](req, res); - - expect(res.json).toHaveBeenCalledWith({ - ok: true, - data: { - success: true, - note: 'Updates are managed by the Vision desktop app via Docker image pulls and the desktop shell updater. No manual action is required.', - }, - }); + const res = await api.post(`${BASE}/update/apply-and-restart`).expect(200); + + expect(res.body).toEqual(okEnvelope({ + success: true, + note: 'Updates are managed by the Vision desktop app via Docker image pulls and the desktop shell updater. No manual action is required.', + })); }); }); @@ -303,28 +332,22 @@ describe('Admin Routes', () => { failed: 0, }); - const req = { query: {} }; - const res = mockResponse(); - await routeHandlers['post:/investments/kinesis/sanitize-history'](req, res); - - expect(res.json).toHaveBeenCalledWith({ - ok: true, - data: { - message: 'Kinesis historical spikes sanitization completed', - processed: 3, - updated: 2, - correctedPoints: 4, - failed: 0, - }, - }); + const res = await api.post(`${BASE}/investments/kinesis/sanitize-history`).expect(200); + + expect(res.body).toEqual(okEnvelope({ + message: 'Kinesis historical spikes sanitization completed', + processed: 3, + updated: 2, + correctedPoints: 4, + failed: 0, + })); }); it('should propagate error when sanitization fails', async () => { sanitizePersistedKinesisHistory.mockRejectedValue(new Error('boom')); - const req = { query: {} }; - const res = mockResponse(); - await expect(routeHandlers['post:/investments/kinesis/sanitize-history'](req, res)).rejects.toThrow('boom'); + const res = await api.post(`${BASE}/investments/kinesis/sanitize-history`).expect(500); + expect(res.body.error.message).toBe('boom'); }); }); @@ -336,44 +359,39 @@ describe('Admin Routes', () => { const providers = [{ provider: 'yahoo', kind: 'price' }, { provider: 'ecb', kind: 'fx' }]; listProviderHealth.mockResolvedValue(providers); - const res = mockResponse(); - await routeHandlers['get:/providers/health']({}, res); + const res = await api.get(`${BASE}/providers/health`).expect(200); - expect(res.json).toHaveBeenCalledWith({ ok: true, data: { items: providers, total: 2 } }); + expect(res.body).toEqual(okEnvelope({ items: providers, total: 2 })); }); it('GET /endpoints returns { items, total }', async () => { const manifest = [{ method: 'GET', path: '/api/health' }]; getRouteManifest.mockReturnValue(manifest); - const res = mockResponse(); - await routeHandlers['get:/endpoints']({}, res); + const res = await api.get(`${BASE}/endpoints`).expect(200); - expect(res.json).toHaveBeenCalledWith({ ok: true, data: { items: manifest, total: 1 } }); + expect(res.body).toEqual(okEnvelope({ items: manifest, total: 1 })); }); it('GET /endpoint-liveness returns { items, total } with live flags', async () => { getRouteManifest.mockReturnValue([{ method: 'GET', path: '/api/health' }]); - const res = mockResponse(); - await routeHandlers['get:/endpoint-liveness']({}, res); + const res = await api.get(`${BASE}/endpoint-liveness`).expect(200); - expect(res.json).toHaveBeenCalledWith({ - ok: true, - data: { items: [{ method: 'GET', path: '/api/health', live: true }], total: 1 }, - }); + expect(res.body).toEqual(okEnvelope({ + items: [{ method: 'GET', path: '/api/health', live: true }], + total: 1, + })); }); it('GET /endpoints reports total 0 for an empty manifest', async () => { getRouteManifest.mockReturnValue([]); - const res = mockResponse(); - await routeHandlers['get:/endpoints']({}, res); + const res = await api.get(`${BASE}/endpoints`).expect(200); - expect(res.json).toHaveBeenCalledWith({ ok: true, data: { items: [], total: 0 } }); + expect(res.body).toEqual(okEnvelope({ items: [], total: 0 })); }); }); - }); function mockGitHubReleaseBody(body) { @@ -407,7 +425,3 @@ function mockGitHubReleaseBody(body) { return request; }); } - -function mockResponse() { - return createMockResponse(); -} diff --git a/apps/node-backend/tests/routes/aggregations.test.js b/apps/node-backend/tests/routes/aggregations.test.js index 44f06f42..03d036b5 100644 --- a/apps/node-backend/tests/routes/aggregations.test.js +++ b/apps/node-backend/tests/routes/aggregations.test.js @@ -4,16 +4,12 @@ * Focused on the cashflow-forecast-rolling window guard, which must emit the * canonical unified-envelope error (ValidationError → code VALIDATION_ERROR) * rather than the old hand-rolled { code: 'BAD_REQUEST' } body. + * + * Runs against the REAL router mounted on a throwaway Express app (see + * tests/helpers/routeApp.js). */ import { describe, it, expect, vi } from 'vitest'; -import { createMockRouter, createMockResponse } from '../helpers/routeHarness.js'; - -const { router: mockRouter, handlers: routeHandlers } = createMockRouter(); - -vi.mock('express', () => ({ - default: { Router: () => mockRouter }, - Router: () => mockRouter, -})); +import { routeAgent, errEnvelope } from '../helpers/routeApp.js'; // The real parseIntClamped caps days_back/days_forward at 365 each, so their sum // can never exceed 730 and the guard is otherwise unreachable. Bypass the clamp @@ -23,21 +19,19 @@ vi.mock('../../src/lib/pagination.js', () => ({ parsePagination: () => ({ limit: 50, offset: 0 }), })); -import { ValidationError } from '../../src/middleware/errorHandler.js'; -await import('../../src/routes/aggregations.js'); +const { default: aggregationsRouter } = await import('../../src/routes/aggregations.js'); -describe('Aggregation Routes — cashflow-forecast-rolling window guard', () => { - it('throws a canonical ValidationError (code VALIDATION_ERROR) when the window exceeds 730 days', async () => { - const req = { query: { days_back: '400', days_forward: '400' }, get: () => undefined }; - const res = createMockResponse(); +const api = routeAgent(aggregationsRouter, { mountPath: '/api/aggregations' }); - await expect(routeHandlers['get:/cashflow-forecast-rolling'](req, res)) - .rejects.toBeInstanceOf(ValidationError); +describe('Aggregation Routes — cashflow-forecast-rolling window guard', () => { + it('answers a canonical 400 VALIDATION_ERROR envelope when the window exceeds 730 days', async () => { + const res = await api + .get('/api/aggregations/cashflow-forecast-rolling?days_back=400&days_forward=400') + .expect(400); - const err = await routeHandlers['get:/cashflow-forecast-rolling'](req, res) - .then(() => null, (e) => e); - expect(err).toBeInstanceOf(ValidationError); - expect(err.code).toBe('VALIDATION_ERROR'); - expect(err.status).toBe(400); + expect(res.body).toEqual(errEnvelope({ + code: 'VALIDATION_ERROR', + message: 'days_back + days_forward must be <= 730', + })); }); }); diff --git a/apps/node-backend/tests/routes/aggregationsForecastBacktest.test.js b/apps/node-backend/tests/routes/aggregationsForecastBacktest.test.js index e6347d30..75191a15 100644 --- a/apps/node-backend/tests/routes/aggregationsForecastBacktest.test.js +++ b/apps/node-backend/tests/routes/aggregationsForecastBacktest.test.js @@ -7,9 +7,12 @@ * path), but they must now parse it through the same shared default-aware helper * so the accepted spellings can't diverge per endpoint (methods previously * accepted any value via `!== 'false'`, rolling only `=== 'true'`). + * + * The router half runs against the REAL router mounted on a throwaway Express + * app (see tests/helpers/routeApp.js). */ import { describe, it, expect, vi, beforeEach } from 'vitest'; -import { createMockRouter, createMockResponse } from '../helpers/routeHarness.js'; +import { routeAgent } from '../helpers/routeApp.js'; import { parseBoolQueryParam } from '../../src/routes/info/_queryParams.js'; describe('parseBoolQueryParam — default-aware boolean query param', () => { @@ -39,13 +42,6 @@ describe('parseBoolQueryParam — default-aware boolean query param', () => { }); }); -const { router: mockRouter, handlers: routeHandlers } = createMockRouter(); - -vi.mock('express', () => ({ - default: { Router: () => mockRouter }, - Router: () => mockRouter, -})); - const methodsSpy = vi.fn(async () => ({ data: {}, meta: {} })); const rollingSpy = vi.fn(async () => ({ data: {}, meta: {} })); @@ -54,11 +50,13 @@ vi.mock('../../src/services/calculations/forecast/index.js', () => ({ computeCashflowForecastRolling: (...a) => rollingSpy(...a), })); -await import('../../src/routes/aggregations.js'); +const { default: aggregationsRouter } = await import('../../src/routes/aggregations.js'); + +const api = routeAgent(aggregationsRouter, { mountPath: '/api/aggregations' }); -const run = async (key, query) => { - const res = createMockResponse(); - await routeHandlers[key]({ query, get: () => undefined }, res); +const run = async (path, query = {}) => { + const qs = new URLSearchParams(query).toString(); + await api.get(`/api/aggregations${path}${qs ? `?${qs}` : ''}`).expect(200); }; describe('cashflow-forecast endpoints — include_backtest default drift', () => { @@ -68,28 +66,28 @@ describe('cashflow-forecast endpoints — include_backtest default drift', () => }); it('methods defaults include_backtest ON when the param is omitted', async () => { - await run('get:/cashflow-forecast-methods', {}); + await run('/cashflow-forecast-methods'); expect(methodsSpy.mock.calls[0][0].includeBacktest).toBe(true); }); it('rolling defaults include_backtest OFF when the param is omitted', async () => { - await run('get:/cashflow-forecast-rolling', {}); + await run('/cashflow-forecast-rolling'); expect(rollingSpy.mock.calls[0][0].includeBacktest).toBe(false); }); it('both endpoints accept the same spellings (methods "0" → false, rolling "1" → true)', async () => { - await run('get:/cashflow-forecast-methods', { include_backtest: '0' }); + await run('/cashflow-forecast-methods', { include_backtest: '0' }); expect(methodsSpy.mock.calls[0][0].includeBacktest).toBe(false); - await run('get:/cashflow-forecast-rolling', { include_backtest: '1' }); + await run('/cashflow-forecast-rolling', { include_backtest: '1' }); expect(rollingSpy.mock.calls[0][0].includeBacktest).toBe(true); }); it('explicit override flips each default', async () => { - await run('get:/cashflow-forecast-methods', { include_backtest: 'false' }); + await run('/cashflow-forecast-methods', { include_backtest: 'false' }); expect(methodsSpy.mock.calls[0][0].includeBacktest).toBe(false); - await run('get:/cashflow-forecast-rolling', { include_backtest: 'true' }); + await run('/cashflow-forecast-rolling', { include_backtest: 'true' }); expect(rollingSpy.mock.calls[0][0].includeBacktest).toBe(true); }); }); diff --git a/apps/node-backend/tests/routes/ai.test.js b/apps/node-backend/tests/routes/ai.test.js index a39129f3..7981cabd 100644 --- a/apps/node-backend/tests/routes/ai.test.js +++ b/apps/node-backend/tests/routes/ai.test.js @@ -1,19 +1,29 @@ /** * AI chat route tests. * - * Focus: POST /api/ai/chat/stream SSE route + shared validator via /chat. - * Mirrors the mockRouter pattern from tests/routes/import.test.js. + * Runs against the REAL router mounted on a throwaway Express app (see + * tests/helpers/routeApp.js). `router.use(enforceAiChatEnabled)` (ai.js:~120) + * was never reachable under the old mock-router harness — `router.use()` was + * recorded but never invoked, so `/chat` and `/chat/stream` validation errors + * were being asserted as rejected promises from a handler called directly, + * bypassing Express entirely. They now travel through the real error handler + * and come back as ADR-026 envelopes. + * + * SSE stream ('/chat/stream'): the harness's real HTTP server means + * `res.writeHead`/`res.write`/`res.end` are the genuine Node response methods, + * not stubs. supertest buffers a `text/event-stream` body (matches its + * `text/*` buffering rule) into `res.text`, so SSE frames are asserted by + * parsing that raw string instead of inspecting `res.write.mock.calls`. + * + * Per the routeApp.js fidelity map, the app-level `/api/ai/chat` rate limiter + * (main.js:342-348, only mounted when `settings.aiChat.enabled`) is a + * module-scoped per-IP counter and is deliberately NOT reproduced here — it + * would 429 this suite's own many `/chat` requests. It is a real gap this + * suite cannot see; not exercised in either the old or new harness. */ -import { describe, it, expect, vi, beforeEach } from 'vitest'; +import { afterEach, describe, it, expect, vi, beforeEach } from 'vitest'; import { mockLogger } from '../helpers/mockLogger.js'; -import { createMockRouter, createMockResponse } from '../helpers/routeHarness.js'; - -const { router: mockRouter, handlers: routeHandlers } = createMockRouter(); - -vi.mock('express', () => ({ - default: { Router: () => mockRouter }, - Router: () => mockRouter, -})); +import { routeAgent, okEnvelope, errEnvelope } from '../helpers/routeApp.js'; vi.mock('../../src/services/aiChatService.js', () => { class AiChatServiceError extends Error { @@ -72,48 +82,32 @@ import { renameConversation, runChatTurn, } from '../../src/services/aiChatService.js'; -import { ValidationError, AppError } from '../../src/middleware/errorHandler.js'; -await import('../../src/routes/ai.js'); - -const UUID = '11111111-2222-4333-8444-555555555555'; +import settings from '../../src/config/config.js'; -function makeListenerStub() { - const listeners = {}; - return { - on: vi.fn((event, cb) => { - listeners[event] = listeners[event] || []; - listeners[event].push(cb); - }), - emit(event, ...args) { - (listeners[event] || []).forEach((cb) => cb(...args)); - }, - }; -} +const { default: aiRouter } = await import('../../src/routes/ai.js'); -function mockResponse() { - return createMockResponse(makeListenerStub()); -} +const api = routeAgent(aiRouter, { mountPath: '/api/ai' }); +const BASE = '/api/ai'; -function mockSseResponse() { - return { - writeHead: vi.fn(), - write: vi.fn(), - end: vi.fn(), - writableEnded: false, - ...makeListenerStub(), - }; -} +const UUID = '11111111-2222-4333-8444-555555555555'; -function mockStreamReq(body) { - const listeners = {}; - return { - body, - on: vi.fn((event, cb) => { - listeners[event] = listeners[event] || []; - listeners[event].push(cb); - }), - emit: (event, ...args) => (listeners[event] || []).forEach((cb) => cb(...args)), - }; +/** + * Split a buffered `text/event-stream` body into `{ name, data }` frames, + * skipping the leading padding comment (a `:`-prefixed line, ignored by the + * SSE spec — used to flush the browser's buffering threshold) and any + * heartbeat comments. + */ +function parseSseFrames(rawText) { + return rawText + .split('\n\n') + .filter((frame) => frame.startsWith('event:')) + .map((frame) => { + const [eventLine, dataLine] = frame.split('\n'); + return { + name: eventLine.replace(/^event: /, ''), + data: JSON.parse(dataLine.replace(/^data: /, '')), + }; + }); } // ────────────────────────────────────────── @@ -122,32 +116,26 @@ function mockStreamReq(body) { describe('POST /api/ai/chat/stream', () => { beforeEach(() => vi.clearAllMocks()); - it('throws ValidationError when message missing (before SSE headers)', async () => { - const req = mockStreamReq({ conversationId: UUID }); - const res = mockSseResponse(); - - await expect(routeHandlers['post:/chat/stream'](req, res)).rejects.toBeInstanceOf(ValidationError); + it('answers a 400 VALIDATION_ERROR envelope when message missing (before SSE headers)', async () => { + const res = await api.post(`${BASE}/chat/stream`).send({ conversationId: UUID }).expect(400); - expect(res.writeHead).not.toHaveBeenCalled(); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); + expect(res.headers['content-type']).toMatch(/json/); expect(runChatTurn).not.toHaveBeenCalled(); }); - it('throws ValidationError when conversationId is not a UUID', async () => { - const req = mockStreamReq({ conversationId: 'not-a-uuid', message: 'hi' }); - const res = mockSseResponse(); - - await expect(routeHandlers['post:/chat/stream'](req, res)).rejects.toBeInstanceOf(ValidationError); + it('answers a 400 VALIDATION_ERROR envelope when conversationId is not a UUID', async () => { + const res = await api.post(`${BASE}/chat/stream`).send({ conversationId: 'not-a-uuid', message: 'hi' }).expect(400); - expect(res.writeHead).not.toHaveBeenCalled(); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); + expect(res.headers['content-type']).toMatch(/json/); }); - it('throws ValidationError when message exceeds max length', async () => { - const req = mockStreamReq({ message: 'x'.repeat(8001) }); - const res = mockSseResponse(); + it('answers a 400 VALIDATION_ERROR envelope when message exceeds max length', async () => { + const res = await api.post(`${BASE}/chat/stream`).send({ message: 'x'.repeat(8001) }).expect(400); - await expect(routeHandlers['post:/chat/stream'](req, res)).rejects.toBeInstanceOf(ValidationError); - - expect(res.writeHead).not.toHaveBeenCalled(); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); + expect(res.headers['content-type']).toMatch(/json/); }); it('streams SSE events in order: user_message → token → tool_call → tool_result → done', async () => { @@ -174,24 +162,15 @@ describe('POST /api/ai/chat/stream', () => { }; }); - const req = mockStreamReq({ message: 'hi', conversationId: UUID }); - const res = mockSseResponse(); - - await routeHandlers['post:/chat/stream'](req, res); + const res = await api.post(`${BASE}/chat/stream`).send({ message: 'hi', conversationId: UUID }).expect(200); - expect(res.writeHead).toHaveBeenCalledWith(200, expect.objectContaining({ - 'Content-Type': 'text/event-stream', - 'Cache-Control': 'no-cache', - Connection: 'keep-alive', - 'X-Accel-Buffering': 'no', - })); + expect(res.headers['content-type']).toMatch(/^text\/event-stream/); + expect(res.headers['cache-control']).toBe('no-cache'); + expect(res.headers.connection).toBe('keep-alive'); + expect(res.headers['x-accel-buffering']).toBe('no'); - const writes = res.write.mock.calls - .map(([payload]) => payload) - // Skip the leading padding comment (lines beginning with `:`) used to - // flush the browser SSE buffer threshold. - .filter((p) => /^event:/.test(p)); - const eventNames = writes.map((p) => p.match(/^event: (\w+)/)[1]); + const frames = parseSseFrames(res.text); + const eventNames = frames.map((f) => f.name); expect(eventNames).toEqual([ 'user_message', 'token', @@ -202,29 +181,19 @@ describe('POST /api/ai/chat/stream', () => { 'done', ]); - const userFrame = writes[0]; - expect(userFrame).toContain('data: '); - expect(JSON.parse(userFrame.split('data: ')[1].trim())).toEqual({ message: userMsg }); + expect(frames[0].data).toEqual({ message: userMsg }); + expect(frames[1].data).toBe('Your '); - const tokenFrame = writes[1]; - expect(JSON.parse(tokenFrame.split('data: ')[1].trim())).toBe('Your '); + expect(frames[4].data.name).toBe('getSpendByCategory'); + expect(frames[4].data.args.from).toBe('2025-01-01'); - const toolCallFrame = writes[4]; - const toolCallPayload = JSON.parse(toolCallFrame.split('data: ')[1].trim()); - expect(toolCallPayload.name).toBe('getSpendByCategory'); - expect(toolCallPayload.args.from).toBe('2025-01-01'); + expect(frames[5].data).toEqual({ message: toolMsg }); - const toolResultFrame = writes[5]; - expect(JSON.parse(toolResultFrame.split('data: ')[1].trim())).toEqual({ message: toolMsg }); - - const doneFrame = writes[6]; - const donePayload = JSON.parse(doneFrame.split('data: ')[1].trim()); + const donePayload = frames[6].data; expect(donePayload.conversation).toEqual(conversation); expect(donePayload.assistantMessage).toEqual(assistantMsg); expect(donePayload.usage.evalCount).toBe(10); expect(donePayload.iterations).toBe(2); - - expect(res.end).toHaveBeenCalledTimes(1); }); it('passes AbortSignal to runChatTurn and streams with streaming:true', async () => { @@ -237,9 +206,7 @@ describe('POST /api/ai/chat/stream', () => { iterations: 1, }); - const req = mockStreamReq({ message: 'hi' }); - const res = mockSseResponse(); - await routeHandlers['post:/chat/stream'](req, res); + await api.post(`${BASE}/chat/stream`).send({ message: 'hi' }).expect(200); expect(runChatTurn).toHaveBeenCalledTimes(1); const callArgs = runChatTurn.mock.calls[0][0]; @@ -255,43 +222,40 @@ describe('POST /api/ai/chat/stream', () => { status: 503, })); - const req = mockStreamReq({ message: 'hi' }); - const res = mockSseResponse(); - await routeHandlers['post:/chat/stream'](req, res); - - const writes = res.write.mock.calls.map(([payload]) => payload); - expect(writes.some((p) => p.startsWith('event: error'))).toBe(true); - - const errFrame = writes.find((p) => p.startsWith('event: error')); - const errPayload = JSON.parse(errFrame.split('data: ')[1].trim()); - expect(errPayload).toEqual({ detail: 'Model unavailable', code: 'OLLAMA_UNREACHABLE' }); + const res = await api.post(`${BASE}/chat/stream`).send({ message: 'hi' }).expect(200); - expect(writes.some((p) => p.startsWith('event: done'))).toBe(false); - expect(res.end).toHaveBeenCalledTimes(1); + const frames = parseSseFrames(res.text); + const errFrame = frames.find((f) => f.name === 'error'); + expect(errFrame.data).toEqual({ detail: 'Model unavailable', code: 'OLLAMA_UNREACHABLE' }); + expect(frames.some((f) => f.name === 'done')).toBe(false); }); it('emits generic error SSE event on unexpected failure', async () => { runChatTurn.mockRejectedValue(new Error('db exploded')); - const req = mockStreamReq({ message: 'hi' }); - const res = mockSseResponse(); - await routeHandlers['post:/chat/stream'](req, res); + const res = await api.post(`${BASE}/chat/stream`).send({ message: 'hi' }).expect(200); - const writes = res.write.mock.calls.map(([payload]) => payload); - const errFrame = writes.find((p) => p.startsWith('event: error')); + const frames = parseSseFrames(res.text); + const errFrame = frames.find((f) => f.name === 'error'); expect(errFrame).toBeDefined(); - const errPayload = JSON.parse(errFrame.split('data: ')[1].trim()); - expect(errPayload).toEqual({ detail: 'Failed to stream AI chat message' }); - expect(errPayload.detail).not.toContain('db exploded'); - expect(res.end).toHaveBeenCalledTimes(1); + expect(errFrame.data).toEqual({ detail: 'Failed to stream AI chat message' }); + expect(JSON.stringify(errFrame.data)).not.toContain('db exploded'); }); it('aborts runChatTurn and stops writing on client disconnect', async () => { let capturedSignal; + let releaseTurn; + const released = new Promise((resolve) => { releaseTurn = resolve; }); + runChatTurn.mockImplementation(async ({ signal, onEvent }) => { capturedSignal = signal; onEvent({ type: 'user_message', data: { id: 'u1' } }); onEvent({ type: 'token', data: 'hello' }); + // Stay pending until the client abort has actually landed server-side + // (mirrors production, where the abort can race the in-flight tool loop) + // rather than resolving before res 'close' has had a chance to fire. + await new Promise((resolve) => signal.addEventListener('abort', resolve, { once: true })); + releaseTurn(); return { conversation: { id: UUID }, userMessage: { id: 'u1' }, @@ -302,17 +266,14 @@ describe('POST /api/ai/chat/stream', () => { }; }); - const req = mockStreamReq({ message: 'hi' }); - const res = mockSseResponse(); - - const promise = routeHandlers['post:/chat/stream'](req, res); - res.emit('close'); - await promise; + const test = api.post(`${BASE}/chat/stream`).send({ message: 'hi' }); + test.end(() => {}); // fire-and-forget: an aborted request rejects, and we assert server-side state instead + // Give the request time to reach the handler and commit the SSE headers. + await new Promise((resolve) => setTimeout(resolve, 30)); + test.abort(); + await released; expect(capturedSignal?.aborted).toBe(true); - const writes = res.write.mock.calls.map(([payload]) => payload); - expect(writes.some((p) => p.startsWith('event: done'))).toBe(false); - expect(res.end).not.toHaveBeenCalled(); }); it('skips writes for unknown onEvent types', async () => { @@ -329,15 +290,9 @@ describe('POST /api/ai/chat/stream', () => { }; }); - const req = mockStreamReq({ message: 'hi' }); - const res = mockSseResponse(); - await routeHandlers['post:/chat/stream'](req, res); - - const writes = res.write.mock.calls.map(([payload]) => payload); - const eventNames = writes - .map((p) => p.match(/^event: (\w+)/)?.[1]) - .filter(Boolean); + const res = await api.post(`${BASE}/chat/stream`).send({ message: 'hi' }).expect(200); + const eventNames = parseSseFrames(res.text).map((f) => f.name); expect(eventNames).not.toContain('unknown_event'); expect(eventNames).not.toContain('assistant_message'); expect(eventNames).toContain('done'); @@ -350,20 +305,16 @@ describe('POST /api/ai/chat/stream', () => { describe('POST /api/ai/chat', () => { beforeEach(() => vi.clearAllMocks()); - it('throws ValidationError when message missing', async () => { - const req = { body: {}, on: vi.fn() }; - const res = mockResponse(); - - await expect(routeHandlers['post:/chat'](req, res)).rejects.toBeInstanceOf(ValidationError); + it('answers a 400 VALIDATION_ERROR envelope when message missing', async () => { + const res = await api.post(`${BASE}/chat`).send({}).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); expect(runChatTurn).not.toHaveBeenCalled(); }); - it('throws ValidationError when model is an empty string', async () => { - const req = { body: { message: 'hi', model: ' ' }, on: vi.fn() }; - const res = mockResponse(); - - await expect(routeHandlers['post:/chat'](req, res)).rejects.toBeInstanceOf(ValidationError); + it('answers a 400 VALIDATION_ERROR envelope when model is an empty string', async () => { + const res = await api.post(`${BASE}/chat`).send({ message: 'hi', model: ' ' }).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); }); it('returns 200 with turn payload on success', async () => { @@ -377,44 +328,54 @@ describe('POST /api/ai/chat', () => { }; runChatTurn.mockResolvedValue(turn); - const req = { body: { message: 'hi' }, on: vi.fn() }; - const res = mockResponse(); - - await routeHandlers['post:/chat'](req, res); + const res = await api.post(`${BASE}/chat`).send({ message: 'hi' }).expect(200); expect(runChatTurn).toHaveBeenCalledTimes(1); const callArgs = runChatTurn.mock.calls[0][0]; expect(callArgs.streaming).toBeUndefined(); expect(callArgs.message).toBe('hi'); - expect(res.json).toHaveBeenCalledWith({ - ok: true, - data: { - conversation: turn.conversation, - userMessage: turn.userMessage, - toolMessages: turn.toolMessages, - assistantMessage: turn.assistantMessage, - usage: turn.usage, - iterations: turn.iterations, - }, - }); + expect(res.body).toEqual(okEnvelope({ + conversation: turn.conversation, + userMessage: turn.userMessage, + toolMessages: turn.toolMessages, + assistantMessage: turn.assistantMessage, + usage: turn.usage, + iterations: turn.iterations, + })); }); - it('throws AppError when AiChatServiceError occurs', async () => { + it('answers with the AiChatServiceError status/code when the service rejects', async () => { runChatTurn.mockRejectedValue(new AiChatServiceError('bad', { code: 'INVALID_INPUT', status: 400 })); - const req = { body: { message: 'hi' }, on: vi.fn() }; - const res = mockResponse(); - - await expect(routeHandlers['post:/chat'](req, res)).rejects.toBeInstanceOf(AppError); + const res = await api.post(`${BASE}/chat`).send({ message: 'hi' }).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'INVALID_INPUT', message: 'bad' })); }); - it('throws AppError on generic error', async () => { + it('answers a sanitized 500 on a generic service error', async () => { runChatTurn.mockRejectedValue(new Error('internal')); - const req = { body: { message: 'hi' }, on: vi.fn() }; - const res = mockResponse(); + const res = await api.post(`${BASE}/chat`).send({ message: 'hi' }).expect(500); + expect(res.body).toEqual(errEnvelope({ code: 'INTERNAL_SERVER_ERROR', message: 'Failed to process AI chat message' })); + }); +}); + +// ────────────────────────────────────────── +// Newly on-path: enforceAiChatEnabled (router.use, never reachable under the +// old mock-router harness — `router.use()` calls were recorded but never +// invoked). +// ────────────────────────────────────────── +describe('AI chat disabled gate (router.use(enforceAiChatEnabled))', () => { + const originalEnabled = settings.aiChat.enabled; + + afterEach(() => { + settings.aiChat.enabled = originalEnabled; + }); + + it('answers 503 SERVICE_UNAVAILABLE for any /api/ai/* route when disabled', async () => { + settings.aiChat.enabled = false; - await expect(routeHandlers['post:/chat'](req, res)).rejects.toBeInstanceOf(AppError); + const res = await api.get(`${BASE}/status`).expect(503); + expect(res.body).toEqual(errEnvelope({ code: 'SERVICE_UNAVAILABLE', message: 'AI chat is disabled' })); }); }); @@ -433,7 +394,7 @@ describe('POST /api/ai/chat body validation', () => { runChatTurn.mockResolvedValue(okTurn); const upper = UUID.toUpperCase(); - await routeHandlers['post:/chat']({ body: { message: 'hi', conversationId: upper }, on: vi.fn() }, mockResponse()); + await api.post(`${BASE}/chat`).send({ message: 'hi', conversationId: upper }).expect(200); expect(runChatTurn).toHaveBeenCalledWith(expect.objectContaining({ conversationId: upper })); }); @@ -441,62 +402,55 @@ describe('POST /api/ai/chat body validation', () => { it('maps a null conversationId to null (new conversation)', async () => { runChatTurn.mockResolvedValue(okTurn); - await routeHandlers['post:/chat']({ body: { message: 'hi', conversationId: null }, on: vi.fn() }, mockResponse()); + await api.post(`${BASE}/chat`).send({ message: 'hi', conversationId: null }).expect(200); expect(runChatTurn).toHaveBeenCalledWith(expect.objectContaining({ conversationId: null })); }); it('rejects a non-string conversationId', async () => { - const req = { body: { message: 'hi', conversationId: 42 }, on: vi.fn() }; - await expect(routeHandlers['post:/chat'](req, mockResponse())).rejects.toBeInstanceOf(ValidationError); + const res = await api.post(`${BASE}/chat`).send({ message: 'hi', conversationId: 42 }).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); expect(runChatTurn).not.toHaveBeenCalled(); }); it('accepts a message of exactly 4000 chars and rejects 4001', async () => { runChatTurn.mockResolvedValue(okTurn); - await routeHandlers['post:/chat']({ body: { message: 'x'.repeat(4000) }, on: vi.fn() }, mockResponse()); + await api.post(`${BASE}/chat`).send({ message: 'x'.repeat(4000) }).expect(200); expect(runChatTurn).toHaveBeenCalledTimes(1); - await expect( - routeHandlers['post:/chat']({ body: { message: 'x'.repeat(4001) }, on: vi.fn() }, mockResponse()), - ).rejects.toBeInstanceOf(ValidationError); + await api.post(`${BASE}/chat`).send({ message: 'x'.repeat(4001) }).expect(400); }); it('rejects a whitespace-only message', async () => { - const req = { body: { message: ' ' }, on: vi.fn() }; - await expect(routeHandlers['post:/chat'](req, mockResponse())).rejects.toBeInstanceOf(ValidationError); + await api.post(`${BASE}/chat`).send({ message: ' ' }).expect(400); }); it('rejects a non-string message', async () => { - const req = { body: { message: 123 }, on: vi.fn() }; - await expect(routeHandlers['post:/chat'](req, mockResponse())).rejects.toBeInstanceOf(ValidationError); + await api.post(`${BASE}/chat`).send({ message: 123 }).expect(400); }); it('maps a null model to null and rejects a non-string model', async () => { runChatTurn.mockResolvedValue(okTurn); - await routeHandlers['post:/chat']({ body: { message: 'hi', model: null }, on: vi.fn() }, mockResponse()); + await api.post(`${BASE}/chat`).send({ message: 'hi', model: null }).expect(200); expect(runChatTurn).toHaveBeenCalledWith(expect.objectContaining({ model: null })); - await expect( - routeHandlers['post:/chat']({ body: { message: 'hi', model: 7 }, on: vi.fn() }, mockResponse()), - ).rejects.toBeInstanceOf(ValidationError); + await api.post(`${BASE}/chat`).send({ message: 'hi', model: 7 }).expect(400); }); it('defaults useTools to true when omitted and honours an explicit false', async () => { runChatTurn.mockResolvedValue(okTurn); - await routeHandlers['post:/chat']({ body: { message: 'hi' }, on: vi.fn() }, mockResponse()); + await api.post(`${BASE}/chat`).send({ message: 'hi' }).expect(200); expect(runChatTurn).toHaveBeenLastCalledWith(expect.objectContaining({ useTools: true })); - await routeHandlers['post:/chat']({ body: { message: 'hi', useTools: false }, on: vi.fn() }, mockResponse()); + await api.post(`${BASE}/chat`).send({ message: 'hi', useTools: false }).expect(200); expect(runChatTurn).toHaveBeenLastCalledWith(expect.objectContaining({ useTools: false })); }); it('rejects a non-boolean useTools', async () => { - const req = { body: { message: 'hi', useTools: 'yes' }, on: vi.fn() }; - await expect(routeHandlers['post:/chat'](req, mockResponse())).rejects.toBeInstanceOf(ValidationError); + await api.post(`${BASE}/chat`).send({ message: 'hi', useTools: 'yes' }).expect(400); }); }); @@ -513,19 +467,15 @@ describe('AI conversation routes validation', () => { const rows = [{ id: UUID, title: 'One' }, { id: UUID, title: 'Two' }]; listConversations.mockResolvedValue(rows); - const res = mockResponse(); - await routeHandlers['get:/conversations']({}, res); - - expect(res.json).toHaveBeenCalledWith({ ok: true, data: { items: rows, total: 2 } }); + const res = await api.get(`${BASE}/conversations`).expect(200); + expect(res.body).toEqual(okEnvelope({ items: rows, total: 2 })); }); it('reports total 0 for an empty list', async () => { listConversations.mockResolvedValue([]); - const res = mockResponse(); - await routeHandlers['get:/conversations']({}, res); - - expect(res.json).toHaveBeenCalledWith({ ok: true, data: { items: [], total: 0 } }); + const res = await api.get(`${BASE}/conversations`).expect(200); + expect(res.body).toEqual(okEnvelope({ items: [], total: 0 })); }); }); @@ -533,17 +483,15 @@ describe('AI conversation routes validation', () => { it('creates with optional title/model absent (even without a body)', async () => { createEmptyConversation.mockResolvedValue({ id: UUID }); - const res = mockResponse(); - await routeHandlers['post:/conversations']({ body: undefined }, res); + await api.post(`${BASE}/conversations`).expect(201); expect(createEmptyConversation).toHaveBeenCalledWith({ title: undefined, model: undefined }); - expect(res.status).toHaveBeenCalledWith(201); }); it('accepts an empty-string title (only type and length are checked)', async () => { createEmptyConversation.mockResolvedValue({ id: UUID }); - await routeHandlers['post:/conversations']({ body: { title: '' } }, mockResponse()); + await api.post(`${BASE}/conversations`).send({ title: '' }).expect(201); expect(createEmptyConversation).toHaveBeenCalledWith({ title: '', model: undefined }); }); @@ -551,29 +499,21 @@ describe('AI conversation routes validation', () => { it('accepts a title of exactly 200 chars and rejects 201', async () => { createEmptyConversation.mockResolvedValue({ id: UUID }); - await routeHandlers['post:/conversations']({ body: { title: 't'.repeat(200) } }, mockResponse()); + await api.post(`${BASE}/conversations`).send({ title: 't'.repeat(200) }).expect(201); expect(createEmptyConversation).toHaveBeenCalledTimes(1); - await expect( - routeHandlers['post:/conversations']({ body: { title: 't'.repeat(201) } }, mockResponse()), - ).rejects.toBeInstanceOf(ValidationError); + await api.post(`${BASE}/conversations`).send({ title: 't'.repeat(201) }).expect(400); }); it('rejects a non-string title', async () => { - await expect( - routeHandlers['post:/conversations']({ body: { title: 5 } }, mockResponse()), - ).rejects.toBeInstanceOf(ValidationError); + const res = await api.post(`${BASE}/conversations`).send({ title: 5 }).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); expect(createEmptyConversation).not.toHaveBeenCalled(); }); it('rejects a blank or null model (null is NOT treated as absent here)', async () => { - await expect( - routeHandlers['post:/conversations']({ body: { model: ' ' } }, mockResponse()), - ).rejects.toBeInstanceOf(ValidationError); - - await expect( - routeHandlers['post:/conversations']({ body: { model: null } }, mockResponse()), - ).rejects.toBeInstanceOf(ValidationError); + await api.post(`${BASE}/conversations`).send({ model: ' ' }).expect(400); + await api.post(`${BASE}/conversations`).send({ model: null }).expect(400); }); }); @@ -581,16 +521,14 @@ describe('AI conversation routes validation', () => { it('renames with the exact (untrimmed) title', async () => { renameConversation.mockResolvedValue({ id: UUID, title: ' Hi ' }); - await routeHandlers['patch:/conversations/:id']({ params: { id: UUID }, body: { title: ' Hi ' } }, mockResponse()); + await api.patch(`${BASE}/conversations/${UUID}`).send({ title: ' Hi ' }).expect(200); expect(renameConversation).toHaveBeenCalledWith(UUID, ' Hi '); }); it('rejects a missing, blank, or non-string title', async () => { for (const body of [{}, { title: ' ' }, { title: 9 }]) { - await expect( - routeHandlers['patch:/conversations/:id']({ params: { id: UUID }, body }, mockResponse()), - ).rejects.toBeInstanceOf(ValidationError); + await api.patch(`${BASE}/conversations/${UUID}`).send(body).expect(400); } expect(renameConversation).not.toHaveBeenCalled(); }); @@ -598,18 +536,14 @@ describe('AI conversation routes validation', () => { it('accepts a title of exactly 200 chars and rejects 201', async () => { renameConversation.mockResolvedValue({ id: UUID }); - await routeHandlers['patch:/conversations/:id']({ params: { id: UUID }, body: { title: 't'.repeat(200) } }, mockResponse()); + await api.patch(`${BASE}/conversations/${UUID}`).send({ title: 't'.repeat(200) }).expect(200); expect(renameConversation).toHaveBeenCalledTimes(1); - await expect( - routeHandlers['patch:/conversations/:id']({ params: { id: UUID }, body: { title: 't'.repeat(201) } }, mockResponse()), - ).rejects.toBeInstanceOf(ValidationError); + await api.patch(`${BASE}/conversations/${UUID}`).send({ title: 't'.repeat(201) }).expect(400); }); it('rejects a malformed conversation id', async () => { - await expect( - routeHandlers['patch:/conversations/:id']({ params: { id: 'nope' }, body: { title: 'x' } }, mockResponse()), - ).rejects.toBeInstanceOf(ValidationError); + await api.patch(`${BASE}/conversations/nope`).send({ title: 'x' }).expect(400); }); }); @@ -618,19 +552,27 @@ describe('AI conversation routes validation', () => { const upper = UUID.toUpperCase(); getConversationWithMessages.mockResolvedValue({ id: upper, messages: [] }); - await routeHandlers['get:/conversations/:id']({ params: { id: upper } }, mockResponse()); + await api.get(`${BASE}/conversations/${upper}`).expect(200); expect(getConversationWithMessages).toHaveBeenCalledWith(upper); }); it('rejects a missing or malformed id', async () => { - await expect( - routeHandlers['get:/conversations/:id']({ params: { id: '123' } }, mockResponse()), - ).rejects.toBeInstanceOf(ValidationError); + await api.get(`${BASE}/conversations/123`).expect(400); + + const res = await api.delete(`${BASE}/conversations/123`).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); + expect(deleteConversation).not.toHaveBeenCalled(); + }); - await expect( - routeHandlers['delete:/conversations/:id']({ params: { id: '' } }, mockResponse()), - ).rejects.toBeInstanceOf(ValidationError); + // The original suite asserted this branch by calling the handler directly + // with a hand-built `{ params: { id: '' } }` — a request Express can never + // actually construct: `/conversations/` with no verb defined for the bare + // collection path other than GET/POST simply doesn't route to the + // GET/DELETE `:id` handler at all, it 404s via the funnel handler first. + it('funnels an empty :id segment to a 404, not the malformed-id ValidationError', async () => { + const res = await api.delete(`${BASE}/conversations/`).expect(404); + expect(res.body).toEqual(errEnvelope({ code: 'NOT_FOUND' })); expect(deleteConversation).not.toHaveBeenCalled(); }); }); diff --git a/apps/node-backend/tests/routes/attachments.test.js b/apps/node-backend/tests/routes/attachments.test.js index b422fd72..ee85740a 100644 --- a/apps/node-backend/tests/routes/attachments.test.js +++ b/apps/node-backend/tests/routes/attachments.test.js @@ -1,16 +1,18 @@ /** * Attachment route tests — upload cleanup on failed DB insert. + * + * Runs against the REAL router mounted on a throwaway Express app (see + * tests/helpers/routeApp.js) — validateIdParam is no longer stubbed. The + * multer upload middleware (`attachmentUpload.single`) stays mocked (as + * before) since it does real disk I/O; the mock now drives `req.file` / + * upload errors through the real middleware chain instead of the test + * hand-building a `req.file`. */ import { describe, it, expect, vi, beforeEach } from 'vitest'; import { mockLogger } from '../helpers/mockLogger.js'; -import { createMockRouter, createMockResponse } from '../helpers/routeHarness.js'; +import { routeAgent, okEnvelope, errEnvelope } from '../helpers/routeApp.js'; -const { router: mockRouter, handlers: routeHandlers } = createMockRouter(); - -vi.mock('express', () => ({ - default: { Router: () => mockRouter }, - Router: () => mockRouter, -})); +const uploadState = vi.hoisted(() => ({ file: null, error: null })); vi.mock('../../src/services/attachmentRecordService.js', () => ({ attachmentRepository: { @@ -24,17 +26,19 @@ vi.mock('../../src/services/attachmentRecordService.js', () => ({ })); vi.mock('../../src/services/attachmentService.js', () => ({ - attachmentUpload: { single: vi.fn(() => vi.fn()) }, + attachmentUpload: { + single: () => (req, _res, cb) => { + if (uploadState.error) return cb(uploadState.error); + req.file = uploadState.file; + cb(); + }, + }, storeAttachment: vi.fn(), resolveAbsolutePath: vi.fn(), removeAttachmentFile: vi.fn(), verifyAttachmentContent: vi.fn(), })); -vi.mock('../../src/middleware/validation.js', () => ({ - validateIdParam: vi.fn(), -})); - vi.mock('../../src/config/logger.js', () => ({ logger: mockLogger(), })); @@ -42,16 +46,17 @@ vi.mock('../../src/config/logger.js', () => ({ import { attachmentRepository } from '../../src/services/attachmentRecordService.js'; import { storeAttachment, removeAttachmentFile, verifyAttachmentContent } from '../../src/services/attachmentService.js'; import { logger } from '../../src/config/logger.js'; -await import('../../src/routes/attachments.js'); -const UPLOAD_REQ = () => ({ - params: { id: '1' }, - file: { originalname: 'receipt.png', size: 1234 }, -}); +const { default: attachmentsRouter } = await import('../../src/routes/attachments.js'); + +const api = routeAgent(attachmentsRouter, { mountPath: '/api/attachments' }); +const BASE = '/api/attachments'; describe('Attachment routes', () => { beforeEach(() => { vi.clearAllMocks(); + uploadState.file = { originalname: 'receipt.png', size: 1234 }; + uploadState.error = null; verifyAttachmentContent.mockReturnValue('image/png'); attachmentRepository.transactionExists.mockResolvedValue(true); storeAttachment.mockResolvedValue('attachments/1/receipt.png'); @@ -61,10 +66,9 @@ describe('Attachment routes', () => { it('stores the file and creates the DB row', async () => { attachmentRepository.create.mockResolvedValue({ id: 7, transaction_id: 1 }); - const res = mockResponse(); - await callHandler(routeHandlers['post:/transaction/:id'], UPLOAD_REQ(), res); + const res = await api.post(`${BASE}/transaction/1`).expect(201); - expect(res.status).toHaveBeenCalledWith(201); + expect(res.body).toEqual(okEnvelope({ id: 7, transaction_id: 1 })); expect(attachmentRepository.create).toHaveBeenCalledWith( expect.objectContaining({ stored_path: 'attachments/1/receipt.png' }), ); @@ -77,25 +81,37 @@ describe('Attachment routes', () => { attachmentRepository.create.mockRejectedValue(new Error('FK violation')); removeAttachmentFile.mockResolvedValue(undefined); - const res = mockResponse(); - await callHandler(routeHandlers['post:/transaction/:id'], UPLOAD_REQ(), res); + await api.post(`${BASE}/transaction/1`).expect(500); expect(removeAttachmentFile).toHaveBeenCalledWith('attachments/1/receipt.png'); - expect(res.status).toHaveBeenCalledWith(500); }); it('still surfaces the insert error when cleanup itself fails', async () => { attachmentRepository.create.mockRejectedValue(new Error('FK violation')); removeAttachmentFile.mockRejectedValue(new Error('EACCES')); - const res = mockResponse(); - await callHandler(routeHandlers['post:/transaction/:id'], UPLOAD_REQ(), res); + const res = await api.post(`${BASE}/transaction/1`).expect(500); - expect(res.status).toHaveBeenCalledWith(500); - const body = res.json.mock.calls[0][0]; - expect(body.error.message).toBe('FK violation'); + expect(res.body.error.message).toBe('FK violation'); expect(logger.warn).toHaveBeenCalled(); }); + + it('rejects a missing file with a 400 VALIDATION_ERROR envelope', async () => { + // Newly on-path: the real middleware chain now runs, so a request with + // no file actually reaches the "No file uploaded" guard. + uploadState.file = undefined; + + const res = await api.post(`${BASE}/transaction/1`).expect(400); + + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); + expect(attachmentRepository.transactionExists).not.toHaveBeenCalled(); + }); + + it('rejects a non-integer :id via the real validateIdParam guard', async () => { + const res = await api.post(`${BASE}/transaction/abc`).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); + expect(attachmentRepository.transactionExists).not.toHaveBeenCalled(); + }); }); describe('DELETE /:id', () => { @@ -104,14 +120,11 @@ describe('Attachment routes', () => { attachmentRepository.deleteById.mockResolvedValue(true); removeAttachmentFile.mockResolvedValue(undefined); - const res = mockResponse(); - await routeHandlers['delete:/:id']({ params: { id: '7' } }, res); + const res = await api.delete(`${BASE}/7`).expect(204); expect(attachmentRepository.deleteById).toHaveBeenCalledWith(7); expect(removeAttachmentFile).toHaveBeenCalledWith('attachments/1/receipt.png'); - expect(res.status).toHaveBeenCalledWith(204); - expect(res.send).toHaveBeenCalledWith(); - expect(res.json).not.toHaveBeenCalled(); + expect(res.text).toBe(''); }); // The row is already gone; an orphaned file is logged, not surfaced. @@ -120,21 +133,18 @@ describe('Attachment routes', () => { attachmentRepository.deleteById.mockResolvedValue(true); removeAttachmentFile.mockRejectedValue(new Error('EACCES')); - const res = mockResponse(); - await routeHandlers['delete:/:id']({ params: { id: '7' } }, res); + const res = await api.delete(`${BASE}/7`).expect(204); expect(logger.warn).toHaveBeenCalled(); - expect(res.status).toHaveBeenCalledWith(204); - expect(res.json).not.toHaveBeenCalled(); + expect(res.text).toBe(''); }); it('404s when the attachment does not exist', async () => { attachmentRepository.findById.mockResolvedValue(null); - const res = mockResponse(); - await callHandler(routeHandlers['delete:/:id'], { params: { id: '99' } }, res); + const res = await api.delete(`${BASE}/99`).expect(404); - expect(res.status).toHaveBeenCalledWith(404); + expect(res.body.error.code).toBe('NOT_FOUND'); expect(attachmentRepository.deleteById).not.toHaveBeenCalled(); }); }); @@ -145,49 +155,21 @@ describe('Attachment routes', () => { it('lists every attachment (unbounded query, no limit/offset echoed)', async () => { attachmentRepository.listByTransaction.mockResolvedValue([{ id: 1 }, { id: 2 }]); - const res = mockResponse(); - await routeHandlers['get:/transaction/:id']({ params: { id: '5' }, query: {} }, res); + const res = await api.get(`${BASE}/transaction/5`).expect(200); expect(attachmentRepository.listByTransaction).toHaveBeenCalledWith(5, {}); expect(attachmentRepository.countByTransaction).not.toHaveBeenCalled(); - expect(res.json).toHaveBeenCalledWith({ - ok: true, - data: { items: [{ id: 1 }, { id: 2 }], total: 2 }, - }); + expect(res.body).toEqual(okEnvelope({ items: [{ id: 1 }, { id: 2 }], total: 2 })); }); it('pages and reports the full total when limit/offset are supplied', async () => { attachmentRepository.listByTransaction.mockResolvedValue([{ id: 2 }]); attachmentRepository.countByTransaction.mockResolvedValue(4); - const res = mockResponse(); - await routeHandlers['get:/transaction/:id']( - { params: { id: '5' }, query: { limit: '1', offset: '1' } }, - res, - ); + const res = await api.get(`${BASE}/transaction/5?limit=1&offset=1`).expect(200); expect(attachmentRepository.listByTransaction).toHaveBeenCalledWith(5, { limit: 1, offset: 1 }); - expect(res.json).toHaveBeenCalledWith({ - ok: true, - data: { items: [{ id: 2 }], total: 4, limit: 1, offset: 1 }, - }); + expect(res.body).toEqual(okEnvelope({ items: [{ id: 2 }], total: 4, limit: 1, offset: 1 })); }); }); }); - -function mockResponse() { - return createMockResponse({ setHeader: vi.fn(), end: vi.fn(), headersSent: false }); -} - -async function callHandler(handler, req, res) { - try { - await handler(req, res); - } catch (err) { - const status = err.status ?? 500; - const code = err.code ?? 'INTERNAL_SERVER_ERROR'; - const message = err.message ?? 'Internal server error'; - const error = { code, message }; - if (err.details !== undefined) error.details = err.details; - res.status(status).json({ ok: false, error }); - } -} diff --git a/apps/node-backend/tests/routes/categories.test.js b/apps/node-backend/tests/routes/categories.test.js index a81cfd37..c60f32ee 100644 --- a/apps/node-backend/tests/routes/categories.test.js +++ b/apps/node-backend/tests/routes/categories.test.js @@ -2,21 +2,17 @@ * Category route tests. * Mirrors: apps/backend/tests/test_categories.py * - * Uses mocked repository layer for unit testing. - * Mocks express Router to avoid dependency issues. + * Runs against the REAL router mounted on a throwaway Express app (see + * tests/helpers/routeApp.js) — validateIdParam is no longer stubbed. */ import { describe, it, expect, vi, beforeEach } from 'vitest'; import { mockLogger } from '../helpers/mockLogger.js'; -import { createMockRouter, createMockResponse } from '../helpers/routeHarness.js'; - -// Mock express Router -const { router: mockRouter, handlers: routeHandlers } = createMockRouter(); - -vi.mock('express', () => ({ - default: { Router: () => mockRouter }, - Router: () => mockRouter, -})); +import { routeAgent, okEnvelope, errEnvelope } from '../helpers/routeApp.js'; +// The route imports its repository through services/categoryService.js, which +// re-exports the default from this module (`export { default } from +// '../repositories/categoryRepository.js'`) — mocking the repository here +// intercepts that same binding. vi.mock('../../src/repositories/categoryRepository.js', () => ({ default: { getAll: vi.fn(), @@ -29,15 +25,20 @@ vi.mock('../../src/repositories/categoryRepository.js', () => ({ }, })); +vi.mock('../../src/services/materializedViewService.js', () => ({ + scheduleRefresh: vi.fn(), +})); + vi.mock('../../src/config/logger.js', () => ({ logger: mockLogger(), })); import categoryRepository from '../../src/repositories/categoryRepository.js'; -import { ValidationError, NotFoundError } from '../../src/middleware/errorHandler.js'; -// Import routes AFTER mocks are set up -await import('../../src/routes/categories.js'); +const { default: categoriesRouter } = await import('../../src/routes/categories.js'); + +const api = routeAgent(categoriesRouter, { mountPath: '/api/categories' }); +const BASE = '/api/categories'; describe('Category Routes', () => { beforeEach(() => vi.clearAllMocks()); @@ -47,11 +48,9 @@ describe('Category Routes', () => { categoryRepository.getAll.mockResolvedValue([]); categoryRepository.getCount.mockResolvedValue(0); - const req = { query: {} }; - const res = mockResponse(); - await routeHandlers['get:/'](req, res); + const res = await api.get(BASE).expect(200); - expect(res.json).toHaveBeenCalledWith(expect.objectContaining({ + expect(res.body).toEqual(expect.objectContaining({ ok: true, data: expect.objectContaining({ items: [], total: 0, limit: 50, offset: 0 }), })); @@ -65,26 +64,20 @@ describe('Category Routes', () => { categoryRepository.getAll.mockResolvedValue(categories); categoryRepository.getCount.mockResolvedValue(2); - const req = { query: {} }; - const res = mockResponse(); - await routeHandlers['get:/'](req, res); + const res = await api.get(BASE).expect(200); - const result = res.json.mock.calls[0][0]; - expect(result.data.total).toBe(2); - expect(result.data.items.length).toBe(2); + expect(res.body.data.total).toBe(2); + expect(res.body.data.items.length).toBe(2); }); it('should respect pagination parameters', async () => { categoryRepository.getAll.mockResolvedValue([]); categoryRepository.getCount.mockResolvedValue(5); - const req = { query: { limit: '2', offset: '1' } }; - const res = mockResponse(); - await routeHandlers['get:/'](req, res); + const res = await api.get(`${BASE}?limit=2&offset=1`).expect(200); - const result = res.json.mock.calls[0][0]; - expect(result.data.limit).toBe(2); - expect(result.data.offset).toBe(1); + expect(res.body.data.limit).toBe(2); + expect(res.body.data.offset).toBe(1); }); }); @@ -95,11 +88,7 @@ describe('Category Routes', () => { created: true, }); - const req = { body: { general: 'groceries', detail: 'food' } }; - const res = mockResponse(); - await routeHandlers['post:/'](req, res); - - expect(res.status).toHaveBeenCalledWith(201); + await api.post(BASE).send({ general: 'groceries', detail: 'food' }).expect(201); }); it('should return 200 for duplicate', async () => { @@ -108,17 +97,12 @@ describe('Category Routes', () => { created: false, }); - const req = { body: { general: 'groceries', detail: 'food' } }; - const res = mockResponse(); - await routeHandlers['post:/'](req, res); - - expect(res.status).toHaveBeenCalledWith(200); + await api.post(BASE).send({ general: 'groceries', detail: 'food' }).expect(200); }); - it('should throw ValidationError for missing fields', async () => { - const req = { body: { general: 'groceries' } }; - const res = mockResponse(); - await expect(routeHandlers['post:/'](req, res)).rejects.toBeInstanceOf(ValidationError); + it('should return a 400 VALIDATION_ERROR envelope for missing fields', async () => { + const res = await api.post(BASE).send({ general: 'groceries' }).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); }); }); @@ -126,19 +110,23 @@ describe('Category Routes', () => { it('should return category by id', async () => { categoryRepository.getById.mockResolvedValue({ id: 1, general: 'GROCERIES', detail: 'FOOD' }); - const req = { params: { id: '1' } }; - const res = mockResponse(); - await routeHandlers['get:/:id'](req, res); - - expect(res.json).toHaveBeenCalled(); + const res = await api.get(`${BASE}/1`).expect(200); + expect(res.body.data.id).toBe(1); }); - it('should throw NotFoundError for non-existent', async () => { + it('should return a 404 NOT_FOUND envelope for non-existent', async () => { categoryRepository.getById.mockResolvedValue(null); - const req = { params: { id: '99999' } }; - const res = mockResponse(); - await expect(routeHandlers['get:/:id'](req, res)).rejects.toBeInstanceOf(NotFoundError); + const res = await api.get(`${BASE}/99999`).expect(404); + expect(res.body).toEqual(errEnvelope({ code: 'NOT_FOUND' })); + }); + + it('rejects a non-integer :id via the real validateIdParam guard', async () => { + // Previously `vi.mock('.../middleware/validation.js')` replaced + // validateIdParam with a pass-through, so this guard was never tested. + const res = await api.get(`${BASE}/abc`).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); + expect(categoryRepository.getById).not.toHaveBeenCalled(); }); }); @@ -146,19 +134,15 @@ describe('Category Routes', () => { it('should update category', async () => { categoryRepository.update.mockResolvedValue({ id: 1, general: 'UPDATED' }); - const req = { params: { id: '1' }, body: { general: 'updated' } }; - const res = mockResponse(); - await routeHandlers['patch:/:id'](req, res); - - expect(res.json).toHaveBeenCalled(); + const res = await api.patch(`${BASE}/1`).send({ general: 'updated' }).expect(200); + expect(res.body.data.general).toBe('UPDATED'); }); - it('should throw NotFoundError for non-existent', async () => { + it('should return a 404 NOT_FOUND envelope for non-existent', async () => { categoryRepository.update.mockResolvedValue(null); - const req = { params: { id: '99999' }, body: { general: 'test' } }; - const res = mockResponse(); - await expect(routeHandlers['patch:/:id'](req, res)).rejects.toBeInstanceOf(NotFoundError); + const res = await api.patch(`${BASE}/99999`).send({ general: 'test' }).expect(404); + expect(res.body).toEqual(errEnvelope({ code: 'NOT_FOUND' })); }); }); @@ -166,21 +150,15 @@ describe('Category Routes', () => { it('should delete and return 204 with no body', async () => { categoryRepository.hardDelete.mockResolvedValue(true); - const req = { params: { id: '1' } }; - const res = mockResponse(); - await routeHandlers['delete:/:id'](req, res); - - expect(res.status).toHaveBeenCalledWith(204); - expect(res.send).toHaveBeenCalledWith(); - expect(res.json).not.toHaveBeenCalled(); + const res = await api.delete(`${BASE}/1`).expect(204); + expect(res.text).toBe(''); }); - it('should throw NotFoundError for non-existent', async () => { + it('should return a 404 NOT_FOUND envelope for non-existent', async () => { categoryRepository.hardDelete.mockResolvedValue(false); - const req = { params: { id: '99999' } }; - const res = mockResponse(); - await expect(routeHandlers['delete:/:id'](req, res)).rejects.toBeInstanceOf(NotFoundError); + const res = await api.delete(`${BASE}/99999`).expect(404); + expect(res.body).toEqual(errEnvelope({ code: 'NOT_FOUND' })); }); }); @@ -188,17 +166,13 @@ describe('Category Routes', () => { it('should assign category to recipients', async () => { categoryRepository.assignToRecipients.mockResolvedValue(2); - const req = { params: { id: '1' }, body: { recipient_ids: [1, 2] } }; - const res = mockResponse(); - await routeHandlers['post:/:id/assign'](req, res); - - expect(res.json.mock.calls[0][0].data.updated_recipients).toBe(2); + const res = await api.post(`${BASE}/1/assign`).send({ recipient_ids: [1, 2] }).expect(200); + expect(res.body.data.updated_recipients).toBe(2); }); - it('should throw ValidationError for missing recipient_ids', async () => { - const req = { params: { id: '1' }, body: {} }; - const res = mockResponse(); - await expect(routeHandlers['post:/:id/assign'](req, res)).rejects.toBeInstanceOf(ValidationError); + it('should return a 400 VALIDATION_ERROR envelope for missing recipient_ids', async () => { + const res = await api.post(`${BASE}/1/assign`).send({}).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); }); }); @@ -211,29 +185,31 @@ describe('Category Routes', () => { }); categoryRepository.assignToRecipients.mockResolvedValue(3); - const req = { body: { category_general: 'GROCERIES', category_detail: 'FOOD', recipient_ids: [1, 2, 3] } }; - const res = mockResponse(); - await routeHandlers['post:/assign'](req, res); - - expect(res.json.mock.calls[0][0].data.updated_recipients).toBe(3); + const res = await api.post(`${BASE}/assign`) + .send({ category_general: 'GROCERIES', category_detail: 'FOOD', recipient_ids: [1, 2, 3] }) + .expect(200); + expect(res.body.data.updated_recipients).toBe(3); }); - it('should throw ValidationError for missing category_general', async () => { - const req = { body: { category_detail: 'FOOD', recipient_ids: [1] } }; - const res = mockResponse(); - await expect(routeHandlers['post:/assign'](req, res)).rejects.toBeInstanceOf(ValidationError); + it('should return a 400 VALIDATION_ERROR envelope for missing category_general', async () => { + const res = await api.post(`${BASE}/assign`) + .send({ category_detail: 'FOOD', recipient_ids: [1] }) + .expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); }); - it('should throw ValidationError for missing category_detail', async () => { - const req = { body: { category_general: 'GROCERIES', recipient_ids: [1] } }; - const res = mockResponse(); - await expect(routeHandlers['post:/assign'](req, res)).rejects.toBeInstanceOf(ValidationError); + it('should return a 400 VALIDATION_ERROR envelope for missing category_detail', async () => { + const res = await api.post(`${BASE}/assign`) + .send({ category_general: 'GROCERIES', recipient_ids: [1] }) + .expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); }); - it('should throw ValidationError for missing recipient_ids', async () => { - const req = { body: { category_general: 'GROCERIES', category_detail: 'FOOD' } }; - const res = mockResponse(); - await expect(routeHandlers['post:/assign'](req, res)).rejects.toBeInstanceOf(ValidationError); + it('should return a 400 VALIDATION_ERROR envelope for missing recipient_ids', async () => { + const res = await api.post(`${BASE}/assign`) + .send({ category_general: 'GROCERIES', category_detail: 'FOOD' }) + .expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); }); it('should handle single recipient_id (not array)', async () => { @@ -243,74 +219,64 @@ describe('Category Routes', () => { }); categoryRepository.assignToRecipients.mockResolvedValue(1); - const req = { body: { category_general: 'GROCERIES', category_detail: 'FOOD', recipient_ids: 42 } }; - const res = mockResponse(); - await routeHandlers['post:/assign'](req, res); - - expect(res.json.mock.calls[0][0].data.updated_recipients).toBe(1); + const res = await api.post(`${BASE}/assign`) + .send({ category_general: 'GROCERIES', category_detail: 'FOOD', recipient_ids: 42 }) + .expect(200); + expect(res.body.data.updated_recipients).toBe(1); }); - it('should propagate error when DB throws', async () => { + it('should propagate a 500 when the DB throws', async () => { categoryRepository.createOrGet.mockRejectedValue(new Error('DB error')); - const req = { body: { category_general: 'GROCERIES', category_detail: 'FOOD', recipient_ids: [1] } }; - const res = mockResponse(); - await expect(routeHandlers['post:/assign'](req, res)).rejects.toThrow('DB error'); + const res = await api.post(`${BASE}/assign`) + .send({ category_general: 'GROCERIES', category_detail: 'FOOD', recipient_ids: [1] }) + .expect(500); + expect(res.body.error.message).toBe('DB error'); }); }); // ── Error paths for existing routes ──────────────────────── describe('Error handling', () => { - it('GET / should propagate error when DB throws', async () => { + it('GET / should answer a 500 when the DB throws', async () => { categoryRepository.getAll.mockRejectedValue(new Error('DB error')); - const req = { query: {} }; - const res = mockResponse(); - await expect(routeHandlers['get:/'](req, res)).rejects.toThrow('DB error'); + const res = await api.get(BASE).expect(500); + expect(res.body.error.message).toBe('DB error'); }); - it('POST / should propagate error when DB throws', async () => { + it('POST / should answer a 500 when the DB throws', async () => { categoryRepository.createOrGet.mockRejectedValue(new Error('DB error')); - const req = { body: { general: 'TEST', detail: 'TEST' } }; - const res = mockResponse(); - await expect(routeHandlers['post:/'](req, res)).rejects.toThrow('DB error'); + const res = await api.post(BASE).send({ general: 'TEST', detail: 'TEST' }).expect(500); + expect(res.body.error.message).toBe('DB error'); }); - it('GET /:id should propagate error when DB throws', async () => { + it('GET /:id should answer a 500 when the DB throws', async () => { categoryRepository.getById.mockRejectedValue(new Error('DB error')); - const req = { params: { id: '1' } }; - const res = mockResponse(); - await expect(routeHandlers['get:/:id'](req, res)).rejects.toThrow('DB error'); + const res = await api.get(`${BASE}/1`).expect(500); + expect(res.body.error.message).toBe('DB error'); }); - it('PATCH /:id should propagate error when DB throws', async () => { + it('PATCH /:id should answer a 500 when the DB throws', async () => { categoryRepository.update.mockRejectedValue(new Error('DB error')); - const req = { params: { id: '1' }, body: { general: 'test' } }; - const res = mockResponse(); - await expect(routeHandlers['patch:/:id'](req, res)).rejects.toThrow('DB error'); + const res = await api.patch(`${BASE}/1`).send({ general: 'test' }).expect(500); + expect(res.body.error.message).toBe('DB error'); }); - it('DELETE /:id should propagate error when DB throws', async () => { + it('DELETE /:id should answer a 500 when the DB throws', async () => { categoryRepository.hardDelete.mockRejectedValue(new Error('DB error')); - const req = { params: { id: '1' } }; - const res = mockResponse(); - await expect(routeHandlers['delete:/:id'](req, res)).rejects.toThrow('DB error'); + const res = await api.delete(`${BASE}/1`).expect(500); + expect(res.body.error.message).toBe('DB error'); }); - it('POST /:id/assign should propagate error when DB throws', async () => { + it('POST /:id/assign should answer a 500 when the DB throws', async () => { categoryRepository.assignToRecipients.mockRejectedValue(new Error('DB error')); - const req = { params: { id: '1' }, body: { recipient_ids: [1] } }; - const res = mockResponse(); - await expect(routeHandlers['post:/:id/assign'](req, res)).rejects.toThrow('DB error'); + const res = await api.post(`${BASE}/1/assign`).send({ recipient_ids: [1] }).expect(500); + expect(res.body.error.message).toBe('DB error'); }); }); }); - -function mockResponse() { - return createMockResponse(); -} diff --git a/apps/node-backend/tests/routes/crossWorkspace.test.js b/apps/node-backend/tests/routes/crossWorkspace.test.js index 45eeed76..1eb444a4 100644 --- a/apps/node-backend/tests/routes/crossWorkspace.test.js +++ b/apps/node-backend/tests/routes/crossWorkspace.test.js @@ -5,26 +5,23 @@ * non-negative Number() coercion, all-zero-sum rejection, model-key lookup, * and the model/targetWeights dispatch (a truthy non-object targetWeights is * IGNORED, falling through to the model branch). + * + * Runs against the REAL router mounted on a throwaway Express app (see + * tests/helpers/routeApp.js). */ import { beforeEach, describe, expect, it, vi } from 'vitest'; -import { createMockRouter, createMockResponse } from '../helpers/routeHarness.js'; - -const { router: mockRouter, handlers: routeHandlers } = createMockRouter(); - -vi.mock('express', () => ({ - default: { Router: () => mockRouter }, - Router: () => mockRouter, -})); +import { routeAgent, errEnvelope } from '../helpers/routeApp.js'; vi.mock('../../src/services/crossWorkspaceDataService.js', () => ({ assembleRebalanceInputs: vi.fn(), })); import { assembleRebalanceInputs } from '../../src/services/crossWorkspaceDataService.js'; -import { ValidationError } from '../../src/middleware/errorHandler.js'; -await import('../../src/routes/crossWorkspace.js'); -const rebalance = (body) => routeHandlers['post:/rebalance']({ body }, createMockResponse()); +const { default: crossWorkspaceRouter } = await import('../../src/routes/crossWorkspace.js'); + +const api = routeAgent(crossWorkspaceRouter, { mountPath: '/api/cross-workspace' }); +const rebalance = (body) => api.post('/api/cross-workspace/rebalance').send(body); describe('POST /rebalance validation', () => { beforeEach(() => { @@ -37,57 +34,59 @@ describe('POST /rebalance validation', () => { }); it('rejects a non-numeric or negative sleeve weight', async () => { - await expect(rebalance({ targetWeights: { stocks: 'abc' } })) - .rejects.toThrow('targetWeights.stocks must be a non-negative number'); - await expect(rebalance({ targetWeights: { stocks: 0.5, bonds: -0.1 } })) - .rejects.toThrow('targetWeights.bonds must be a non-negative number'); + const res1 = await rebalance({ targetWeights: { stocks: 'abc' } }).expect(400); + expect(res1.body).toEqual(errEnvelope({ + code: 'VALIDATION_ERROR', + message: 'targetWeights.stocks must be a non-negative number', + })); + + const res2 = await rebalance({ targetWeights: { stocks: 0.5, bonds: -0.1 } }).expect(400); + expect(res2.body.error.message).toBe('targetWeights.bonds must be a non-negative number'); }); it('rejects all-zero weights (and an empty record) with the zero-sum message', async () => { - await expect(rebalance({ targetWeights: { stocks: 0, bonds: 0 } })) - .rejects.toThrow('targetWeights must include at least one positive weight'); - await expect(rebalance({ targetWeights: {} })) - .rejects.toThrow('targetWeights must include at least one positive weight'); + const res1 = await rebalance({ targetWeights: { stocks: 0, bonds: 0 } }).expect(400); + expect(res1.body.error.message).toBe('targetWeights must include at least one positive weight'); + + const res2 = await rebalance({ targetWeights: {} }).expect(400); + expect(res2.body.error.message).toBe('targetWeights must include at least one positive weight'); }); it('rejects an unknown model with the preset list', async () => { - await expect(rebalance({ model: 'yolo' })).rejects.toThrow(/Unknown model 'yolo'.*sixty_forty/); + const res = await rebalance({ model: 'yolo' }).expect(400); + expect(res.body.error.message).toMatch(/Unknown model 'yolo'.*sixty_forty/); }); it('requires either model or targetWeights', async () => { - await expect(rebalance({})).rejects.toBeInstanceOf(ValidationError); - await expect(rebalance({})).rejects.toThrow(/Provide either/); + const res = await rebalance({}).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); + expect(res.body.error.message).toMatch(/Provide either/); }); it('ignores a truthy non-object targetWeights and falls through to the model branch', async () => { - await expect(rebalance({ targetWeights: 'garbage' })).rejects.toThrow(/Provide either/); + const res = await rebalance({ targetWeights: 'garbage' }).expect(400); + expect(res.body.error.message).toMatch(/Provide either/); }); it('coerces numeric-string weights and normalizes them to sum to 1', async () => { - const res = createMockResponse(); - await routeHandlers['post:/rebalance']({ body: { targetWeights: { stocks: '3', bonds: 1 } } }, res); - const payload = res.json.mock.calls[0][0]; - expect(payload.ok).toBe(true); - expect(payload.data.targetWeights).toEqual({ stocks: 0.75, bonds: 0.25 }); + const res = await rebalance({ targetWeights: { stocks: '3', bonds: 1 } }).expect(200); + expect(res.body.ok).toBe(true); + expect(res.body.data.targetWeights).toEqual({ stocks: 0.75, bonds: 0.25 }); }); it('accepts a classic model preset and folds unrepresentable sleeves', async () => { - const res = createMockResponse(); - await routeHandlers['post:/rebalance']({ body: { model: 'three_fund' } }, res); - const payload = res.json.mock.calls[0][0]; + const res = await rebalance({ model: 'three_fund' }).expect(200); // intl_stocks folds into stocks: 0.48 + 0.12 = 0.6 - expect(payload.data.targetWeights.stocks).toBeCloseTo(0.6, 10); - expect(payload.data.targetWeights.bonds).toBeCloseTo(0.4, 10); + expect(res.body.data.targetWeights.stocks).toBeCloseTo(0.6, 10); + expect(res.body.data.targetWeights.bonds).toBeCloseTo(0.4, 10); }); it('uppercases a string currency and defaults non-strings to EUR', async () => { - const res = createMockResponse(); - await routeHandlers['post:/rebalance']({ body: { model: 'sixty_forty', currency: 'usd' } }, res); - expect(res.json.mock.calls[0][0].data.currency).toBe('USD'); + const res = await rebalance({ model: 'sixty_forty', currency: 'usd' }).expect(200); + expect(res.body.data.currency).toBe('USD'); expect(assembleRebalanceInputs).toHaveBeenCalledWith({ currency: 'USD' }); - const res2 = createMockResponse(); - await routeHandlers['post:/rebalance']({ body: { model: 'sixty_forty', currency: 42 } }, res2); - expect(res2.json.mock.calls[0][0].data.currency).toBe('EUR'); + const res2 = await rebalance({ model: 'sixty_forty', currency: 42 }).expect(200); + expect(res2.body.data.currency).toBe('EUR'); }); }); diff --git a/apps/node-backend/tests/routes/import.test.js b/apps/node-backend/tests/routes/import.test.js index 3bd7c2f9..ab939edc 100644 --- a/apps/node-backend/tests/routes/import.test.js +++ b/apps/node-backend/tests/routes/import.test.js @@ -1,22 +1,40 @@ /** * Import route tests. * Mirrors: apps/backend/tests/test_import.py + * + * Runs against the REAL router mounted on a throwaway Express app (see + * tests/helpers/routeApp.js), which also puts the router's own trailing + * error middleware (`router.use(csvUploadErrorTranslator)`, + * routes/importRoutes.js:580) on the tested path, and the zod-validated + * batch/row id parsing (parseBatchIdParam / parseBatchRowIdParams) — both + * silently dropped/bypassed by the old mock-router harness. + * + * Mount path is /api/import, behind importRateLimiter at the app level + * (main.js:325) — a module-scoped per-IP counter deliberately NOT reproduced + * here per the routeApp.js fidelity map (it would 429 this suite's own many + * requests). + * + * multer is stubbed to a pass-through; requests that need `req.file` set it + * via a shared `uploadState` (vi.hoisted), the same pattern as + * attachments.test.js. This lets one suite cover both "no file uploaded" + * guards AND the csvUploadErrorTranslator's multer-error mapping (a fixed + * `before`-middleware injection, as importValidationPins.test.js uses, can + * only ever inject a successful upload). */ import { describe, it, expect, vi, beforeEach } from 'vitest'; import { mockConnection } from '../helpers/repoMocks.js'; import { mockLogger } from '../helpers/mockLogger.js'; -import { createMockRouter, createMockResponse } from '../helpers/routeHarness.js'; +import { routeAgent, okEnvelope, errEnvelope } from '../helpers/routeApp.js'; -const { router: mockRouter, handlers: routeHandlers } = createMockRouter(); - -vi.mock('express', () => ({ - default: { Router: () => mockRouter }, - Router: () => mockRouter, -})); +const uploadState = vi.hoisted(() => ({ file: null, error: null })); vi.mock('multer', () => { const multer = vi.fn(() => ({ - single: vi.fn(() => (req, res, next) => next()), + single: () => (req, _res, next) => { + if (uploadState.error) return next(uploadState.error); + req.file = uploadState.file ?? undefined; + next(); + }, })); multer.MulterError = class MulterError extends Error { constructor(code) { super(code); this.code = code; } @@ -45,6 +63,7 @@ vi.mock('os', () => ({ vi.mock('../../src/services/importPipeline/index.js', () => ({ runImportPipeline: vi.fn(), + commitImport: vi.fn(), })); vi.mock('../../src/services/bankAdapters.js', () => ({ @@ -100,7 +119,6 @@ vi.mock('../../src/repositories/customParserConfigRepository.js', () => ({ vi.mock('../../src/database/connection.js', () => mockConnection()); import { runImportPipeline } from '../../src/services/importPipeline/index.js'; -import { getSupportedBanks } from '../../src/services/bankAdapters.js'; import { importRecipientsCSV, importCategoriesCSV } from '../../src/services/dataImportService.js'; import { listBatches, @@ -111,114 +129,108 @@ import { overrideCategory, categoryExists, } from '../../src/repositories/importBatchRepository.js'; -import { query } from '../../src/database/connection.js'; import multer from 'multer'; -import { ValidationError, NotFoundError, ConflictError } from '../../src/middleware/errorHandler.js'; import customParserConfigRepository from '../../src/repositories/customParserConfigRepository.js'; -await import('../../src/routes/importRoutes.js'); + +const { default: importRouter } = await import('../../src/routes/importRoutes.js'); + +const BASE = '/api/import'; +const api = routeAgent(importRouter, { mountPath: BASE }); +// Same router behind an error handler in production mode (main.js:401 passes +// `settings.isProduction`) — used only for the one test below that pins the +// message-sanitization branch (errorHandler.js:234-235); the harness defaults +// isProduction to false so unsanitized 5xx messages stay visible elsewhere, +// matching a dev/test run. +const apiProd = routeAgent(importRouter, { mountPath: BASE, isProduction: () => true }); + +const FILE = { path: '/tmp/test.csv', originalname: 'test.csv', size: 100 }; + +/** Split a buffered `text/event-stream` body into `{ name, data }` frames. */ +function parseSseFrames(rawText) { + return rawText + .split('\n\n') + .filter((frame) => frame.startsWith('event:')) + .map((frame) => { + const [eventLine, dataLine] = frame.split('\n'); + return { + name: eventLine.replace(/^event: /, ''), + data: JSON.parse(dataLine.replace(/^data: /, '')), + }; + }); +} describe('Import Routes', () => { - beforeEach(() => vi.clearAllMocks()); + beforeEach(() => { + vi.clearAllMocks(); + uploadState.file = { ...FILE }; + uploadState.error = null; + }); // ────────────────────────────────────────── // POST /api/import/csv // ────────────────────────────────────────── describe('POST /csv', () => { it('should return 400 when no file uploaded', async () => { - const req = { file: null, query: { bank_name: 'belfius' }, body: {} }; - const res = mockResponse(); + uploadState.file = null; - await expect(routeHandlers['post:/csv'](req, res)).rejects.toBeInstanceOf(ValidationError); + const res = await api.post(`${BASE}/csv`).query({ bank_name: 'belfius' }).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); }); it('should return 400 when bank_name missing', async () => { - const req = { file: { path: '/tmp/test.csv', originalname: 'test.csv' }, query: {}, body: {} }; - const res = mockResponse(); - - await expect(routeHandlers['post:/csv'](req, res)).rejects.toBeInstanceOf(ValidationError); + const res = await api.post(`${BASE}/csv`).send({}).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); }); it('should return 201 on successful import', async () => { - runImportPipeline.mockResolvedValue({ - total: 5, imported: 4, duplicates: 1, errors: 0, - }); + runImportPipeline.mockResolvedValue({ total: 5, imported: 4, duplicates: 1, errors: 0 }); - const req = { - file: { path: '/tmp/test.csv', originalname: 'test.csv' }, - query: { bank_name: 'belfius' }, - body: {}, - }; - const res = mockResponse(); - await routeHandlers['post:/csv'](req, res); - - expect(res.status).toHaveBeenCalledWith(201); - const body = res.json.mock.calls[0][0]; - expect(body.data.total).toBe(5); - expect(body.data.imported).toBe(4); - expect(body.data.duplicates).toBe(1); - expect(body.data.errors).toBe(0); - expect(body.data.status).toBe('completed'); + const res = await api.post(`${BASE}/csv`).query({ bank_name: 'belfius' }).expect(201); + + expect(res.body.data.total).toBe(5); + expect(res.body.data.imported).toBe(4); + expect(res.body.data.duplicates).toBe(1); + expect(res.body.data.errors).toBe(0); + expect(res.body.data.status).toBe('completed'); }); it('should return completed_with_errors status', async () => { - runImportPipeline.mockResolvedValue({ - total: 10, imported: 8, duplicates: 1, errors: 1, - }); + runImportPipeline.mockResolvedValue({ total: 10, imported: 8, duplicates: 1, errors: 1 }); - const req = { - file: { path: '/tmp/test.csv', originalname: 'test.csv' }, - query: { bank_name: 'kbc' }, - body: {}, - }; - const res = mockResponse(); - await routeHandlers['post:/csv'](req, res); + const res = await api.post(`${BASE}/csv`).query({ bank_name: 'kbc' }).expect(201); - expect(res.status).toHaveBeenCalledWith(201); - const body = res.json.mock.calls[0][0]; - expect(body.data.status).toBe('completed_with_errors'); + expect(res.body.data.status).toBe('completed_with_errors'); }); it('should return 400 for invalid bank config', async () => { runImportPipeline.mockRejectedValue(new Error('No configuration found for bank')); - const req = { - file: { path: '/tmp/test.csv', originalname: 'test.csv' }, - query: { bank_name: 'UnknownBank' }, - body: {}, - }; - const res = mockResponse(); - - await expect(routeHandlers['post:/csv'](req, res)).rejects.toBeInstanceOf(ValidationError); + const res = await api.post(`${BASE}/csv`).query({ bank_name: 'UnknownBank' }).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); }); it('should return 500 on general import failure', async () => { runImportPipeline.mockRejectedValue(new Error('Parse error')); - const req = { - file: { path: '/tmp/test.csv', originalname: 'test.csv' }, - query: { bank_name: 'belfius' }, - body: {}, - }; - const res = mockResponse(); - - await expect(routeHandlers['post:/csv'](req, res)).rejects.toThrow('Parse error'); + const res = await api.post(`${BASE}/csv`).query({ bank_name: 'belfius' }).expect(500); + expect(res.body).toEqual(errEnvelope({ code: 'INTERNAL_SERVER_ERROR', message: 'Parse error' })); }); - it('should not leak internal error details on import failure', async () => { + // Was `.rejects.toThrow()` against the directly-invoked handler — that only + // proved SOMETHING threw, not that the response the client actually + // receives is sanitized. Pinned for real here: dev/test mode still echoes + // the raw message (existing behavior — see the `apiProd` assertion below + // for the production branch this test's title actually describes). + it('should not leak internal error details on import failure (production mode)', async () => { runImportPipeline.mockRejectedValue(new Error('sensitive parser trace')); - const req = { - file: { - path: '/tmp/test.csv', - originalname: 'test.csv', - mimetype: 'text/csv', - }, - query: { bank_name: 'belfius' }, - body: {}, - }; - const res = mockResponse(); + const res = await apiProd.post(`${BASE}/csv`).query({ bank_name: 'belfius' }).expect(500); - await expect(routeHandlers['post:/csv'](req, res)).rejects.toThrow(); + expect(res.body).toEqual(errEnvelope({ + code: 'INTERNAL_SERVER_ERROR', + message: 'An internal server error occurred. Please try again later.', + })); + expect(JSON.stringify(res.body)).not.toContain('sensitive parser trace'); }); }); @@ -227,88 +239,55 @@ describe('Import Routes', () => { // ────────────────────────────────────────── describe('POST /csv/custom', () => { it('should return 400 when no file uploaded', async () => { - const req = { file: null, query: {}, body: {} }; - const res = mockResponse(); + uploadState.file = null; - await expect(routeHandlers['post:/csv/custom'](req, res)).rejects.toBeInstanceOf(ValidationError); + const res = await api.post(`${BASE}/csv/custom`).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); }); it('should return 400 for missing required params', async () => { - const req = { - file: { path: '/tmp/custom.csv', originalname: 'custom.csv' }, - query: { bank_name: 'Custom' }, - body: {}, - }; - const res = mockResponse(); - - await expect(routeHandlers['post:/csv/custom'](req, res)).rejects.toThrow('Missing required parameters'); + const res = await api.post(`${BASE}/csv/custom`).query({ bank_name: 'Custom' }).expect(400); + expect(res.body.error.message).toContain('Missing required parameters'); }); it('should return 400 for invalid separator', async () => { - const req = { - file: { path: '/tmp/custom.csv', originalname: 'custom.csv' }, - query: { - bank_name: 'Custom', date_format: '%d/%m/%Y', - date_column: 'Date', recipient_column: 'Desc', - amount_column: 'Amount', separator: ';;', - }, - body: {}, - }; - const res = mockResponse(); - - await expect(routeHandlers['post:/csv/custom'](req, res)).rejects.toThrow('separator'); + const res = await api.post(`${BASE}/csv/custom`).query({ + bank_name: 'Custom', date_format: '%d/%m/%Y', + date_column: 'Date', recipient_column: 'Desc', + amount_column: 'Amount', separator: ';;', + }).expect(400); + expect(res.body.error.message).toMatch(/separator/); }); it('should return 400 for negative skip_rows', async () => { // csv-parse throws a raw error on a negative `from` — must 400 up front. - const req = { - file: { path: '/tmp/custom.csv', originalname: 'custom.csv' }, - query: { - bank_name: 'Custom', date_format: '%d/%m/%Y', - date_column: 'Date', recipient_column: 'Desc', - amount_column: 'Amount', skip_rows: '-3', - }, - body: {}, - }; - const res = mockResponse(); + const res = await api.post(`${BASE}/csv/custom`).query({ + bank_name: 'Custom', date_format: '%d/%m/%Y', + date_column: 'Date', recipient_column: 'Desc', + amount_column: 'Amount', skip_rows: '-3', + }).expect(400); - await expect(routeHandlers['post:/csv/custom'](req, res)).rejects.toThrow('skip_rows'); + expect(res.body.error.message).toMatch(/skip_rows/); expect(runImportPipeline).not.toHaveBeenCalled(); }); it('should return 201 on success', async () => { - runImportPipeline.mockResolvedValue({ - total: 1, imported: 1, duplicates: 0, errors: 0, - }); + runImportPipeline.mockResolvedValue({ total: 1, imported: 1, duplicates: 0, errors: 0 }); - const req = { - file: { path: '/tmp/custom.csv', originalname: 'custom.csv' }, - query: { - bank_name: 'Custom', date_format: '%d/%m/%Y', - date_column: 'Date', recipient_column: 'Desc', amount_column: 'Amount', - }, - body: {}, - }; - const res = mockResponse(); - await routeHandlers['post:/csv/custom'](req, res); - - expect(res.status).toHaveBeenCalledWith(201); + await api.post(`${BASE}/csv/custom`).query({ + bank_name: 'Custom', date_format: '%d/%m/%Y', + date_column: 'Date', recipient_column: 'Desc', amount_column: 'Amount', + }).expect(201); }); it('should return 500 on error', async () => { runImportPipeline.mockRejectedValue(new Error('Import failed')); - const req = { - file: { path: '/tmp/custom.csv', originalname: 'custom.csv' }, - query: { - bank_name: 'Custom', date_format: '%d/%m/%Y', - date_column: 'Date', recipient_column: 'Desc', amount_column: 'Amount', - }, - body: {}, - }; - const res = mockResponse(); - - await expect(routeHandlers['post:/csv/custom'](req, res)).rejects.toThrow('Import failed'); + const res = await api.post(`${BASE}/csv/custom`).query({ + bank_name: 'Custom', date_format: '%d/%m/%Y', + date_column: 'Date', recipient_column: 'Desc', amount_column: 'Amount', + }).expect(500); + expect(res.body).toEqual(errEnvelope({ code: 'INTERNAL_SERVER_ERROR', message: 'Import failed' })); }); }); @@ -322,82 +301,62 @@ describe('Import Routes', () => { return { total: 2, imported: 2, duplicates: 0, errors: 0 }; }); - const req = { - file: { path: '/tmp/stream.csv', originalname: 'stream.csv' }, - query: { bank_name: 'belfius' }, - body: {}, - on: vi.fn(), - }; - const res = mockSseResponse(); + const res = await api.post(`${BASE}/csv/stream`).query({ bank_name: 'belfius' }).expect(200); - await routeHandlers['post:/csv/stream'](req, res); + expect(res.headers['content-type']).toMatch(/^text\/event-stream/); + const frames = parseSseFrames(res.text); + const names = frames.map((f) => f.name); + expect(names).toContain('progress'); + expect(names).toContain('complete'); - expect(res.writeHead).toHaveBeenCalledWith(200, expect.objectContaining({ - 'Content-Type': 'text/event-stream', + const complete = frames.find((f) => f.name === 'complete'); + expect(complete.data).toEqual(expect.objectContaining({ + total_processed: 2, imported: 2, duplicates: 0, errors: 0, status: 'completed', percent: 100, })); - - const writes = res.write.mock.calls.map(([payload]) => payload); - expect(writes.some(payload => payload.includes('event: progress'))).toBe(true); - expect(writes.some(payload => payload.includes('event: complete'))).toBe(true); - expect(res.end).toHaveBeenCalledTimes(1); }); - it('should throw ValidationError when bank_name missing', async () => { - const req = { - file: { path: '/tmp/stream.csv', originalname: 'stream.csv' }, - query: {}, - body: {}, - on: vi.fn(), - }; - const res = mockResponse(); + it('should return a 400 VALIDATION_ERROR envelope (no SSE) when bank_name missing', async () => { + const res = await api.post(`${BASE}/csv/stream`).send({}).expect(400); - await expect(routeHandlers['post:/csv/stream'](req, res)).rejects.toBeInstanceOf(ValidationError); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); + expect(res.headers['content-type']).toMatch(/json/); }); - it('should emit SSE error event and end response on failure', async () => { + it('should emit an SSE error event on pipeline failure', async () => { runImportPipeline.mockRejectedValue(new Error('adapter failed')); - const req = { - file: { path: '/tmp/stream.csv', originalname: 'stream.csv' }, - query: { bank_name: 'belfius' }, - body: {}, - on: vi.fn(), - }; - const res = mockSseResponse(); - - await routeHandlers['post:/csv/stream'](req, res); + const res = await api.post(`${BASE}/csv/stream`).query({ bank_name: 'belfius' }).expect(200); - const writes = res.write.mock.calls.map(([payload]) => payload); - expect(writes.some(payload => payload.includes('event: error'))).toBe(true); - expect(writes.some(payload => payload.includes('Import failed'))).toBe(true); - expect(res.end).toHaveBeenCalledTimes(1); + const frames = parseSseFrames(res.text); + const errFrame = frames.find((f) => f.name === 'error'); + // A plain Error (not a ValidationError) maps to the generic detail — + // the raw 'adapter failed' message is deliberately not echoed to the + // client (importProgress.js's streamImport). + expect(errFrame.data).toEqual({ detail: 'Import failed' }); + expect(frames.some((f) => f.name === 'complete')).toBe(false); }); - it('should not emit complete or error event after client abort', async () => { - let closeHandler; + // The underlying "write()/end() are no-ops once the client has + // disconnected" invariant is unit-pinned in tests/sseWriter.test.js + // ("no-ops write() after client disconnect"); this is the integration-level + // regression check that a mid-stream client abort doesn't hang the + // in-flight pipeline or throw an unhandled rejection. + it('does not hang or throw when the client disconnects mid-stream', async () => { + let pipelineSettled = false; runImportPipeline.mockImplementation(async ({ onProgress }) => { await onProgress({ phase: 'importing', current: 1, total: 2, imported: 1, duplicates: 0, errors: 0, percent: 50 }); - closeHandler(); + await new Promise((resolve) => setTimeout(resolve, 30)); + pipelineSettled = true; return { total: 2, imported: 2, duplicates: 0, errors: 0 }; }); - const req = { - file: { path: '/tmp/stream.csv', originalname: 'stream.csv' }, - query: { bank_name: 'belfius' }, - body: {}, - on: vi.fn(), - }; - const res = mockSseResponse(); - res.on = vi.fn((event, cb) => { - if (event === 'close') closeHandler = cb; - }); - - await routeHandlers['post:/csv/stream'](req, res); + const test = api.post(`${BASE}/csv/stream`).query({ bank_name: 'belfius' }); + test.end(() => {}); // fire-and-forget: an aborted request rejects the client promise + await new Promise((resolve) => setTimeout(resolve, 15)); + test.abort(); - const writes = res.write.mock.calls.map(([payload]) => payload); - expect(writes.some(payload => payload.includes('event: complete'))).toBe(false); - expect(writes.some(payload => payload.includes('event: error'))).toBe(false); - expect(res.end).not.toHaveBeenCalled(); + await new Promise((resolve) => setTimeout(resolve, 45)); + expect(pipelineSettled).toBe(true); }); }); @@ -408,63 +367,29 @@ describe('Import Routes', () => { it('should return 201 with completed status on successful import', async () => { importRecipientsCSV.mockResolvedValue({ total_processed: 2, imported: 2, skipped: 0, errors: 0 }); - const req = { - file: { path: '/tmp/recipients.csv', originalname: 'recipients.csv' }, - query: {}, - body: {}, - }; - const res = mockResponse(); - - await routeHandlers['post:/recipients'](req, res); + const res = await api.post(`${BASE}/recipients`).send({}).expect(201); - expect(res.status).toHaveBeenCalledWith(201); - expect(res.json).toHaveBeenCalledWith(expect.objectContaining({ - ok: true, - data: expect.objectContaining({ status: 'completed' }), - })); + expect(res.body).toEqual(okEnvelope(expect.objectContaining({ status: 'completed' }))); }); it('should return 201 with completed_with_errors status when errors > 0', async () => { importRecipientsCSV.mockResolvedValue({ total_processed: 2, imported: 1, skipped: 0, errors: 1 }); - const req = { - file: { path: '/tmp/recipients.csv', originalname: 'recipients.csv' }, - query: {}, - body: {}, - }; - const res = mockResponse(); - - await routeHandlers['post:/recipients'](req, res); + const res = await api.post(`${BASE}/recipients`).send({}).expect(201); - expect(res.status).toHaveBeenCalledWith(201); - expect(res.json).toHaveBeenCalledWith(expect.objectContaining({ - ok: true, - data: expect.objectContaining({ status: 'completed_with_errors' }), - })); + expect(res.body).toEqual(okEnvelope(expect.objectContaining({ status: 'completed_with_errors' }))); }); it('should throw ValidationError for invalid separator', async () => { - const req = { - file: { path: '/tmp/recipients.csv', originalname: 'recipients.csv' }, - query: { separator: ';;' }, - body: {}, - }; - const res = mockResponse(); - - await expect(routeHandlers['post:/recipients'](req, res)).rejects.toThrow('separator must be a single character'); + const res = await api.post(`${BASE}/recipients`).query({ separator: ';;' }).send({}).expect(400); + expect(res.body.error.message).toMatch(/separator must be a single character/); }); it('should propagate service error', async () => { importRecipientsCSV.mockRejectedValue(new Error('boom')); - const req = { - file: { path: '/tmp/recipients.csv', originalname: 'recipients.csv' }, - query: {}, - body: {}, - }; - const res = mockResponse(); - - await expect(routeHandlers['post:/recipients'](req, res)).rejects.toThrow('boom'); + const res = await api.post(`${BASE}/recipients`).send({}).expect(500); + expect(res.body).toEqual(errEnvelope({ code: 'INTERNAL_SERVER_ERROR', message: 'boom' })); }); }); @@ -475,63 +400,29 @@ describe('Import Routes', () => { it('should return 201 with completed status on successful import', async () => { importCategoriesCSV.mockResolvedValue({ total_processed: 2, imported: 2, skipped: 0, errors: 0 }); - const req = { - file: { path: '/tmp/categories.csv', originalname: 'categories.csv' }, - query: {}, - body: {}, - }; - const res = mockResponse(); - - await routeHandlers['post:/categories'](req, res); + const res = await api.post(`${BASE}/categories`).send({}).expect(201); - expect(res.status).toHaveBeenCalledWith(201); - expect(res.json).toHaveBeenCalledWith(expect.objectContaining({ - ok: true, - data: expect.objectContaining({ status: 'completed' }), - })); + expect(res.body).toEqual(okEnvelope(expect.objectContaining({ status: 'completed' }))); }); it('should return 201 with completed_with_errors status when errors > 0', async () => { importCategoriesCSV.mockResolvedValue({ total_processed: 2, imported: 1, skipped: 0, errors: 1 }); - const req = { - file: { path: '/tmp/categories.csv', originalname: 'categories.csv' }, - query: {}, - body: {}, - }; - const res = mockResponse(); - - await routeHandlers['post:/categories'](req, res); + const res = await api.post(`${BASE}/categories`).send({}).expect(201); - expect(res.status).toHaveBeenCalledWith(201); - expect(res.json).toHaveBeenCalledWith(expect.objectContaining({ - ok: true, - data: expect.objectContaining({ status: 'completed_with_errors' }), - })); + expect(res.body).toEqual(okEnvelope(expect.objectContaining({ status: 'completed_with_errors' }))); }); it('should throw ValidationError for invalid separator', async () => { - const req = { - file: { path: '/tmp/categories.csv', originalname: 'categories.csv' }, - query: { separator: ';;' }, - body: {}, - }; - const res = mockResponse(); - - await expect(routeHandlers['post:/categories'](req, res)).rejects.toThrow('separator must be a single character'); + const res = await api.post(`${BASE}/categories`).query({ separator: ';;' }).send({}).expect(400); + expect(res.body.error.message).toMatch(/separator must be a single character/); }); it('should propagate service error', async () => { importCategoriesCSV.mockRejectedValue(new Error('boom')); - const req = { - file: { path: '/tmp/categories.csv', originalname: 'categories.csv' }, - query: {}, - body: {}, - }; - const res = mockResponse(); - - await expect(routeHandlers['post:/categories'](req, res)).rejects.toThrow('boom'); + const res = await api.post(`${BASE}/categories`).send({}).expect(500); + expect(res.body).toEqual(errEnvelope({ code: 'INTERNAL_SERVER_ERROR', message: 'boom' })); }); }); @@ -545,28 +436,22 @@ describe('Import Routes', () => { total: 1, }); - const req = { query: {} }; - const res = mockResponse(); - await routeHandlers['get:/batches'](req, res); + const res = await api.get(`${BASE}/batches`).expect(200); expect(listBatches).toHaveBeenCalledWith({ limit: 50, offset: 0 }); - const body = res.json.mock.calls[0][0]; - expect(body.ok).toBe(true); // Canonical collection shape: { items, total, limit, offset } — the old // `batches` key is gone from the wire (the service still uses it). - expect(body.data.batches).toBeUndefined(); - expect(body.data.items).toHaveLength(1); - expect(body.data.total).toBe(1); - expect(body.data.limit).toBe(50); - expect(body.data.offset).toBe(0); + expect(res.body.data.batches).toBeUndefined(); + expect(res.body).toEqual(okEnvelope({ + items: [{ id: 1, adapter_name: 'belfius', status: 'complete', rows_imported: 10 }], + total: 1, limit: 50, offset: 0, + })); }); it('clamps limit to 200', async () => { listBatches.mockResolvedValue({ batches: [], total: 0 }); - const req = { query: { limit: '999', offset: '0' } }; - const res = mockResponse(); - await routeHandlers['get:/batches'](req, res); + await api.get(`${BASE}/batches`).query({ limit: '999', offset: '0' }).expect(200); expect(listBatches).toHaveBeenCalledWith({ limit: 200, offset: 0 }); }); @@ -574,9 +459,7 @@ describe('Import Routes', () => { it('passes custom limit and offset', async () => { listBatches.mockResolvedValue({ batches: [], total: 5 }); - const req = { query: { limit: '10', offset: '20' } }; - const res = mockResponse(); - await routeHandlers['get:/batches'](req, res); + await api.get(`${BASE}/batches`).query({ limit: '10', offset: '20' }).expect(200); expect(listBatches).toHaveBeenCalledWith({ limit: 10, offset: 20 }); }); @@ -591,39 +474,29 @@ describe('Import Routes', () => { id: 7, adapter_name: 'kbc', status: 'complete', rows_imported: 20, transactions_remaining: 20, }); - const req = { params: { id: '7' } }; - const res = mockResponse(); - await routeHandlers['get:/batches/:id'](req, res); + const res = await api.get(`${BASE}/batches/7`).expect(200); expect(getBatch).toHaveBeenCalledWith(7); - const body = res.json.mock.calls[0][0]; - expect(body.ok).toBe(true); - expect(body.data.id).toBe(7); + expect(res.body.data.id).toBe(7); }); it('throws ValidationError for non-numeric id', async () => { - const req = { params: { id: 'abc' } }; - const res = mockResponse(); - - await expect(routeHandlers['get:/batches/:id'](req, res)).rejects.toBeInstanceOf(ValidationError); + const res = await api.get(`${BASE}/batches/abc`).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); expect(getBatch).not.toHaveBeenCalled(); }); it('throws ValidationError for a trailing-garbage id like "12abc" (was treated as batch 12)', async () => { - const req = { params: { id: '12abc' } }; - const res = mockResponse(); - - await expect(routeHandlers['get:/batches/:id'](req, res)).rejects.toBeInstanceOf(ValidationError); + const res = await api.get(`${BASE}/batches/12abc`).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); expect(getBatch).not.toHaveBeenCalled(); }); it('throws NotFoundError when batch does not exist', async () => { getBatch.mockResolvedValue(null); - const req = { params: { id: '999' } }; - const res = mockResponse(); - - await expect(routeHandlers['get:/batches/:id'](req, res)).rejects.toBeInstanceOf(NotFoundError); + const res = await api.get(`${BASE}/batches/999`).expect(404); + expect(res.body).toEqual(errEnvelope({ code: 'NOT_FOUND' })); }); }); @@ -635,41 +508,31 @@ describe('Import Routes', () => { getBatch.mockResolvedValue({ id: 3, status: 'complete' }); rollbackBatch.mockResolvedValue({ deleted: 15 }); - const req = { params: { id: '3' } }; - const res = mockResponse(); - await routeHandlers['delete:/batches/:id'](req, res); + const res = await api.delete(`${BASE}/batches/3`).expect(200); expect(rollbackBatch).toHaveBeenCalledWith(3); - const body = res.json.mock.calls[0][0]; - expect(body.ok).toBe(true); - expect(body.data.deleted).toBe(15); + expect(res.body.data.deleted).toBe(15); }); it('throws ValidationError for non-numeric id', async () => { - const req = { params: { id: 'nope' } }; - const res = mockResponse(); - - await expect(routeHandlers['delete:/batches/:id'](req, res)).rejects.toBeInstanceOf(ValidationError); + const res = await api.delete(`${BASE}/batches/nope`).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); expect(getBatch).not.toHaveBeenCalled(); }); it('throws NotFoundError when batch does not exist', async () => { getBatch.mockResolvedValue(null); - const req = { params: { id: '42' } }; - const res = mockResponse(); - - await expect(routeHandlers['delete:/batches/:id'](req, res)).rejects.toBeInstanceOf(NotFoundError); + const res = await api.delete(`${BASE}/batches/42`).expect(404); + expect(res.body).toEqual(errEnvelope({ code: 'NOT_FOUND' })); expect(rollbackBatch).not.toHaveBeenCalled(); }); it('throws ValidationError when batch already aborted', async () => { getBatch.mockResolvedValue({ id: 5, status: 'aborted' }); - const req = { params: { id: '5' } }; - const res = mockResponse(); - - await expect(routeHandlers['delete:/batches/:id'](req, res)).rejects.toBeInstanceOf(ValidationError); + const res = await api.delete(`${BASE}/batches/5`).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); expect(rollbackBatch).not.toHaveBeenCalled(); }); @@ -678,10 +541,8 @@ describe('Import Routes', () => { async (status) => { getBatch.mockResolvedValue({ id: 6, status }); - const req = { params: { id: '6' } }; - const res = mockResponse(); - - await expect(routeHandlers['delete:/batches/:id'](req, res)).rejects.toBeInstanceOf(ValidationError); + const res = await api.delete(`${BASE}/batches/6`).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); expect(rollbackBatch).not.toHaveBeenCalled(); } ); @@ -690,67 +551,44 @@ describe('Import Routes', () => { getBatch.mockResolvedValue({ id: 8, status: 'complete' }); rollbackBatch.mockResolvedValue({ deleted: 0 }); - const req = { params: { id: '8' } }; - const res = mockResponse(); - await routeHandlers['delete:/batches/:id'](req, res); + const res = await api.delete(`${BASE}/batches/8`).expect(200); - const body = res.json.mock.calls[0][0]; - expect(body.data.deleted).toBe(0); + expect(res.body.data.deleted).toBe(0); }); }); + // ────────────────────────────────────────── + // multer error middleware (csvUploadErrorTranslator, importRoutes.js:580) + // ────────────────────────────────────────── describe('multer error middleware', () => { - it('should forward LIMIT_FILE_SIZE as ValidationError', () => { - const errorHandler = routeHandlers.use.at(-1); - const err = new multer.MulterError('LIMIT_FILE_SIZE'); - const req = {}; - const res = mockResponse(); - const next = vi.fn(); - - errorHandler(err, req, res, next); + it('maps LIMIT_FILE_SIZE to a 400 VALIDATION_ERROR envelope', async () => { + uploadState.error = new multer.MulterError('LIMIT_FILE_SIZE'); - expect(next).toHaveBeenCalledWith(expect.any(ValidationError)); - expect(res.status).not.toHaveBeenCalled(); + const res = await api.post(`${BASE}/csv`).query({ bank_name: 'belfius' }).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR', message: 'File size exceeds maximum of 50MB' })); }); - it('should forward generic multer errors as ValidationError', () => { - const errorHandler = routeHandlers.use.at(-1); + it('maps other multer errors to a 400 VALIDATION_ERROR envelope', async () => { const err = new multer.MulterError('LIMIT_UNEXPECTED_FILE'); err.message = 'unexpected file'; - const req = {}; - const res = mockResponse(); - const next = vi.fn(); + uploadState.error = err; - errorHandler(err, req, res, next); - - expect(next).toHaveBeenCalledWith(expect.any(ValidationError)); - expect(res.status).not.toHaveBeenCalled(); + const res = await api.post(`${BASE}/csv`).query({ bank_name: 'belfius' }).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR', message: 'Upload error: unexpected file' })); }); - it('should forward non-csv file errors as ValidationError', () => { - const errorHandler = routeHandlers.use.at(-1); - const err = new Error('File must be a CSV'); - const req = {}; - const res = mockResponse(); - const next = vi.fn(); - - errorHandler(err, req, res, next); + it('maps a non-CSV fileFilter rejection to a 400 VALIDATION_ERROR envelope', async () => { + uploadState.error = new Error('File must be a CSV'); - expect(next).toHaveBeenCalledWith(expect.any(ValidationError)); - expect(res.status).not.toHaveBeenCalled(); + const res = await api.post(`${BASE}/csv`).query({ bank_name: 'belfius' }).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR', message: 'File must be a CSV' })); }); - it('should pass unknown errors to next middleware', () => { - const errorHandler = routeHandlers.use.at(-1); - const err = new Error('unknown'); - const req = {}; - const res = mockResponse(); - const next = vi.fn(); - - errorHandler(err, req, res, next); + it('passes unrelated errors through to the generic 500 handler', async () => { + uploadState.error = new Error('unknown'); - expect(next).toHaveBeenCalledWith(err); - expect(res.status).not.toHaveBeenCalled(); + const res = await api.post(`${BASE}/csv`).query({ bank_name: 'belfius' }).expect(500); + expect(res.body).toEqual(errEnvelope({ code: 'INTERNAL_SERVER_ERROR', message: 'unknown' })); }); }); @@ -758,65 +596,53 @@ describe('Import Routes', () => { // POST /api/import/batches/:id/rows/:rowId/category-override // ────────────────────────────────────────── describe('POST /batches/:id/rows/:rowId/category-override', () => { - const route = 'post:/batches/:id/rows/:rowId/category-override'; + const url = (id, rowId) => `${BASE}/batches/${id}/rows/${rowId}/category-override`; it('rejects non-numeric ids', async () => { - const req = { params: { id: 'abc', rowId: '5' }, body: { category_id: 1 } }; - const res = mockResponse(); - await expect(routeHandlers[route](req, res)).rejects.toBeInstanceOf(ValidationError); + const res = await api.post(url('abc', '5')).send({ category_id: 1 }).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); }); it('rejects non-integer category_id', async () => { - const req = { params: { id: '1', rowId: '5' }, body: { category_id: 'foo' } }; - const res = mockResponse(); - await expect(routeHandlers[route](req, res)).rejects.toBeInstanceOf(ValidationError); + const res = await api.post(url('1', '5')).send({ category_id: 'foo' }).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); }); it('clears the override when category_id is null', async () => { overrideCategory.mockResolvedValueOnce(1); - const req = { params: { id: '1', rowId: '5' }, body: { category_id: null } }; - const res = mockResponse(); - await routeHandlers[route](req, res); + const res = await api.post(url('1', '5')).send({ category_id: null }).expect(200); expect(categoryExists).not.toHaveBeenCalled(); expect(overrideCategory).toHaveBeenCalledTimes(1); expect(overrideCategory).toHaveBeenCalledWith({ batchId: 1, rowId: 5, categoryId: null }); - - const body = res.json.mock.calls[0][0]; - expect(body.data).toEqual({ row_id: 5, override_category_id: null }); + expect(res.body.data).toEqual({ row_id: 5, override_category_id: null }); }); it('sets the override after verifying category exists', async () => { categoryExists.mockResolvedValueOnce(true); overrideCategory.mockResolvedValueOnce(1); - const req = { params: { id: '7', rowId: '42' }, body: { category_id: 12 } }; - const res = mockResponse(); - await routeHandlers[route](req, res); + const res = await api.post(url('7', '42')).send({ category_id: 12 }).expect(200); expect(categoryExists).toHaveBeenCalledWith(12); expect(overrideCategory).toHaveBeenCalledWith({ batchId: 7, rowId: 42, categoryId: 12 }); - - const body = res.json.mock.calls[0][0]; - expect(body.data).toEqual({ row_id: 42, override_category_id: 12 }); + expect(res.body.data).toEqual({ row_id: 42, override_category_id: 12 }); }); it('rejects unknown category_id with ValidationError', async () => { categoryExists.mockResolvedValueOnce(false); - const req = { params: { id: '1', rowId: '5' }, body: { category_id: 9999 } }; - const res = mockResponse(); - await expect(routeHandlers[route](req, res)).rejects.toBeInstanceOf(ValidationError); + const res = await api.post(url('1', '5')).send({ category_id: 9999 }).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); }); it('throws NotFoundError when staging row not found or wrong status', async () => { categoryExists.mockResolvedValueOnce(true); overrideCategory.mockResolvedValueOnce(0); - const req = { params: { id: '1', rowId: '5' }, body: { category_id: 1 } }; - const res = mockResponse(); - await expect(routeHandlers[route](req, res)).rejects.toBeInstanceOf(NotFoundError); + const res = await api.post(url('1', '5')).send({ category_id: 1 }).expect(404); + expect(res.body).toEqual(errEnvelope({ code: 'NOT_FOUND' })); }); }); @@ -824,8 +650,6 @@ describe('Import Routes', () => { // GET /api/import/batches/:id/preview — category fields // ────────────────────────────────────────── describe('GET /batches/:id/preview category fields', () => { - const route = 'get:/batches/:id/preview'; - it('exposes recipient_default_category_id and current_category_label per group', async () => { getBatch.mockResolvedValue({ id: 1, status: 'awaiting_review' }); getPreviewRows.mockResolvedValueOnce([ @@ -855,13 +679,10 @@ describe('Import Routes', () => { }, ]); - const req = { params: { id: '1' } }; - const res = mockResponse(); - await routeHandlers[route](req, res); + const res = await api.get(`${BASE}/batches/1/preview`).expect(200); - const body = res.json.mock.calls[0][0]; - expect(body.data.groups).toHaveLength(1); - const g = body.data.groups[0]; + expect(res.body.data.groups).toHaveLength(1); + const g = res.body.data.groups[0]; expect(g.recipient_default_category_id).toBe(12); expect(g.recipient_default_category_label).toBe('Groceries: Food'); expect(g.current_category_id).toBe(12); @@ -898,11 +719,9 @@ describe('Import Routes', () => { }, ]); - const req = { params: { id: '1' } }; - const res = mockResponse(); - await routeHandlers[route](req, res); + const res = await api.get(`${BASE}/batches/1/preview`).expect(200); - const g = res.json.mock.calls[0][0].data.groups[0]; + const g = res.body.data.groups[0]; expect(g.override_category_id).toBe(22); expect(g.current_category_id).toBe(22); expect(g.current_category_label).toBe('Hardware: Tools'); @@ -910,7 +729,6 @@ describe('Import Routes', () => { }); }); - // ────────────────────────────────────────── // GET /api/import/supported-banks was removed (dead route; adapter catalog is // served from /api/info/supported-adapters, registry-derived — see info.test.js). }); @@ -924,20 +742,20 @@ describe('Saved custom parser routes', () => { it('returns the list of saved parsers', async () => { const items = [{ id: 1, name: 'My Bank', config: validConfig }]; customParserConfigRepository.getAll.mockResolvedValue(items); - const res = mockResponse(); - await routeHandlers['get:/parsers']({ query: {} }, res); + + const res = await api.get(`${BASE}/parsers`).expect(200); // Canonical collection shape: { items, total } (total = row count here). - expect(res.json.mock.calls[0][0].data).toEqual({ items, total: 1 }); + expect(res.body.data).toEqual({ items, total: 1 }); }); }); describe('POST /parsers', () => { it('creates a parser and returns 201', async () => { customParserConfigRepository.create.mockResolvedValue({ id: 7, name: 'My Bank', config: validConfig }); - const res = mockResponse(); - await routeHandlers['post:/parsers']({ body: { name: 'My Bank', config: validConfig } }, res); - expect(res.status).toHaveBeenCalledWith(201); - expect(res.json.mock.calls[0][0].data.id).toBe(7); + + const res = await api.post(`${BASE}/parsers`).send({ name: 'My Bank', config: validConfig }).expect(201); + + expect(res.body.data.id).toBe(7); expect(customParserConfigRepository.create).toHaveBeenCalledWith({ name: 'My Bank', config: expect.objectContaining({ dateColumn: 'Date', dateFormat: '%Y-%m-%d', separator: ',', encoding: 'utf-8', skipRows: 0 }), @@ -946,82 +764,58 @@ describe('Saved custom parser routes', () => { }); it('rejects a missing name', async () => { - const res = mockResponse(); - await expect( - routeHandlers['post:/parsers']({ body: { config: validConfig } }, res), - ).rejects.toBeInstanceOf(ValidationError); + const res = await api.post(`${BASE}/parsers`).send({ config: validConfig }).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); }); it('rejects a config missing required columns', async () => { - const res = mockResponse(); - await expect( - routeHandlers['post:/parsers']({ body: { name: 'X', config: { dateColumn: 'Date' } } }, res), - ).rejects.toBeInstanceOf(ValidationError); + const res = await api.post(`${BASE}/parsers`).send({ name: 'X', config: { dateColumn: 'Date' } }).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); }); it('maps a unique-violation to ConflictError', async () => { const err = Object.assign(new Error('dup'), { code: '23505', constraint: 'uq_custom_parser_configs_name_kind' }); customParserConfigRepository.create.mockRejectedValue(err); - const res = mockResponse(); - await expect( - routeHandlers['post:/parsers']({ body: { name: 'My Bank', config: validConfig } }, res), - ).rejects.toBeInstanceOf(ConflictError); + + const res = await api.post(`${BASE}/parsers`).send({ name: 'My Bank', config: validConfig }).expect(409); + expect(res.body).toEqual(errEnvelope({ code: 'CONFLICT' })); }); }); describe('PATCH /parsers/:id', () => { it('returns 404 when the parser does not exist', async () => { customParserConfigRepository.update.mockResolvedValue(undefined); - const res = mockResponse(); - await expect( - routeHandlers['patch:/parsers/:id']({ params: { id: '5' }, body: { name: 'New' } }, res), - ).rejects.toBeInstanceOf(NotFoundError); + + const res = await api.patch(`${BASE}/parsers/5`).send({ name: 'New' }).expect(404); + expect(res.body).toEqual(errEnvelope({ code: 'NOT_FOUND' })); }); it('maps a unique-violation to ConflictError', async () => { const err = Object.assign(new Error('dup'), { code: '23505', constraint: 'uq_custom_parser_configs_name_kind' }); customParserConfigRepository.update.mockRejectedValue(err); - const res = mockResponse(); - await expect( - routeHandlers['patch:/parsers/:id']({ params: { id: '5' }, body: { name: 'Taken' } }, res), - ).rejects.toBeInstanceOf(ConflictError); + + const res = await api.patch(`${BASE}/parsers/5`).send({ name: 'Taken' }).expect(409); + expect(res.body).toEqual(errEnvelope({ code: 'CONFLICT' })); }); it('rejects an invalid id', async () => { - const res = mockResponse(); - await expect( - routeHandlers['patch:/parsers/:id']({ params: { id: 'abc' }, body: { name: 'X' } }, res), - ).rejects.toBeInstanceOf(ValidationError); + const res = await api.patch(`${BASE}/parsers/abc`).send({ name: 'X' }).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); }); }); describe('DELETE /parsers/:id', () => { it('returns 204 when deleted', async () => { customParserConfigRepository.delete.mockResolvedValue(true); - const res = mockResponse(); - await routeHandlers['delete:/parsers/:id']({ params: { id: '3' } }, res); - expect(res.status).toHaveBeenCalledWith(204); - expect(res.send).toHaveBeenCalled(); + + await api.delete(`${BASE}/parsers/3`).expect(204); }); it('returns 404 when nothing was deleted', async () => { customParserConfigRepository.delete.mockResolvedValue(false); - const res = mockResponse(); - await expect( - routeHandlers['delete:/parsers/:id']({ params: { id: '3' } }, res), - ).rejects.toBeInstanceOf(NotFoundError); + + const res = await api.delete(`${BASE}/parsers/3`).expect(404); + expect(res.body).toEqual(errEnvelope({ code: 'NOT_FOUND' })); }); }); }); - -function mockResponse() { - return createMockResponse(); -} - -function mockSseResponse() { - return { - writeHead: vi.fn(), - write: vi.fn(), - end: vi.fn(), - }; -} diff --git a/apps/node-backend/tests/routes/importValidationPins.test.js b/apps/node-backend/tests/routes/importValidationPins.test.js index b82d1b11..f7b936df 100644 --- a/apps/node-backend/tests/routes/importValidationPins.test.js +++ b/apps/node-backend/tests/routes/importValidationPins.test.js @@ -1,23 +1,24 @@ /** * Validation-behavior pins for importRoutes.js (ZOD-07). * - * Pins the exact accept/reject/coercion behavior of the hand-rolled request - * parsing BEFORE the zod migration: the Number()-coerced batch-id guard - * (copy-pasted per route), the multipart CSV option coercion - * (parseCsvImportOptions), the csv/custom required/trim/default build, and - * normalizeParserConfig's defaults — so the swap cannot change the wire. + * Pins the exact accept/reject/coercion behavior of the request parsing: the + * Number()-coerced batch-id guard (copy-pasted per route), the multipart CSV + * option coercion (parseCsvImportOptions), the csv/custom required/trim/default + * build, and normalizeParserConfig's defaults — so a change cannot alter the wire. + * + * Driven over HTTP against the real router (tests/helpers/routeApp.js), which + * also puts the router's own trailing error middleware + * (`router.use(csvUploadErrorTranslator)`, routes/importRoutes.js:580) on the + * tested path — the mock-router harness dropped it entirely. + * + * multer is still stubbed to a pass-through (no real multipart parsing); the + * uploaded file is injected by a `before` middleware, which is the same slot + * `main.js` uses for per-mount middleware. */ import { describe, it, expect, vi, beforeEach } from 'vitest'; import { mockConnection } from '../helpers/repoMocks.js'; import { mockLogger } from '../helpers/mockLogger.js'; -import { createMockRouter, createMockResponse } from '../helpers/routeHarness.js'; - -const { router: mockRouter, handlers: routeHandlers } = createMockRouter(); - -vi.mock('express', () => ({ - default: { Router: () => mockRouter }, - Router: () => mockRouter, -})); +import { routeAgent } from '../helpers/routeApp.js'; vi.mock('multer', () => { const multer = vi.fn(() => ({ @@ -81,15 +82,29 @@ vi.mock('../../src/repositories/customParserConfigRepository.js', () => ({ vi.mock('../../src/database/connection.js', () => mockConnection()); -import { runImportPipeline } from '../../src/services/importPipeline/index.js'; +import { runImportPipeline, commitImport } from '../../src/services/importPipeline/index.js'; +// NOT mocked: only .../importPipeline/index.js is. This is the real boundary +// function, run here over the mocked pg connection. +import { createBatch } from '../../src/services/importPipeline/stage.js'; import { importRecipientsCSV } from '../../src/services/dataImportService.js'; -import { getBatch, overrideRecipient } from '../../src/repositories/importBatchRepository.js'; +import { getBatch, getPreviewRows, overrideRecipient } from '../../src/repositories/importBatchRepository.js'; +import { query as dbQuery } from '../../src/database/connection.js'; import customParserConfigRepository from '../../src/repositories/customParserConfigRepository.js'; -import { ValidationError, NotFoundError } from '../../src/middleware/errorHandler.js'; -await import('../../src/routes/importRoutes.js'); -const mockResponse = () => createMockResponse(); -const file = () => ({ path: '/tmp/pin.csv', originalname: 'pin.csv', size: 10 }); +const { default: importRouter } = await import('../../src/routes/importRoutes.js'); + +const UPLOAD = { path: '/tmp/pin.csv', originalname: 'pin.csv', size: 10 }; + +const BASE = '/api/import'; +// multer is stubbed, so nothing populates req.file — inject it in the same +// per-mount slot main.js uses (main.js:325 mounts importRateLimiter there). +const api = routeAgent(importRouter, { + mountPath: BASE, + before: [(req, _res, next) => { req.file = { ...UPLOAD }; next(); }], +}); + +/** Encode a path segment so ids with spaces survive the URL round-trip. */ +const seg = (v) => encodeURIComponent(String(v)); beforeEach(() => { vi.clearAllMocks(); @@ -102,30 +117,37 @@ describe('batch-id coercion pins (Number() then integer > 0)', () => { it("accepts '12.0', ' 12 ' and '0x10' exactly like Number() coercion", async () => { getBatch.mockResolvedValue({ id: 12, status: 'complete' }); - await routeHandlers['get:/batches/:id']({ params: { id: '12.0' } }, mockResponse()); + await api.get(`${BASE}/batches/${seg('12.0')}`).expect(200); expect(getBatch).toHaveBeenLastCalledWith(12); - await routeHandlers['get:/batches/:id']({ params: { id: ' 12 ' } }, mockResponse()); + await api.get(`${BASE}/batches/${seg(' 12 ')}`).expect(200); expect(getBatch).toHaveBeenLastCalledWith(12); - await routeHandlers['get:/batches/:id']({ params: { id: '0x10' } }, mockResponse()); + await api.get(`${BASE}/batches/${seg('0x10')}`).expect(200); expect(getBatch).toHaveBeenLastCalledWith(16); }); it('does NOT 400 an absurdly large integral id — it 404s downstream', async () => { getBatch.mockResolvedValue(undefined); - await expect(routeHandlers['get:/batches/:id']({ params: { id: '1e300' } }, mockResponse())) - .rejects.toBeInstanceOf(NotFoundError); + + const res = await api.get(`${BASE}/batches/1e300`).expect(404); + + expect(res.body.error.code).toBe('NOT_FOUND'); expect(getBatch).toHaveBeenCalledWith(1e300); }); it('rejects fractional, trailing-garbage, zero, and negative ids on every site', async () => { const badIds = ['12.5', '12abc', '0', '-1', 'abc']; - const sites = ['get:/batches/:id', 'delete:/batches/:id', 'get:/batches/:id/preview', 'post:/batches/:id/commit']; + const sites = [ + (id) => api.get(`${BASE}/batches/${seg(id)}`), + (id) => api.delete(`${BASE}/batches/${seg(id)}`), + (id) => api.get(`${BASE}/batches/${seg(id)}/preview`), + (id) => api.post(`${BASE}/batches/${seg(id)}/commit`).send({}), + ]; for (const site of sites) { for (const id of badIds) { - await expect(routeHandlers[site]({ params: { id }, body: {} }, mockResponse())) - .rejects.toBeInstanceOf(ValidationError); + const res = await site(id).expect(400); + expect(res.body.error.code).toBe('VALIDATION_ERROR'); } } expect(getBatch).not.toHaveBeenCalled(); @@ -133,59 +155,60 @@ describe('batch-id coercion pins (Number() then integer > 0)', () => { it('row-override sites validate both ids and coerce them', async () => { overrideRecipient.mockResolvedValue(1); - await routeHandlers['post:/batches/:id/rows/:rowId/override']( - { params: { id: '5', rowId: ' 6 ' }, body: { recipient_id: null } }, - mockResponse(), - ); + + await api.post(`${BASE}/batches/5/rows/${seg(' 6 ')}/override`) + .send({ recipient_id: null }) + .expect(200); expect(overrideRecipient).toHaveBeenCalledWith({ batchId: 5, rowId: 6, recipientId: null }); - for (const params of [{ id: '5', rowId: '0' }, { id: 'abc', rowId: '6' }, { id: '5.5', rowId: '6' }]) { - await expect( - routeHandlers['post:/batches/:id/rows/:rowId/override']({ params, body: {} }, mockResponse()), - ).rejects.toBeInstanceOf(ValidationError); - await expect( - routeHandlers['post:/batches/:id/rows/:rowId/category-override']({ params, body: {} }, mockResponse()), - ).rejects.toBeInstanceOf(ValidationError); + for (const { id, rowId } of [ + { id: '5', rowId: '0' }, + { id: 'abc', rowId: '6' }, + { id: '5.5', rowId: '6' }, + ]) { + await api.post(`${BASE}/batches/${seg(id)}/rows/${seg(rowId)}/override`).send({}).expect(400); + await api.post(`${BASE}/batches/${seg(id)}/rows/${seg(rowId)}/category-override`).send({}).expect(400); } }); }); describe('parseCsvImportOptions pins (POST /recipients)', () => { const run = (query, body = {}) => - routeHandlers['post:/recipients']({ file: file(), query, body }, mockResponse()); + api.post(`${BASE}/recipients`).query(query).send(body); it('defaults separator to "," and encoding to "utf-8"', async () => { - await run({}); + await run({}).expect(201); expect(importRecipientsCSV).toHaveBeenCalledWith('/tmp/pin.csv', { separator: ',', encoding: 'utf-8' }); }); it('query wins over body; body is the fallback', async () => { - await run({ separator: ';' }, { separator: '|' }); + await run({ separator: ';' }, { separator: '|' }).expect(201); expect(importRecipientsCSV).toHaveBeenLastCalledWith('/tmp/pin.csv', { separator: ';', encoding: 'utf-8' }); - await run({}, { separator: '|', encoding: 'latin1' }); + await run({}, { separator: '|', encoding: 'latin1' }).expect(201); expect(importRecipientsCSV).toHaveBeenLastCalledWith('/tmp/pin.csv', { separator: '|', encoding: 'latin1' }); }); it('stringifies a numeric separator (multipart/JSON tolerance)', async () => { - await run({}, { separator: 5 }); + await run({}, { separator: 5 }).expect(201); expect(importRecipientsCSV).toHaveBeenLastCalledWith('/tmp/pin.csv', { separator: '5', encoding: 'utf-8' }); }); it('empty-string separator falls back to the default', async () => { - await run({ separator: '' }); + await run({ separator: '' }).expect(201); expect(importRecipientsCSV).toHaveBeenLastCalledWith('/tmp/pin.csv', { separator: ',', encoding: 'utf-8' }); }); it('rejects a multi-character separator', async () => { - await expect(run({ separator: ';;' })).rejects.toThrow(/separator/); + const res = await run({ separator: ';;' }).expect(400); + expect(res.body.error.message).toMatch(/separator/); expect(importRecipientsCSV).not.toHaveBeenCalled(); }); }); describe('POST /csv/custom config-build pins', () => { const run = (query, body = {}) => - routeHandlers['post:/csv/custom']({ file: file(), query, body }, mockResponse()); + api.post(`${BASE}/csv/custom`).query(query).send(body); const requiredQuery = { bank_name: 'Custom', date_format: '%d/%m/%Y', @@ -197,7 +220,7 @@ describe('POST /csv/custom config-build pins', () => { bank_name: ' My Bank ', date_format: ' %d/%m/%Y ', date_column: ' Date ', recipient_column: ' Desc ', amount_column: ' Amt ', memo_column: ' Memo ', separator: ';', encoding: 'latin1', skip_rows: '2.9', - }); + }).expect(201); expect(runImportPipeline).toHaveBeenCalledWith(expect.objectContaining({ adapterName: ' My Bank ', customConfig: { @@ -212,7 +235,7 @@ describe('POST /csv/custom config-build pins', () => { }); it('applies defaults: memo "", separator ",", encoding utf-8, skip_rows 0', async () => { - await run(requiredQuery); + await run(requiredQuery).expect(201); expect(runImportPipeline).toHaveBeenCalledWith(expect.objectContaining({ customConfig: expect.objectContaining({ encoding: 'utf-8', separator: ',', skip_rows: 0, @@ -222,14 +245,14 @@ describe('POST /csv/custom config-build pins', () => { }); it('coerces unparseable skip_rows to 0 and accepts an empty separator as default', async () => { - await run({ ...requiredQuery, skip_rows: 'abc', separator: '' }); + await run({ ...requiredQuery, skip_rows: 'abc', separator: '' }).expect(201); expect(runImportPipeline).toHaveBeenCalledWith(expect.objectContaining({ customConfig: expect.objectContaining({ skip_rows: 0, separator: ',' }), })); }); it('body fields override query fields', async () => { - await run(requiredQuery, { amount_column: 'BodyAmt' }); + await run(requiredQuery, { amount_column: 'BodyAmt' }).expect(201); expect(runImportPipeline).toHaveBeenCalledWith(expect.objectContaining({ customConfig: expect.objectContaining({ column_mapping: expect.objectContaining({ amount: 'BodyAmt' }), @@ -239,14 +262,13 @@ describe('POST /csv/custom config-build pins', () => { }); describe('normalizeParserConfig pins (POST /parsers)', () => { - const create = (config) => - routeHandlers['post:/parsers']({ body: { name: 'P', config } }, mockResponse()); + const create = (config) => api.post(`${BASE}/parsers`).send({ name: 'P', config }); const base = { dateColumn: 'Date', recipientColumn: 'Name', amountColumn: 'Amount' }; it('coerces skipRows: numeric strings parse, floats floor, negatives become 0', async () => { for (const [input, expected] of [['3', 3], [-5, 0], ['2.9', 2], ['abc', 0]]) { - await create({ ...base, skipRows: input }); + await create({ ...base, skipRows: input }).expect(201); expect(customParserConfigRepository.create).toHaveBeenLastCalledWith( expect.objectContaining({ config: expect.objectContaining({ skipRows: expected }) }), ); @@ -254,14 +276,14 @@ describe('normalizeParserConfig pins (POST /parsers)', () => { }); it('does NOT enforce a single-char separator here — any non-empty string sticks', async () => { - await create({ ...base, separator: ';;' }); + await create({ ...base, separator: ';;' }).expect(201); expect(customParserConfigRepository.create).toHaveBeenLastCalledWith( expect.objectContaining({ config: expect.objectContaining({ separator: ';;' }) }), ); }); it('strips unknown keys and blanks a non-string memoColumn', async () => { - await create({ ...base, foo: 'bar', memoColumn: 123 }); + await create({ ...base, foo: 'bar', memoColumn: 123 }).expect(201); const { config } = customParserConfigRepository.create.mock.calls.at(-1)[0]; expect('foo' in config).toBe(false); expect(config.memoColumn).toBe(''); @@ -272,15 +294,88 @@ describe('normalizeParserConfig pins (POST /parsers)', () => { }); it('rejects missing required columns with the per-key message', async () => { - await expect(create({ dateColumn: 'Date', amountColumn: 'A' })) - .rejects.toThrow('config.recipientColumn is required'); - await expect(create({ ...base, dateColumn: ' ' })) - .rejects.toThrow('config.dateColumn is required'); + const res1 = await create({ dateColumn: 'Date', amountColumn: 'A' }).expect(400); + expect(res1.body.error.message).toContain('config.recipientColumn is required'); + + const res2 = await create({ ...base, dateColumn: ' ' }).expect(400); + expect(res2.body.error.message).toContain('config.dateColumn is required'); }); it('rejects a non-object or array config', async () => { for (const config of [null, 'str', [1]]) { - await expect(create(config)).rejects.toThrow('Missing or invalid "config"'); + const res = await create(config).expect(400); + expect(res.body.error.message).toContain('Missing or invalid "config"'); } }); }); + +/** + * `batch_id` wire type. + * + * The immediate-import responses (`POST /csv`, 201 and the 202 review variant) + * relay `pipelineResult.batchId` straight from `createBatch`, while the + * review-commit route re-reads the id off the URL through `coercedIdSchema`. + * Those two producers disagreed until `createBatch` was normalized + * (services/importPipeline/stage.js — pinned by tests/importPipeline.stage.test.js), + * so `batch_id` was a string on one response and a number on the other and a + * client doing `a.batch_id === b.batch_id` across them always saw false. + * NUMBER is the single wire type; these pin every response that carries the field. + */ +describe('batch_id wire type', () => { + const BATCH_ID = 12; + + // The pipeline is mocked here, but its `batchId` is produced by the REAL + // `createBatch` running over the mocked pg connection primed with what + // node-postgres actually returns for a BIGSERIAL: the STRING '12'. That keeps + // these route pins honest — before the stage-boundary fix they failed with + // `batch_id: "12"`, exactly the wire split the finding describes. + const realBatchId = async () => { + dbQuery.mockResolvedValueOnce({ rows: [{ id: String(BATCH_ID) }] }); + return createBatch({ adapterName: 'vision' }); + }; + + it('POST /csv (201, committed) emits a numeric batch_id', async () => { + runImportPipeline.mockImplementation(async () => ({ + batchId: await realBatchId(), total: 1, imported: 1, duplicates: 0, errors: 0, + })); + + const res = await api.post(`${BASE}/csv`).query({ bank_name: 'vision' }).expect(201); + + expect(typeof res.body.data.batch_id).toBe('number'); + expect(res.body.data.batch_id).toBe(BATCH_ID); + }); + + it('POST /csv (202, review required) emits a numeric batch_id', async () => { + runImportPipeline.mockImplementation(async () => ({ + batchId: await realBatchId(), requiresReview: true, matchSourceCounts: { exact: 1 }, + })); + + const res = await api.post(`${BASE}/csv`).query({ bank_name: 'vision' }).expect(202); + + expect(typeof res.body.data.batch_id).toBe('number'); + expect(res.body.data.batch_id).toBe(BATCH_ID); + }); + + it('POST /batches/:id/commit emits the SAME type and value for the same batch', async () => { + runImportPipeline.mockImplementation(async () => ({ + batchId: await realBatchId(), requiresReview: true, matchSourceCounts: {}, + })); + const started = await api.post(`${BASE}/csv`).query({ bank_name: 'vision' }).expect(202); + + getBatch.mockResolvedValue({ id: BATCH_ID, status: 'awaiting_review' }); + commitImport.mockResolvedValue({ imported: 1, duplicates: 0, errors: 0, autoLinkedCount: 0 }); + const committed = await api.post(`${BASE}/batches/${BATCH_ID}/commit`).send({}).expect(200); + + expect(typeof committed.body.data.batch_id).toBe('number'); + // The whole point of the finding: strict equality across the two responses. + expect(committed.body.data.batch_id).toStrictEqual(started.body.data.batch_id); + }); + + it('GET /batches/:id/preview also emits a numeric batch_id', async () => { + getPreviewRows.mockResolvedValue([]); + + const res = await api.get(`${BASE}/batches/${BATCH_ID}/preview`).expect(200); + + expect(typeof res.body.data.batch_id).toBe('number'); + }); +}); diff --git a/apps/node-backend/tests/routes/info.test.js b/apps/node-backend/tests/routes/info.test.js index 5f916391..b95f6702 100644 --- a/apps/node-backend/tests/routes/info.test.js +++ b/apps/node-backend/tests/routes/info.test.js @@ -1,17 +1,37 @@ /** * Info/Statistics route tests. * Mirrors: apps/backend/tests/test_info.py + * + * Runs against the REAL router mounted on a throwaway Express app (see + * tests/helpers/routeApp.js). `info.js` is a barrel over six sub-routers + * (statistics/netWorth/rates/performance/portfolioSummary/maintenance); the + * old mock-router harness flattened all of them into one handler map by + * aliasing every nested `Router()` call to the same stub — which also meant + * every per-route guard (`rateLimiter`/`adminRateLimiter` declared INSIDE + * netWorth.js, rates.js, performance.js, portfolioSummary.js) was silently + * dropped. Those limiters are real now; this suite's request counts per + * keyPrefix stay far under every limit (30-500/60s) so nothing 429s. + * + * Mount path is /api/info — no app-level per-mount middleware (main.js:322). + * + * Several handlers here cascade into real (unmocked) services — + * getPortfolioSummary, resolveLivePortfolioValue, settingsRepository, + * portfolioTransactionRepository, the adapter registry — exactly as they did + * under the old harness (only Express itself was mocked there, never these + * imports). `database/connection.js`'s `query` is mocked to `{rows: []}` by + * default so that cascade resolves to an empty-but-valid summary instead of + * hitting a real DB. + * + * Dropped: "should register /refresh-views and /inflation-rates/refresh + * routes" — that was a structural check against the mock router's handler + * map (`routeHandlers['post:/refresh-views']` etc. being a function), which + * has no equivalent against a real router and is redundant with the + * dedicated POST /refresh-views and POST /inflation-rates/refresh tests + * below, which already prove the routes exist and work. */ import { describe, it, expect, vi, beforeEach } from 'vitest'; import { mockLogger } from '../helpers/mockLogger.js'; -import { createMockRouter, createMockResponse } from '../helpers/routeHarness.js'; - -const { router: mockRouter, handlers: routeHandlers } = createMockRouter(); - -vi.mock('express', () => ({ - default: { Router: () => mockRouter }, - Router: () => mockRouter, -})); +import { routeAgent, okEnvelope, errEnvelope } from '../helpers/routeApp.js'; vi.mock('../../src/repositories/infoRepository.js', () => ({ default: { @@ -80,8 +100,12 @@ vi.mock('../../src/services/portfolioPerformanceSnapshotService.js', () => ({ import infoRepository from '../../src/repositories/infoRepository.js'; import { logger } from '../../src/config/logger.js'; +const { default: infoRouter } = await import('../../src/routes/info.js'); const { warmInfoCaches } = await import('../../src/routes/info.js'); +const BASE = '/api/info'; +const api = routeAgent(infoRouter, { mountPath: BASE }); + describe('Info Routes', () => { beforeEach(() => { vi.clearAllMocks(); @@ -93,17 +117,10 @@ describe('Info Routes', () => { mockGetSnapshots.mockResolvedValue([]); }); - it('should register /refresh-views and /inflation-rates/refresh routes', () => { - expect(routeHandlers['post:/refresh-views']).toBeTypeOf('function'); - expect(routeHandlers['post:/inflation-rates/refresh']).toBeTypeOf('function'); - }); - describe('GET /supported-adapters', () => { it('serves the registry-derived adapter list (no hardcoded drift)', async () => { - const req = { query: {} }; - const res = mockResponse(); - await routeHandlers['get:/supported-adapters'](req, res); - const { data } = res.json.mock.calls[0][0]; + const res = await api.get(`${BASE}/supported-adapters`).expect(200); + const { data } = res.body; // Registry-derived: one entry per non-generic adapter, keyed by name with // the adapter's bankName label. Adding an adapter exposes it automatically. @@ -117,37 +134,38 @@ describe('Info Routes', () => { expect(bnp.name).toBe('BNP Paribas Fortis'); expect(bnp.adapter_class).toBeUndefined(); }); + + it('should return supported adapters', async () => { + const res = await api.get(`${BASE}/supported-adapters`).expect(200); + + const result = res.body.data; + expect(result.items).toBeDefined(); + expect(result.total).toBeGreaterThan(0); + }); }); describe('GET /banks', () => { it('should return bank list', async () => { infoRepository.getBanks.mockResolvedValue(['Chase', 'Revolut']); - const req = { query: {} }; - const res = mockResponse(); - await routeHandlers['get:/banks'](req, res); + const res = await api.get(`${BASE}/banks`).expect(200); - expect(res.json.mock.calls[0][0].data.items).toHaveLength(2); + expect(res.body.data.items).toHaveLength(2); }); it('should return empty for no banks', async () => { infoRepository.getBanks.mockResolvedValue([]); - const req = { query: {} }; - const res = mockResponse(); - await routeHandlers['get:/banks'](req, res); + const res = await api.get(`${BASE}/banks`).expect(200); - expect(res.json.mock.calls[0][0].data).toEqual({ items: [], total: 0 }); + expect(res.body.data).toEqual({ items: [], total: 0 }); }); it('should handle database errors', async () => { infoRepository.getBanks.mockRejectedValue(new Error('DB error')); - const req = { query: {} }; - const res = mockResponse(); - await callHandler(routeHandlers['get:/banks'], req, res); - - expect(res.status).toHaveBeenCalledWith(500); + const res = await api.get(`${BASE}/banks`).expect(500); + expect(res.body).toEqual(errEnvelope({ code: 'INTERNAL_SERVER_ERROR', message: 'DB error' })); }); }); @@ -155,43 +173,24 @@ describe('Info Routes', () => { it('should return count', async () => { infoRepository.getTransactionCount.mockResolvedValue(42); - const req = { query: {} }; - const res = mockResponse(); - await routeHandlers['get:/transaction-count'](req, res); + const res = await api.get(`${BASE}/transaction-count`).expect(200); - expect(res.json.mock.calls[0][0].data.total_transactions).toBe(42); + expect(res.body.data.total_transactions).toBe(42); }); it('should return 0 for empty', async () => { infoRepository.getTransactionCount.mockResolvedValue(0); - const req = { query: {} }; - const res = mockResponse(); - await routeHandlers['get:/transaction-count'](req, res); + const res = await api.get(`${BASE}/transaction-count`).expect(200); - expect(res.json.mock.calls[0][0].data.total_transactions).toBe(0); + expect(res.body.data.total_transactions).toBe(0); }); it('should handle errors', async () => { infoRepository.getTransactionCount.mockRejectedValue(new Error('DB error')); - const req = { query: {} }; - const res = mockResponse(); - await callHandler(routeHandlers['get:/transaction-count'], req, res); - - expect(res.status).toHaveBeenCalledWith(500); - }); - }); - - describe('GET /supported-adapters', () => { - it('should return supported adapters', async () => { - const req = { query: {} }; - const res = mockResponse(); - await routeHandlers['get:/supported-adapters'](req, res); - - const result = res.json.mock.calls[0][0].data; - expect(result.items).toBeDefined(); - expect(result.total).toBeGreaterThan(0); + const res = await api.get(`${BASE}/transaction-count`).expect(500); + expect(res.body).toEqual(errEnvelope({ code: 'INTERNAL_SERVER_ERROR', message: 'DB error' })); }); }); @@ -199,21 +198,15 @@ describe('Info Routes', () => { it('should return planned expenses', async () => { infoRepository.getPlannedExpensesNextMonth.mockResolvedValue({ total: 500 }); - const req = { query: {} }; - const res = mockResponse(); - await routeHandlers['get:/planned-expenses-next-month'](req, res); - - expect(res.json).toHaveBeenCalled(); + const res = await api.get(`${BASE}/planned-expenses-next-month`).expect(200); + expect(res.body.ok).toBe(true); }); it('should handle errors', async () => { infoRepository.getPlannedExpensesNextMonth.mockRejectedValue(new Error('DB error')); - const req = { query: {} }; - const res = mockResponse(); - await callHandler(routeHandlers['get:/planned-expenses-next-month'], req, res); - - expect(res.status).toHaveBeenCalledWith(500); + const res = await api.get(`${BASE}/planned-expenses-next-month`).expect(500); + expect(res.body).toEqual(errEnvelope({ code: 'INTERNAL_SERVER_ERROR', message: 'DB error' })); }); }); @@ -230,11 +223,9 @@ describe('Info Routes', () => { ], }); - const req = { query: { currency: 'EUR' } }; - const res = mockResponse(); - await routeHandlers['get:/net-worth'](req, res); + const res = await api.get(`${BASE}/net-worth`).query({ currency: 'EUR' }).expect(200); - const result = res.json.mock.calls[0][0].data; + const result = res.body.data; expect(result.current.netWorth).toBe(15000); expect(result.monthlyChange).toBe(500); expect(result.snapshots).toHaveLength(3); @@ -248,11 +239,9 @@ describe('Info Routes', () => { snapshots: [], }); - const req = { query: { currency: 'USD' } }; - const res = mockResponse(); - await routeHandlers['get:/net-worth'](req, res); + const res = await api.get(`${BASE}/net-worth`).query({ currency: 'USD' }).expect(200); - const result = res.json.mock.calls[0][0].data; + const result = res.body.data; expect(result.current.netWorth).toBe(0); expect(result.snapshots).toHaveLength(0); }); @@ -260,11 +249,8 @@ describe('Info Routes', () => { it('should handle errors', async () => { infoRepository.getNetWorthFromSnapshots.mockRejectedValue(new Error('DB error')); - const req = { query: { currency: 'GBP' } }; - const res = mockResponse(); - await callHandler(routeHandlers['get:/net-worth'], req, res); - - expect(res.status).toHaveBeenCalledWith(500); + const res = await api.get(`${BASE}/net-worth`).query({ currency: 'GBP' }).expect(500); + expect(res.body).toEqual(errEnvelope({ message: expect.any(String) })); }); it('should paginate snapshots newest-first when limit/offset supplied', async () => { @@ -281,12 +267,9 @@ describe('Info Routes', () => { ], }); - const req = { query: { currency: 'AUD', limit: '2', offset: '0' } }; - const res = mockResponse(); - await routeHandlers['get:/net-worth'](req, res); + const res = await api.get(`${BASE}/net-worth`).query({ currency: 'AUD', limit: '2', offset: '0' }).expect(200); - const body = res.json.mock.calls[0][0]; - const result = body.data; + const result = res.body.data; expect(result.snapshots).toHaveLength(2); expect(result.snapshots[0].date).toBe('2026-03-05'); expect(result.snapshots[1].date).toBe('2026-03-04'); @@ -295,7 +278,7 @@ describe('Info Routes', () => { // meta.pagination convention is retired (packages/types/src/api.js). expect(result.snapshotsLimit).toBe(2); expect(result.snapshotsOffset).toBe(0); - expect(body.meta).toBeUndefined(); + expect(res.body.meta.requestId).toEqual(expect.any(String)); }); it('should honor offset for pagination', async () => { @@ -311,11 +294,9 @@ describe('Info Routes', () => { ], }); - const req = { query: { currency: 'CAD', limit: '2', offset: '2' } }; - const res = mockResponse(); - await routeHandlers['get:/net-worth'](req, res); + const res = await api.get(`${BASE}/net-worth`).query({ currency: 'CAD', limit: '2', offset: '2' }).expect(200); - const result = res.json.mock.calls[0][0].data; + const result = res.body.data; expect(result.snapshots).toHaveLength(2); expect(result.snapshots[0].date).toBe('2026-03-02'); expect(result.snapshots[1].date).toBe('2026-03-01'); @@ -334,19 +315,15 @@ describe('Info Routes', () => { ], }); - const req = { query: { currency: 'CHF' } }; - const res = mockResponse(); - await routeHandlers['get:/net-worth'](req, res); + const res = await api.get(`${BASE}/net-worth`).query({ currency: 'CHF' }).expect(200); - const body = res.json.mock.calls[0][0]; - const result = body.data; + const result = res.body.data; expect(result.snapshots).toHaveLength(2); expect(result.snapshots[0].date).toBe('2026-03-01'); // Unpaginated: no pagination fields at all — the body IS the whole series. expect(result.snapshotsTotal).toBeUndefined(); expect(result.snapshotsLimit).toBeUndefined(); expect(result.snapshotsOffset).toBeUndefined(); - expect(body.meta).toBeUndefined(); }); }); @@ -357,28 +334,21 @@ describe('Info Routes', () => { total: 1, }); - const req = { query: {} }; - const res = mockResponse(); - await routeHandlers['get:/recurring-patterns'](req, res); + const res = await api.get(`${BASE}/recurring-patterns`).expect(200); - expect(res.json).toHaveBeenCalledWith({ - ok: true, - data: { patterns: [{ recipient: 'Netflix', interval_days: 30 }], total: 1 }, - }); + expect(res.body).toEqual(okEnvelope({ patterns: [{ recipient: 'Netflix', interval_days: 30 }], total: 1 })); }); it('should return empty recurring payload when detector fails', async () => { mockDetectRecurringPatterns.mockRejectedValue(new Error('detector failed')); - const req = { query: {} }; - const res = mockResponse(); - await routeHandlers['get:/recurring-patterns'](req, res); + const res = await api.get(`${BASE}/recurring-patterns`).expect(200); expect(logger.error).toHaveBeenCalledWith( 'Error detecting recurring patterns; returning empty result', expect.objectContaining({ error: 'detector failed' }) ); - expect(res.json).toHaveBeenCalledWith({ ok: true, data: { patterns: [], total: 0 } }); + expect(res.body).toEqual(okEnvelope({ patterns: [], total: 0 })); }); }); @@ -398,30 +368,25 @@ describe('Info Routes', () => { ], }); - const req = { query: {} }; - const res = mockResponse(); - await routeHandlers['get:/exchange-rates'](req, res); + const res = await api.get(`${BASE}/exchange-rates`).expect(200); expect(mockClearMemoryCache).toHaveBeenCalledTimes(1); expect(mockWarmCache).toHaveBeenCalledTimes(1); - expect(res.json).toHaveBeenCalledWith({ - ok: true, - data: { - total_rates: 1, - rates: [ - { - currency: 'USD', - rate_to_eur: 1.2345, - rate_date: '2026-04-10', - fetched_at: '2026-04-10T08:30:00.000Z', - }, - ], - fallback_rates: { USD: 1.1 }, - source: 'database', - is_stale: true, - last_fetched_at: '2026-04-10T08:30:00.000Z', - }, - }); + expect(res.body).toEqual(okEnvelope({ + total_rates: 1, + rates: [ + { + currency: 'USD', + rate_to_eur: 1.2345, + rate_date: '2026-04-10', + fetched_at: '2026-04-10T08:30:00.000Z', + }, + ], + fallback_rates: { USD: 1.1 }, + source: 'database', + is_stale: true, + last_fetched_at: '2026-04-10T08:30:00.000Z', + })); } finally { vi.useRealTimers(); } @@ -442,21 +407,14 @@ describe('Info Routes', () => { ], }); - const req = { query: {} }; - const res = mockResponse(); - await routeHandlers['get:/exchange-rates'](req, res); + const res = await api.get(`${BASE}/exchange-rates`).expect(200); expect(mockClearMemoryCache).not.toHaveBeenCalled(); expect(mockWarmCache).not.toHaveBeenCalled(); - expect(res.json).toHaveBeenCalledWith( - expect.objectContaining({ - ok: true, - data: expect.objectContaining({ - total_rates: 1, - rates: [expect.objectContaining({ currency: 'GBP', rate_date: '2026-04-11' })], - }), - }) - ); + expect(res.body).toEqual(okEnvelope(expect.objectContaining({ + total_rates: 1, + rates: [expect.objectContaining({ currency: 'GBP', rate_date: '2026-04-11' })], + }))); } finally { vi.useRealTimers(); } @@ -478,9 +436,8 @@ describe('Info Routes', () => { }); mockWarmCache.mockRejectedValueOnce(new Error('refresh failed')); - const req = { query: {} }; - const res = mockResponse(); - await routeHandlers['get:/exchange-rates'](req, res); + await api.get(`${BASE}/exchange-rates`).expect(200); + await Promise.resolve(); await Promise.resolve(); expect(logger.warn).toHaveBeenCalledWith( @@ -495,68 +452,43 @@ describe('Info Routes', () => { it('should handle exchange-rate query errors', async () => { mockListLatestStoredRates.mockRejectedValue(new Error('query failed')); - const req = { query: {} }; - const res = mockResponse(); - await callHandler(routeHandlers['get:/exchange-rates'], req, res); - - expect(res.status).toHaveBeenCalledWith(500); - expect(res.json).toHaveBeenCalledWith( - expect.objectContaining({ ok: false, error: expect.objectContaining({ message: expect.any(String) }) }) - ); + const res = await api.get(`${BASE}/exchange-rates`).expect(500); + expect(res.body).toEqual(errEnvelope({ message: expect.any(String) })); }); }); describe('POST /exchange-rates/refresh', () => { it('should clear cache and refresh exchange rates', async () => { - const req = { query: {} }; - const res = mockResponse(); - await routeHandlers['post:/exchange-rates/refresh'](req, res); + const res = await api.post(`${BASE}/exchange-rates/refresh`).send({}).expect(200); expect(mockClearMemoryCache).toHaveBeenCalledTimes(1); expect(mockWarmCache).toHaveBeenCalledTimes(1); - expect(res.json).toHaveBeenCalledWith({ ok: true, data: { message: 'Exchange rates refreshed from ECB' } }); + expect(res.body).toEqual(okEnvelope({ message: 'Exchange rates refreshed from ECB' })); }); it('should handle exchange refresh errors', async () => { mockWarmCache.mockRejectedValueOnce(new Error('ecb down')); - const req = { query: {} }; - const res = mockResponse(); - await callHandler(routeHandlers['post:/exchange-rates/refresh'], req, res); - - expect(res.status).toHaveBeenCalledWith(500); - expect(res.json).toHaveBeenCalledWith( - expect.objectContaining({ ok: false, error: expect.objectContaining({ message: expect.any(String) }) }) - ); + const res = await api.post(`${BASE}/exchange-rates/refresh`).send({}).expect(500); + expect(res.body).toEqual(errEnvelope({ message: expect.any(String) })); }); }); describe('POST /refresh-views', () => { it('should refresh materialized views and return duration', async () => { - const req = { query: {} }; - const res = mockResponse(); - await routeHandlers['post:/refresh-views'](req, res); + const res = await api.post(`${BASE}/refresh-views`).send({}).expect(200); expect(mockRefreshMaterializedViews).toHaveBeenCalledTimes(1); - expect(res.json).toHaveBeenCalledWith( - expect.objectContaining({ - ok: true, - data: expect.objectContaining({ message: 'Materialized views refreshed', duration_ms: expect.any(Number) }), - }) - ); + expect(res.body).toEqual(okEnvelope(expect.objectContaining({ + message: 'Materialized views refreshed', duration_ms: expect.any(Number), + }))); }); it('should handle refresh-view failures', async () => { mockRefreshMaterializedViews.mockRejectedValueOnce(new Error('view refresh failed')); - const req = { query: {} }; - const res = mockResponse(); - await callHandler(routeHandlers['post:/refresh-views'], req, res); - - expect(res.status).toHaveBeenCalledWith(500); - expect(res.json).toHaveBeenCalledWith( - expect.objectContaining({ ok: false, error: expect.objectContaining({ message: expect.any(String) }) }) - ); + const res = await api.post(`${BASE}/refresh-views`).send({}).expect(500); + expect(res.body).toEqual(errEnvelope({ message: expect.any(String) })); }); }); @@ -582,37 +514,30 @@ describe('Info Routes', () => { }, ]); - const req = { query: { currency: 'USD' } }; - const res = mockResponse(); - await routeHandlers['get:/portfolio-performance'](req, res); + const res = await api.get(`${BASE}/portfolio-performance`).query({ currency: 'USD' }).expect(200); expect(mockGetSnapshots).toHaveBeenCalledWith('2000-01-01', '2026-04-11', 'USD'); - expect(res.json).toHaveBeenCalledWith( - expect.objectContaining({ - ok: true, - data: expect.objectContaining({ - currency: 'USD', - start_date: '2000-01-01', - end_date: '2026-04-11', - snapshots: [ - { - date: '2026-04-10', - invested: 1000.5, - value: 1234.56, - stocks_etfs_value: 500, - crypto_value: 200, - metals_value: 100, - stocks_etfs_invested: 450, - crypto_invested: 180, - metals_invested: 90, - inflation_adjusted_value: 1234.56, - gain_loss: 234.06, - return_pct: 23.4, - }, - ], - }), - }) - ); + expect(res.body).toEqual(okEnvelope(expect.objectContaining({ + currency: 'USD', + start_date: '2000-01-01', + end_date: '2026-04-11', + snapshots: [ + { + date: '2026-04-10', + invested: 1000.5, + value: 1234.56, + stocks_etfs_value: 500, + crypto_value: 200, + metals_value: 100, + stocks_etfs_invested: 450, + crypto_invested: 180, + metals_invested: 90, + inflation_adjusted_value: 1234.56, + gain_loss: 234.06, + return_pct: 23.4, + }, + ], + }))); } finally { vi.useRealTimers(); } @@ -624,9 +549,7 @@ describe('Info Routes', () => { vi.setSystemTime(new Date('2026-04-11T10:00:00.000Z')); mockGetSnapshots.mockResolvedValue([]); - const req = { query: { currency: 'invalid-currency' } }; - const res = mockResponse(); - await routeHandlers['get:/portfolio-performance'](req, res); + await api.get(`${BASE}/portfolio-performance`).query({ currency: 'invalid-currency' }).expect(200); expect(mockGetSnapshots).toHaveBeenCalledWith('2000-01-01', '2026-04-11', 'EUR'); } finally { @@ -637,14 +560,10 @@ describe('Info Routes', () => { it('should handle portfolio performance errors', async () => { mockGetSnapshots.mockRejectedValue(new Error('snapshots failed')); - const req = { query: { start_date: '2026-01-01', end_date: '2026-01-31' } }; - const res = mockResponse(); - await callHandler(routeHandlers['get:/portfolio-performance'], req, res); - - expect(res.status).toHaveBeenCalledWith(500); - expect(res.json).toHaveBeenCalledWith( - expect.objectContaining({ ok: false, error: expect.objectContaining({ message: expect.any(String) }) }) - ); + const res = await api.get(`${BASE}/portfolio-performance`) + .query({ start_date: '2026-01-01', end_date: '2026-01-31' }) + .expect(500); + expect(res.body).toEqual(errEnvelope({ message: expect.any(String) })); }); }); @@ -677,23 +596,15 @@ describe('Info Routes', () => { await warmInfoCaches('JPY'); - const netWorthReq = { query: { currency: 'JPY' } }; - const netWorthRes = mockResponse(); - await routeHandlers['get:/net-worth'](netWorthReq, netWorthRes); - - const perfReq = { query: { currency: 'JPY' } }; - const perfRes = mockResponse(); - await routeHandlers['get:/portfolio-performance'](perfReq, perfRes); + const netWorthRes = await api.get(`${BASE}/net-worth`).query({ currency: 'JPY' }).expect(200); + const perfRes = await api.get(`${BASE}/portfolio-performance`).query({ currency: 'JPY' }).expect(200); expect(infoRepository.getNetWorthFromSnapshots).toHaveBeenCalledTimes(1); expect(mockGetSnapshots).toHaveBeenCalledTimes(1); - expect(netWorthRes.json).toHaveBeenCalledWith({ ok: true, data: netWorthPayload }); - expect(perfRes.json).toHaveBeenCalledWith( - expect.objectContaining({ - ok: true, - data: expect.objectContaining({ currency: 'JPY', start_date: '2000-01-01', end_date: '2026-04-11' }), - }) - ); + expect(netWorthRes.body).toEqual(okEnvelope(netWorthPayload)); + expect(perfRes.body).toEqual(okEnvelope(expect.objectContaining({ + currency: 'JPY', start_date: '2000-01-01', end_date: '2026-04-11', + }))); } finally { vi.useRealTimers(); } @@ -747,9 +658,7 @@ describe('Info Routes', () => { ], }); - const req = { query: { start_month: '2024-01', end_month: '2024-12' } }; - const res = mockResponse(); - await routeHandlers['get:/inflation-rates'](req, res); + const res = await api.get(`${BASE}/inflation-rates`).query({ start_month: '2024-01', end_month: '2024-12' }).expect(200); expect(mockInflationService.getInflationRates).toHaveBeenCalledWith({ startMonth: '2024-01', @@ -757,7 +666,7 @@ describe('Info Routes', () => { dbOnly: true, scheduleBackgroundRefresh: true, }); - const payload = res.json.mock.calls[0][0].data; + const payload = res.body.data; expect(payload.total_rates).toBe(2); expect(payload.source).toBe('database'); }); @@ -765,9 +674,7 @@ describe('Info Routes', () => { it('should ignore invalid month params', async () => { mockInflationService.getInflationRates.mockResolvedValue({ source: 'memory', rates: [] }); - const req = { query: { start_month: 'invalid', end_month: '2024/01' } }; - const res = mockResponse(); - await routeHandlers['get:/inflation-rates'](req, res); + await api.get(`${BASE}/inflation-rates`).query({ start_month: 'invalid', end_month: '2024/01' }).expect(200); expect(mockInflationService.getInflationRates).toHaveBeenCalledWith({ startMonth: undefined, @@ -783,9 +690,7 @@ describe('Info Routes', () => { rates: [{ month: '2024-01', monthly_rate: 0.004 }], }); - const req = { query: { db_only: 'true', start_month: '2024-01' } }; - const res = mockResponse(); - await routeHandlers['get:/inflation-rates'](req, res); + await api.get(`${BASE}/inflation-rates`).query({ db_only: 'true', start_month: '2024-01' }).expect(200); expect(mockInflationService.getInflationRates).toHaveBeenCalledWith({ startMonth: '2024-01', @@ -801,9 +706,7 @@ describe('Info Routes', () => { rates: [{ month: '2024-01', monthly_rate: 0.004 }], }); - const req = { query: { db_only: 'false' } }; - const res = mockResponse(); - await routeHandlers['get:/inflation-rates'](req, res); + await api.get(`${BASE}/inflation-rates`).query({ db_only: 'false' }).expect(200); expect(mockInflationService.getInflationRates).toHaveBeenCalledWith({ startMonth: undefined, @@ -816,11 +719,8 @@ describe('Info Routes', () => { it('should handle inflation route errors', async () => { mockInflationService.getInflationRates.mockRejectedValue(new Error('boom')); - const req = { query: {} }; - const res = mockResponse(); - await callHandler(routeHandlers['get:/inflation-rates'], req, res); - - expect(res.status).toHaveBeenCalledWith(500); + const res = await api.get(`${BASE}/inflation-rates`).expect(500); + expect(res.body).toEqual(errEnvelope({ message: expect.any(String) })); }); }); @@ -828,13 +728,11 @@ describe('Info Routes', () => { it('should refresh Belgian inflation rates', async () => { mockInflationService.getInflationRates.mockResolvedValue({ source: 'statbel', rates: [{ month: '2024-01', monthly_rate: 0.004 }] }); - const req = { query: {} }; - const res = mockResponse(); - await routeHandlers['post:/inflation-rates/refresh'](req, res); + const res = await api.post(`${BASE}/inflation-rates/refresh`).send({}).expect(200); expect(mockInflationService.clearInflationMemoryCache).toHaveBeenCalled(); expect(mockInflationService.getInflationRates).toHaveBeenCalledWith({ forceRefresh: true }); - const payload = res.json.mock.calls[0][0].data; + const payload = res.body.data; expect(payload.total_rates).toBe(1); expect(payload.source).toBe('statbel'); }); @@ -842,34 +740,8 @@ describe('Info Routes', () => { it('should handle refresh errors', async () => { mockInflationService.getInflationRates.mockRejectedValue(new Error('refresh failed')); - const req = { query: {} }; - const res = mockResponse(); - await callHandler(routeHandlers['post:/inflation-rates/refresh'], req, res); - - expect(res.status).toHaveBeenCalledWith(500); + const res = await api.post(`${BASE}/inflation-rates/refresh`).send({}).expect(500); + expect(res.body).toEqual(errEnvelope({ message: expect.any(String) })); }); }); }); - -function mockResponse() { - return createMockResponse(); -} - -/** - * Simulates Express error-handler middleware for routes that throw typed errors. - * Routes use `throw new AppError / NotFoundError / ValidationError` which - * propagates to the centralized error handler in production. In unit tests we - * catch the error here and replicate the handler's response shape. - */ -async function callHandler(handler, req, res) { - try { - await handler(req, res); - } catch (err) { - const status = err.status ?? 500; - const code = err.code ?? 'INTERNAL_SERVER_ERROR'; - const message = err.message ?? 'Internal server error'; - const error = { code, message }; - if (err.details !== undefined) error.details = err.details; - res.status(status).json({ ok: false, error }); - } -} diff --git a/apps/node-backend/tests/routes/investments.test.js b/apps/node-backend/tests/routes/investments.test.js index 92b1e120..1d46be70 100644 --- a/apps/node-backend/tests/routes/investments.test.js +++ b/apps/node-backend/tests/routes/investments.test.js @@ -1,17 +1,21 @@ /** * Investment route tests. * Tests all CRUD endpoints for investments and portfolio transactions. + * + * Runs against the REAL router mounted on a throwaway Express app (see + * tests/helpers/routeApp.js) — validateIdParam (routes/investments.js:35-41) + * is no longer stubbed; it runs for real on every `/:id`-prefixed route. No + * test here exercised an invalid id against one of those routes under the + * old stub (all used valid numeric ids), so nothing was fake-passing — + * validateIdParam is simply exercised for real now instead of bypassed. + * + * Mount path is /api/investments, behind investmentRateLimiter at the app + * level (main.js:327) — a module-scoped per-IP counter deliberately NOT + * reproduced here per the routeApp.js fidelity map. */ import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest'; import { mockLogger } from '../helpers/mockLogger.js'; -import { createMockRouter, createMockResponse } from '../helpers/routeHarness.js'; - -const { router: mockRouter, handlers: routeHandlers } = createMockRouter(); - -vi.mock('express', () => ({ - default: { Router: () => mockRouter }, - Router: () => mockRouter, -})); +import { routeAgent, okEnvelope, errEnvelope } from '../helpers/routeApp.js'; vi.mock('../../src/repositories/investmentRepository.js', () => ({ default: { @@ -71,13 +75,6 @@ vi.mock('../../src/config/kinesisConfig.js', () => ({ }), })); -vi.mock('../../src/middleware/validation.js', async (importOriginal) => ({ - // Keep the real value helpers (assertCurrency, assertMaxLength, validateNumber); - // only stub the id-param middleware to a no-op. - ...(await importOriginal()), - validateIdParam: (req, res, next) => next(), -})); - vi.mock('../../src/config/logger.js', () => ({ logger: mockLogger(), })); @@ -85,14 +82,13 @@ vi.mock('../../src/config/logger.js', () => ({ import investmentRepository from '../../src/repositories/investmentRepository.js'; import portfolioTransactionRepository from '../../src/repositories/portfolioTransactionRepository.js'; import { fetchHistoricalPrices, fetchLivePricesDetailed } from '../../src/services/priceProviderService.js'; -import { ValidationError, NotFoundError } from '../../src/middleware/errorHandler.js'; -await import('../../src/routes/investments.js'); -let nowSpy; +const { default: investmentsRouter } = await import('../../src/routes/investments.js'); -function mockResponse() { - return createMockResponse({ end: vi.fn() }); -} +const BASE = '/api/investments'; +const api = routeAgent(investmentsRouter, { mountPath: BASE }); + +let nowSpy; describe('Investment Routes', () => { beforeEach(() => { @@ -106,23 +102,6 @@ describe('Investment Routes', () => { describe('GET /transactions', () => { it('should cache and differentiate entries by limit parameter', async () => { - const reqLimit10 = { - query: { - investment_ids: '1,2', - per_investment_limit: '1000', - limit: '10', - offset: '0', - }, - }; - const reqLimit20 = { - query: { - investment_ids: '1,2', - per_investment_limit: '1000', - limit: '20', - offset: '0', - }, - }; - const repoResultA = [{ id: 101, amount: 1 }]; const repoResultB = [{ id: 202, amount: 2 }]; @@ -131,23 +110,17 @@ describe('Investment Routes', () => { .mockResolvedValueOnce(repoResultB); portfolioTransactionRepository.getCount.mockResolvedValue(2); - const resA1 = mockResponse(); - await routeHandlers['get:/transactions'](reqLimit10, resA1); - expect(resA1.json).toHaveBeenCalledWith(expect.objectContaining({ - data: expect.objectContaining({ items: repoResultA, limit: 10 }), - })); + const query10 = { investment_ids: '1,2', per_investment_limit: '1000', limit: '10', offset: '0' }; + const query20 = { investment_ids: '1,2', per_investment_limit: '1000', limit: '20', offset: '0' }; - const resA2 = mockResponse(); - await routeHandlers['get:/transactions'](reqLimit10, resA2); - expect(resA2.json).toHaveBeenCalledWith(expect.objectContaining({ - data: expect.objectContaining({ items: repoResultA, limit: 10 }), - })); + const resA1 = await api.get(`${BASE}/transactions`).query(query10).expect(200); + expect(resA1.body).toEqual(okEnvelope(expect.objectContaining({ items: repoResultA, limit: 10 }))); - const resB = mockResponse(); - await routeHandlers['get:/transactions'](reqLimit20, resB); - expect(resB.json).toHaveBeenCalledWith(expect.objectContaining({ - data: expect.objectContaining({ items: repoResultB, limit: 20 }), - })); + const resA2 = await api.get(`${BASE}/transactions`).query(query10).expect(200); + expect(resA2.body).toEqual(okEnvelope(expect.objectContaining({ items: repoResultA, limit: 10 }))); + + const resB = await api.get(`${BASE}/transactions`).query(query20).expect(200); + expect(resB.body).toEqual(okEnvelope(expect.objectContaining({ items: repoResultB, limit: 20 }))); expect(portfolioTransactionRepository.getAllByInvestmentIds).toHaveBeenCalledTimes(2); expect(portfolioTransactionRepository.getAllByInvestmentIds).toHaveBeenNthCalledWith( @@ -169,31 +142,24 @@ describe('Investment Routes', () => { total: 1, }); - const req = { query: {} }; - const res = mockResponse(); - await routeHandlers['get:/'](req, res); + const res = await api.get(`${BASE}/`).expect(200); - const data = res.json.mock.calls[0][0]; - expect(data.data.items).toHaveLength(1); - expect(data.data.total).toBe(1); + expect(res.body.data.items).toHaveLength(1); + expect(res.body.data.total).toBe(1); }); it('should return empty list', async () => { investmentRepository.getAllWithCount.mockResolvedValue({ rows: [], total: 0 }); - const req = { query: {} }; - const res = mockResponse(); - await routeHandlers['get:/'](req, res); + const res = await api.get(`${BASE}/`).expect(200); - expect(res.json.mock.calls[0][0].data.items).toEqual([]); + expect(res.body.data.items).toEqual([]); }); it('should respect pagination and filters', async () => { investmentRepository.getAllWithCount.mockResolvedValue({ rows: [], total: 0 }); - const req = { query: { limit: '10', offset: '5', asset_class: 'crypto', active: 'true' } }; - const res = mockResponse(); - await routeHandlers['get:/'](req, res); + await api.get(`${BASE}/`).query({ limit: '10', offset: '5', asset_class: 'crypto', active: 'true' }).expect(200); expect(investmentRepository.getAllWithCount).toHaveBeenCalledWith(expect.objectContaining({ limit: 10, offset: 5, assetClass: 'crypto', active: true, @@ -203,9 +169,8 @@ describe('Investment Routes', () => { it('should handle errors', async () => { investmentRepository.getAllWithCount.mockRejectedValue(new Error('DB error')); - const req = { query: {} }; - const res = mockResponse(); - await expect(routeHandlers['get:/'](req, res)).rejects.toThrow('DB error'); + const res = await api.get(`${BASE}/`).expect(500); + expect(res.body).toEqual(errEnvelope({ code: 'INTERNAL_SERVER_ERROR', message: 'DB error' })); }); }); @@ -214,44 +179,34 @@ describe('Investment Routes', () => { it('should create investment with 201', async () => { investmentRepository.create.mockResolvedValue({ id: 1, name: 'Bitcoin', asset_class: 'crypto' }); - const req = { body: { name: 'Bitcoin', asset_class: 'crypto', currency: 'EUR' } }; - const res = mockResponse(); - await routeHandlers['post:/'](req, res); - - expect(res.status).toHaveBeenCalledWith(201); + await api.post(`${BASE}/`).send({ name: 'Bitcoin', asset_class: 'crypto', currency: 'EUR' }).expect(201); }); it('should throw ValidationError for missing required fields', async () => { - const req = { body: { name: 'Bitcoin' } }; - const res = mockResponse(); - await expect(routeHandlers['post:/'](req, res)).rejects.toBeInstanceOf(ValidationError); + const res = await api.post(`${BASE}/`).send({ name: 'Bitcoin' }).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); }); it('should throw ValidationError for missing name', async () => { - const req = { body: { asset_class: 'crypto' } }; - const res = mockResponse(); - await expect(routeHandlers['post:/'](req, res)).rejects.toBeInstanceOf(ValidationError); + const res = await api.post(`${BASE}/`).send({ asset_class: 'crypto' }).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); }); it('should handle errors', async () => { investmentRepository.create.mockRejectedValue(new Error('DB error')); - const req = { body: { name: 'Bitcoin', asset_class: 'crypto' } }; - const res = mockResponse(); - await expect(routeHandlers['post:/'](req, res)).rejects.toThrow('DB error'); + const res = await api.post(`${BASE}/`).send({ name: 'Bitcoin', asset_class: 'crypto' }).expect(500); + expect(res.body).toEqual(errEnvelope({ code: 'INTERNAL_SERVER_ERROR', message: 'DB error' })); }); }); // ── GET /api/investments/providers ───────────────────────── describe('GET /providers', () => { it('should return supported providers', async () => { - const req = {}; - const res = mockResponse(); - await routeHandlers['get:/providers'](req, res); + const res = await api.get(`${BASE}/providers`).expect(200); - const data = res.json.mock.calls[0][0]; - expect(data.data.providers).toBeDefined(); - expect(data.data.providers.length).toBeGreaterThan(0); + expect(res.body.data.providers).toBeDefined(); + expect(res.body.data.providers.length).toBeGreaterThan(0); }); }); @@ -265,11 +220,9 @@ describe('Investment Routes', () => { // Faithful stand-in for the real bulk update: one statement, N rows. investmentRepository.updatePricesBulk.mockImplementation(async (updates) => updates.length); - const req = { body: {} }; - const res = mockResponse(); - await routeHandlers['post:/refresh-prices'](req, res); + const res = await api.post(`${BASE}/refresh-prices`).send({}).expect(200); - const data = res.json.mock.calls[0][0].data; + const data = res.body.data; expect(data.updated).toBe(1); expect(data.priceSources).toEqual({ 1: 'live' }); expect(investmentRepository.updatePricesBulk).toHaveBeenCalledTimes(1); @@ -285,11 +238,9 @@ describe('Investment Routes', () => { fetchLivePricesDetailed.mockResolvedValue({ 1: { price: 123.45, source: 'cached' } }); investmentRepository.updatePricesBulk.mockImplementation(async (updates) => updates.length); - const req = { body: {} }; - const res = mockResponse(); - await routeHandlers['post:/refresh-prices'](req, res); + const res = await api.post(`${BASE}/refresh-prices`).send({}).expect(200); - const data = res.json.mock.calls[0][0].data; + const data = res.body.data; expect(data.updated).toBe(0); expect(data.prices).toEqual({ 1: 123.45 }); expect(data.priceSources).toEqual({ 1: 'cached' }); @@ -304,11 +255,9 @@ describe('Investment Routes', () => { fetchLivePricesDetailed.mockResolvedValue({ 1: { price: 188.4, source: 'live' } }); investmentRepository.updatePricesBulk.mockImplementation(async (updates) => updates.length); - const req = { body: {} }; - const res = mockResponse(); - await routeHandlers['post:/refresh-prices'](req, res); + const res = await api.post(`${BASE}/refresh-prices`).send({}).expect(200); - const data = res.json.mock.calls[0][0].data; + const data = res.body.data; expect(data.total).toBe(1); expect(data.updated).toBe(1); expect(investmentRepository.updatePricesBulk).toHaveBeenCalledTimes(1); @@ -321,11 +270,9 @@ describe('Investment Routes', () => { fetchLivePricesDetailed.mockResolvedValue({ 1: { price: 101.25, source: 'live' } }); investmentRepository.updatePricesBulk.mockImplementation(async (updates) => updates.length); - const req = { body: {} }; - const res = mockResponse(); - await routeHandlers['post:/refresh-prices'](req, res); + const res = await api.post(`${BASE}/refresh-prices`).send({}).expect(200); - const data = res.json.mock.calls[0][0].data; + const data = res.body.data; expect(data.total).toBe(1); expect(data.updated).toBe(1); expect(data.prices).toEqual({ 1: 101.25 }); @@ -338,19 +285,16 @@ describe('Investment Routes', () => { { id: 1, price_provider: 'manual', price_provider_id: null }, ]); - const req = { body: {} }; - const res = mockResponse(); - await routeHandlers['post:/refresh-prices'](req, res); + const res = await api.post(`${BASE}/refresh-prices`).send({}).expect(200); - expect(res.json.mock.calls[0][0].data.updated).toBe(0); + expect(res.body.data.updated).toBe(0); }); it('should handle errors', async () => { investmentRepository.getAll.mockRejectedValue(new Error('DB error')); - const req = { body: {} }; - const res = mockResponse(); - await expect(routeHandlers['post:/refresh-prices'](req, res)).rejects.toThrow('DB error'); + const res = await api.post(`${BASE}/refresh-prices`).send({}).expect(500); + expect(res.body).toEqual(errEnvelope({ code: 'INTERNAL_SERVER_ERROR', message: 'DB error' })); }); }); @@ -362,22 +306,19 @@ describe('Investment Routes', () => { { timestampMs: 1700000000000, price: 700 }, ]); - const req = { params: { id: '12' }, query: { from_ms: '1699999999999', to_ms: '1700000000001' } }; - const res = mockResponse(); - await routeHandlers['get:/:id/price-history'](req, res); + const res = await api.get(`${BASE}/12/price-history`) + .query({ from_ms: '1699999999999', to_ms: '1700000000001' }) + .expect(200); expect(fetchHistoricalPrices).toHaveBeenCalledWith( { id: 12, price_provider: 'custom' }, { fromMs: 1699999999999, toMs: 1700000000001, dbOnly: true } ); - expect(res.json).toHaveBeenCalledWith({ - ok: true, - data: { - investment_id: 12, - provider: 'custom', - points: [{ timestampMs: 1700000000000, price: 700 }], - }, - }); + expect(res.body).toEqual(okEnvelope({ + investment_id: 12, + provider: 'custom', + points: [{ timestampMs: 1700000000000, price: 700 }], + })); }); }); @@ -386,27 +327,23 @@ describe('Investment Routes', () => { it('should return investment by id', async () => { investmentRepository.getById.mockResolvedValue({ id: 1, name: 'Bitcoin' }); - const req = { params: { id: '1' } }; - const res = mockResponse(); - await routeHandlers['get:/:id'](req, res); + const res = await api.get(`${BASE}/1`).expect(200); - expect(res.json.mock.calls[0][0].data.name).toBe('Bitcoin'); + expect(res.body.data.name).toBe('Bitcoin'); }); it('should throw NotFoundError for non-existent', async () => { investmentRepository.getById.mockResolvedValue(null); - const req = { params: { id: '999' } }; - const res = mockResponse(); - await expect(routeHandlers['get:/:id'](req, res)).rejects.toBeInstanceOf(NotFoundError); + const res = await api.get(`${BASE}/999`).expect(404); + expect(res.body).toEqual(errEnvelope({ code: 'NOT_FOUND' })); }); it('should handle errors', async () => { investmentRepository.getById.mockRejectedValue(new Error('DB error')); - const req = { params: { id: '1' } }; - const res = mockResponse(); - await expect(routeHandlers['get:/:id'](req, res)).rejects.toThrow('DB error'); + const res = await api.get(`${BASE}/1`).expect(500); + expect(res.body).toEqual(errEnvelope({ code: 'INTERNAL_SERVER_ERROR', message: 'DB error' })); }); }); @@ -415,27 +352,23 @@ describe('Investment Routes', () => { it('should update investment', async () => { investmentRepository.update.mockResolvedValue({ id: 1, name: 'Updated' }); - const req = { params: { id: '1' }, body: { name: 'Updated' } }; - const res = mockResponse(); - await routeHandlers['patch:/:id'](req, res); + const res = await api.patch(`${BASE}/1`).send({ name: 'Updated' }).expect(200); - expect(res.json.mock.calls[0][0].data.name).toBe('Updated'); + expect(res.body.data.name).toBe('Updated'); }); it('should throw NotFoundError for non-existent', async () => { investmentRepository.update.mockResolvedValue(null); - const req = { params: { id: '999' }, body: { name: 'Updated' } }; - const res = mockResponse(); - await expect(routeHandlers['patch:/:id'](req, res)).rejects.toBeInstanceOf(NotFoundError); + const res = await api.patch(`${BASE}/999`).send({ name: 'Updated' }).expect(404); + expect(res.body).toEqual(errEnvelope({ code: 'NOT_FOUND' })); }); it('should handle errors', async () => { investmentRepository.update.mockRejectedValue(new Error('DB error')); - const req = { params: { id: '1' }, body: { name: 'Updated' } }; - const res = mockResponse(); - await expect(routeHandlers['patch:/:id'](req, res)).rejects.toThrow('DB error'); + const res = await api.patch(`${BASE}/1`).send({ name: 'Updated' }).expect(500); + expect(res.body).toEqual(errEnvelope({ code: 'INTERNAL_SERVER_ERROR', message: 'DB error' })); }); it('should throw ValidationError for validation errors', async () => { @@ -443,9 +376,8 @@ describe('Investment Routes', () => { err.code = 'VALIDATION_ERROR'; investmentRepository.update.mockRejectedValue(err); - const req = { params: { id: '1' }, body: { symbol: 'AAPL' } }; - const res = mockResponse(); - await expect(routeHandlers['patch:/:id'](req, res)).rejects.toBeInstanceOf(ValidationError); + const res = await api.patch(`${BASE}/1`).send({ symbol: 'AAPL' }).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); }); }); @@ -454,28 +386,23 @@ describe('Investment Routes', () => { it('should delete and return 204', async () => { investmentRepository.hardDelete.mockResolvedValue(true); - const req = { params: { id: '1' } }; - const res = mockResponse(); - await routeHandlers['delete:/:id'](req, res); + await api.delete(`${BASE}/1`).expect(204); expect(investmentRepository.hardDelete).toHaveBeenCalledWith(1); - expect(res.status).toHaveBeenCalledWith(204); }); it('should throw NotFoundError for non-existent', async () => { investmentRepository.hardDelete.mockResolvedValue(false); - const req = { params: { id: '999' } }; - const res = mockResponse(); - await expect(routeHandlers['delete:/:id'](req, res)).rejects.toBeInstanceOf(NotFoundError); + const res = await api.delete(`${BASE}/999`).expect(404); + expect(res.body).toEqual(errEnvelope({ code: 'NOT_FOUND' })); }); it('should handle errors', async () => { investmentRepository.hardDelete.mockRejectedValue(new Error('DB error')); - const req = { params: { id: '1' } }; - const res = mockResponse(); - await expect(routeHandlers['delete:/:id'](req, res)).rejects.toThrow('DB error'); + const res = await api.delete(`${BASE}/1`).expect(500); + expect(res.body).toEqual(errEnvelope({ code: 'INTERNAL_SERVER_ERROR', message: 'DB error' })); }); }); @@ -487,21 +414,16 @@ describe('Investment Routes', () => { total: 1, }); - const req = { params: { id: '1' }, query: {} }; - const res = mockResponse(); - await routeHandlers['get:/:id/transactions'](req, res); + const res = await api.get(`${BASE}/1/transactions`).expect(200); - const data = res.json.mock.calls[0][0].data; - expect(data.items).toHaveLength(1); - expect(data.total).toBe(1); + expect(res.body.data.items).toHaveLength(1); + expect(res.body.data.total).toBe(1); }); it('should filter by type', async () => { portfolioTransactionRepository.getAllWithCount.mockResolvedValue({ rows: [], total: 0 }); - const req = { params: { id: '1' }, query: { type: 'buy' } }; - const res = mockResponse(); - await routeHandlers['get:/:id/transactions'](req, res); + await api.get(`${BASE}/1/transactions`).query({ type: 'buy' }).expect(200); expect(portfolioTransactionRepository.getAllWithCount).toHaveBeenCalledWith( expect.objectContaining({ type: 'buy' }) @@ -511,9 +433,8 @@ describe('Investment Routes', () => { it('should handle errors', async () => { portfolioTransactionRepository.getAllWithCount.mockRejectedValue(new Error('DB error')); - const req = { params: { id: '1' }, query: {} }; - const res = mockResponse(); - await expect(routeHandlers['get:/:id/transactions'](req, res)).rejects.toThrow('DB error'); + const res = await api.get(`${BASE}/1/transactions`).expect(500); + expect(res.body).toEqual(errEnvelope({ code: 'INTERNAL_SERVER_ERROR', message: 'DB error' })); }); }); @@ -525,23 +446,16 @@ describe('Investment Routes', () => { id: 1, type: 'buy', amount: 1000, }); - const req = { params: { id: '1' }, body: { type: 'buy', date: '2026-01-15', amount: 1000 } }; - const res = mockResponse(); - await routeHandlers['post:/:id/transactions'](req, res); - - expect(res.status).toHaveBeenCalledWith(201); + await api.post(`${BASE}/1/transactions`).send({ type: 'buy', date: '2026-01-15', amount: 1000 }).expect(201); }); it('should pass fx_rate_to_eur to repository create', async () => { investmentRepository.getById.mockResolvedValue({ id: 1, currency: 'USD' }); portfolioTransactionRepository.create.mockResolvedValue({ id: 1, type: 'buy', amount: 1000, fx_rate_to_eur: 0.92 }); - const req = { - params: { id: '1' }, - body: { type: 'buy', date: '2026-01-15', amount: 1000, fx_rate_to_eur: 0.92 }, - }; - const res = mockResponse(); - await routeHandlers['post:/:id/transactions'](req, res); + await api.post(`${BASE}/1/transactions`) + .send({ type: 'buy', date: '2026-01-15', amount: 1000, fx_rate_to_eur: 0.92 }) + .expect(201); expect(portfolioTransactionRepository.create).toHaveBeenCalledWith( expect.objectContaining({ @@ -549,23 +463,20 @@ describe('Investment Routes', () => { fx_rate_to_eur: 0.92, }) ); - expect(res.status).toHaveBeenCalledWith(201); }); it('should throw NotFoundError if investment not found', async () => { investmentRepository.getById.mockResolvedValue(null); - const req = { params: { id: '999' }, body: { type: 'buy', date: '2026-01-15', amount: 1000 } }; - const res = mockResponse(); - await expect(routeHandlers['post:/:id/transactions'](req, res)).rejects.toBeInstanceOf(NotFoundError); + const res = await api.post(`${BASE}/999/transactions`).send({ type: 'buy', date: '2026-01-15', amount: 1000 }).expect(404); + expect(res.body).toEqual(errEnvelope({ code: 'NOT_FOUND' })); }); it('should throw ValidationError for missing required fields', async () => { investmentRepository.getById.mockResolvedValue({ id: 1, currency: 'EUR' }); - const req = { params: { id: '1' }, body: { type: 'buy' } }; - const res = mockResponse(); - await expect(routeHandlers['post:/:id/transactions'](req, res)).rejects.toBeInstanceOf(ValidationError); + const res = await api.post(`${BASE}/1/transactions`).send({ type: 'buy' }).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); }); it('should throw ValidationError when repository raises validation error', async () => { @@ -574,18 +485,16 @@ describe('Investment Routes', () => { err.code = 'VALIDATION_ERROR'; portfolioTransactionRepository.create.mockRejectedValue(err); - const req = { params: { id: '1' }, body: { type: 'buy', date: '2026-01-15' } }; - const res = mockResponse(); - await expect(routeHandlers['post:/:id/transactions'](req, res)).rejects.toBeInstanceOf(ValidationError); + const res = await api.post(`${BASE}/1/transactions`).send({ type: 'buy', date: '2026-01-15' }).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); }); it('should handle errors', async () => { investmentRepository.getById.mockResolvedValue({ id: 1, currency: 'EUR' }); portfolioTransactionRepository.create.mockRejectedValue(new Error('DB error')); - const req = { params: { id: '1' }, body: { type: 'buy', date: '2026-01-15', amount: 1000 } }; - const res = mockResponse(); - await expect(routeHandlers['post:/:id/transactions'](req, res)).rejects.toThrow('DB error'); + const res = await api.post(`${BASE}/1/transactions`).send({ type: 'buy', date: '2026-01-15', amount: 1000 }).expect(500); + expect(res.body).toEqual(errEnvelope({ code: 'INTERNAL_SERVER_ERROR', message: 'DB error' })); }); }); @@ -595,34 +504,27 @@ describe('Investment Routes', () => { portfolioTransactionRepository.getById.mockResolvedValue({ id: 1, investment_id: 10 }); portfolioTransactionRepository.hardDelete.mockResolvedValue(true); - const req = { params: { txnId: '1' } }; - const res = mockResponse(); - await routeHandlers['delete:/transactions/:txnId'](req, res); - - expect(res.status).toHaveBeenCalledWith(204); + await api.delete(`${BASE}/transactions/1`).expect(204); }); it('should throw NotFoundError for non-existent', async () => { portfolioTransactionRepository.getById.mockResolvedValue(null); - const req = { params: { txnId: '999' } }; - const res = mockResponse(); - await expect(routeHandlers['delete:/transactions/:txnId'](req, res)).rejects.toBeInstanceOf(NotFoundError); + const res = await api.delete(`${BASE}/transactions/999`).expect(404); + expect(res.body).toEqual(errEnvelope({ code: 'NOT_FOUND' })); }); it('should throw ValidationError for invalid ID', async () => { - const req = { params: { txnId: 'abc' } }; - const res = mockResponse(); - await expect(routeHandlers['delete:/transactions/:txnId'](req, res)).rejects.toBeInstanceOf(ValidationError); + const res = await api.delete(`${BASE}/transactions/abc`).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); }); it('should handle errors', async () => { portfolioTransactionRepository.getById.mockResolvedValue({ id: 1, investment_id: 10 }); portfolioTransactionRepository.hardDelete.mockRejectedValue(new Error('DB error')); - const req = { params: { txnId: '1' } }; - const res = mockResponse(); - await expect(routeHandlers['delete:/transactions/:txnId'](req, res)).rejects.toThrow('DB error'); + const res = await api.delete(`${BASE}/transactions/1`).expect(500); + expect(res.body).toEqual(errEnvelope({ code: 'INTERNAL_SERVER_ERROR', message: 'DB error' })); }); }); @@ -631,34 +533,29 @@ describe('Investment Routes', () => { it('should update portfolio transaction', async () => { portfolioTransactionRepository.update.mockResolvedValue({ id: 1, amount: 1200 }); - const req = { params: { txnId: '1' }, body: { amount: 1200 } }; - const res = mockResponse(); - await routeHandlers['patch:/transactions/:txnId'](req, res); + const res = await api.patch(`${BASE}/transactions/1`).send({ amount: 1200 }).expect(200); expect(portfolioTransactionRepository.update).toHaveBeenCalledWith(1, { amount: 1200 }); - expect(res.json).toHaveBeenCalledWith({ ok: true, data: { id: 1, amount: 1200 } }); + expect(res.body).toEqual(okEnvelope({ id: 1, amount: 1200 })); }); it('should throw NotFoundError for non-existent transaction', async () => { portfolioTransactionRepository.update.mockResolvedValue(null); - const req = { params: { txnId: '999' }, body: { amount: 1200 } }; - const res = mockResponse(); - await expect(routeHandlers['patch:/transactions/:txnId'](req, res)).rejects.toBeInstanceOf(NotFoundError); + const res = await api.patch(`${BASE}/transactions/999`).send({ amount: 1200 }).expect(404); + expect(res.body).toEqual(errEnvelope({ code: 'NOT_FOUND' })); }); it('should throw ValidationError for invalid ID', async () => { - const req = { params: { txnId: 'abc' }, body: { amount: 1200 } }; - const res = mockResponse(); - await expect(routeHandlers['patch:/transactions/:txnId'](req, res)).rejects.toBeInstanceOf(ValidationError); + const res = await api.patch(`${BASE}/transactions/abc`).send({ amount: 1200 }).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); }); it('should handle errors', async () => { portfolioTransactionRepository.update.mockRejectedValue(new Error('DB error')); - const req = { params: { txnId: '1' }, body: { amount: 1200 } }; - const res = mockResponse(); - await expect(routeHandlers['patch:/transactions/:txnId'](req, res)).rejects.toThrow('DB error'); + const res = await api.patch(`${BASE}/transactions/1`).send({ amount: 1200 }).expect(500); + expect(res.body).toEqual(errEnvelope({ code: 'INTERNAL_SERVER_ERROR', message: 'DB error' })); }); it('should throw ValidationError for validation errors', async () => { @@ -666,25 +563,22 @@ describe('Investment Routes', () => { err.code = 'VALIDATION_ERROR'; portfolioTransactionRepository.update.mockRejectedValue(err); - const req = { params: { txnId: '1' }, body: { amount: 1200 } }; - const res = mockResponse(); - await expect(routeHandlers['patch:/transactions/:txnId'](req, res)).rejects.toBeInstanceOf(ValidationError); + const res = await api.patch(`${BASE}/transactions/1`).send({ amount: 1200 }).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); }); it('rejects a free-text or cleared currency and uppercases a valid one', async () => { // PATCH forwarded the raw value to the VARCHAR(10) column (create // validates): garbage stored, >10 chars 22001'd, null hit NOT NULL. for (const currency of ['euro', null, '']) { - const req = { params: { txnId: '1' }, body: { currency } }; - await expect(routeHandlers['patch:/transactions/:txnId'](req, mockResponse())) - .rejects.toBeInstanceOf(ValidationError); + const res = await api.patch(`${BASE}/transactions/1`).send({ currency }).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); } expect(portfolioTransactionRepository.update).not.toHaveBeenCalled(); portfolioTransactionRepository.update.mockResolvedValue({ id: 1, investment_id: 10, currency: 'USD' }); // fx_rate_to_eur supplied explicitly → the fx recompute path is skipped. - const req = { params: { txnId: '1' }, body: { currency: 'usd', fx_rate_to_eur: 0.9 } }; - await routeHandlers['patch:/transactions/:txnId'](req, mockResponse()); + await api.patch(`${BASE}/transactions/1`).send({ currency: 'usd', fx_rate_to_eur: 0.9 }).expect(200); expect(portfolioTransactionRepository.update).toHaveBeenCalledWith( 1, expect.objectContaining({ currency: 'USD' }), @@ -699,11 +593,9 @@ describe('Investment Routes', () => { { type: 'buy', total_amount: 5000, count: 3 }, ]); - const req = { params: { id: '1' } }; - const res = mockResponse(); - await routeHandlers['get:/:id/summary'](req, res); + const res = await api.get(`${BASE}/1/summary`).expect(200); - const data = res.json.mock.calls[0][0].data; + const data = res.body.data; expect(data.investment_id).toBe(1); expect(data.breakdown).toHaveLength(1); }); @@ -711,9 +603,8 @@ describe('Investment Routes', () => { it('should handle errors', async () => { portfolioTransactionRepository.getSummary.mockRejectedValue(new Error('DB error')); - const req = { params: { id: '1' } }; - const res = mockResponse(); - await expect(routeHandlers['get:/:id/summary'](req, res)).rejects.toThrow('DB error'); + const res = await api.get(`${BASE}/1/summary`).expect(500); + expect(res.body).toEqual(errEnvelope({ code: 'INTERNAL_SERVER_ERROR', message: 'DB error' })); }); }); }); diff --git a/apps/node-backend/tests/routes/marketLookup.test.js b/apps/node-backend/tests/routes/marketLookup.test.js index 5939dcd0..f78c6602 100644 --- a/apps/node-backend/tests/routes/marketLookup.test.js +++ b/apps/node-backend/tests/routes/marketLookup.test.js @@ -1,19 +1,27 @@ +/** + * Market Lookup route tests. + * + * Runs against the REAL router mounted on a throwaway Express app (see + * tests/helpers/routeApp.js). All validation in marketLookup.js is inline in + * the handlers (no route-level guard middleware), so this migration is mostly + * mechanical: `routeHandlers[...]` calls become supertest requests and + * `.rejects.toBeInstanceOf(...)` assertions become status-code + envelope + * assertions against the real error handler. + * + * Per the routeApp.js fidelity map, the app-level `marketRateLimiter` + * (main.js:329, mounted before this router for both `/api/market` and + * `/api/research`) is a module-scoped per-IP counter and is deliberately NOT + * reproduced here — it would 429 this suite's own many requests. + */ import { beforeEach, describe, expect, it, vi } from 'vitest'; import { mockLogger } from '../helpers/mockLogger.js'; -import { createMockRouter, createMockResponse } from '../helpers/routeHarness.js'; - -const { router: mockRouter, handlers: routeHandlers } = createMockRouter(); +import { routeAgent, okEnvelope, errEnvelope } from '../helpers/routeApp.js'; const mockYahooSearch = vi.fn(); const mockYahooQuote = vi.fn(); const mockYahooQuoteSummary = vi.fn(); const mockYahooChart = vi.fn(); -vi.mock('express', () => ({ - default: { Router: () => mockRouter }, - Router: () => mockRouter, -})); - vi.mock('yahoo-finance2', () => ({ default: vi.fn().mockImplementation(function MockYahooFinance() { return { @@ -29,12 +37,10 @@ vi.mock('../../src/config/logger.js', () => ({ logger: mockLogger(), })); -import { ValidationError, AppError } from '../../src/middleware/errorHandler.js'; -const { __clearQuoteCacheForTests } = await import('../../src/routes/marketLookup.js'); +const { default: marketLookupRouter, __clearQuoteCacheForTests } = await import('../../src/routes/marketLookup.js'); -function mockResponse() { - return createMockResponse(); -} +const api = routeAgent(marketLookupRouter, { mountPath: '/api/market' }); +const BASE = '/api/market'; describe('Market Lookup Routes', () => { beforeEach(() => { @@ -46,10 +52,8 @@ describe('Market Lookup Routes', () => { describe('GET /quote', () => { it('should throw ValidationError when symbols query parameter is missing', async () => { - const req = { query: {} }; - const res = mockResponse(); - - await expect(routeHandlers['get:/quote'](req, res)).rejects.toBeInstanceOf(ValidationError); + const res = await api.get(`${BASE}/quote`).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); }); it('should map quote and summary fields into API response', async () => { @@ -92,12 +96,9 @@ describe('Market Lookup Routes', () => { }, }); - const req = { query: { symbols: 'AAPL' } }; - const res = mockResponse(); - - await routeHandlers['get:/quote'](req, res); + const res = await api.get(`${BASE}/quote`).query({ symbols: 'AAPL' }).expect(200); - const body = res.json.mock.calls[0][0]; + const body = res.body; expect(body.data.items).toHaveLength(1); expect(body.data.items[0]).toMatchObject({ symbol: 'AAPL', @@ -131,13 +132,10 @@ describe('Market Lookup Routes', () => { regularMarketPreviousClose: 189.01, }); - const req = { query: { symbols: 'AAPL', detail: 'basic' } }; - const res = mockResponse(); - - await routeHandlers['get:/quote'](req, res); + const res = await api.get(`${BASE}/quote`).query({ symbols: 'AAPL', detail: 'basic' }).expect(200); expect(mockYahooQuoteSummary).not.toHaveBeenCalled(); - const quote = res.json.mock.calls[0][0].data.items[0]; + const quote = res.body.data.items[0]; expect(quote).toMatchObject({ symbol: 'AAPL', name: 'Apple Inc.', @@ -155,23 +153,17 @@ describe('Market Lookup Routes', () => { mockYahooQuote.mockRejectedValue(new Error('upstream quote failure')); mockYahooQuoteSummary.mockResolvedValue({}); - const req = { query: { symbols: 'AAPL,MSFT' } }; - const res = mockResponse(); + const res = await api.get(`${BASE}/quote`).query({ symbols: 'AAPL,MSFT' }).expect(200); - await routeHandlers['get:/quote'](req, res); - - expect(res.json).toHaveBeenCalledWith({ ok: true, data: { items: [], total: 0 } }); + expect(res.body).toEqual(okEnvelope({ items: [], total: 0 })); }); }); describe('GET /search', () => { it('returns empty list for empty query', async () => { - const req = { query: { q: '' } }; - const res = mockResponse(); - - await routeHandlers['get:/search'](req, res); + const res = await api.get(`${BASE}/search`).query({ q: '' }).expect(200); - expect(res.json).toHaveBeenCalledWith({ ok: true, data: { items: [] } }); + expect(res.body).toEqual(okEnvelope({ items: [] })); expect(mockYahooSearch).not.toHaveBeenCalled(); }); @@ -193,44 +185,36 @@ describe('Market Lookup Routes', () => { ], }); - const req = { query: { q: 'apple' } }; - const res = mockResponse(); - - await routeHandlers['get:/search'](req, res); + const res = await api.get(`${BASE}/search`).query({ q: 'apple' }).expect(200); expect(mockYahooSearch).toHaveBeenCalledWith( 'apple', { quotesCount: 8, newsCount: 0 }, { validateResult: false }, ); - expect(res.json).toHaveBeenCalledWith({ - ok: true, - data: { - items: [ - { - symbol: 'AAPL', - name: 'Apple Inc.', - type: 'EQUITY', - exchange: 'NasdaqGS', - }, - { - symbol: 'VUSA.AS', - name: 'Vanguard S&P 500 UCITS ETF', - type: 'ETF', - exchange: 'AEX', - }, - ], - }, - }); + expect(res.body).toEqual(okEnvelope({ + items: [ + { + symbol: 'AAPL', + name: 'Apple Inc.', + type: 'EQUITY', + exchange: 'NasdaqGS', + }, + { + symbol: 'VUSA.AS', + name: 'Vanguard S&P 500 UCITS ETF', + type: 'ETF', + exchange: 'AEX', + }, + ], + })); }); it('throws AppError (502) when upstream search throws', async () => { mockYahooSearch.mockRejectedValue(new Error('upstream down')); - const req = { query: { q: 'apple' } }; - const res = mockResponse(); - - await expect(routeHandlers['get:/search'](req, res)).rejects.toBeInstanceOf(AppError); + const res = await api.get(`${BASE}/search`).query({ q: 'apple' }).expect(502); + expect(res.body).toEqual(errEnvelope({ code: 'BAD_GATEWAY' })); }); it('filters entries without symbol and applies fallback fields', async () => { @@ -241,45 +225,34 @@ describe('Market Lookup Routes', () => { ], }); - const req = { query: { q: 'tsla' } }; - const res = mockResponse(); - - await routeHandlers['get:/search'](req, res); + const res = await api.get(`${BASE}/search`).query({ q: 'tsla' }).expect(200); - expect(res.json).toHaveBeenCalledWith({ - ok: true, - data: { - items: [ - { - symbol: 'TSLA', - name: 'TSLA', - type: 'UNKNOWN', - exchange: '', - }, - ], - }, - }); + expect(res.body).toEqual(okEnvelope({ + items: [ + { + symbol: 'TSLA', + name: 'TSLA', + type: 'UNKNOWN', + exchange: '', + }, + ], + })); }); }); describe('GET /chart', () => { it('throws ValidationError when symbol is missing', async () => { - const req = { query: {} }; - const res = mockResponse(); - - await expect(routeHandlers['get:/chart'](req, res)).rejects.toBeInstanceOf(ValidationError); + const res = await api.get(`${BASE}/chart`).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); expect(mockYahooChart).not.toHaveBeenCalled(); }); it('returns empty points when chart payload is empty', async () => { mockYahooChart.mockResolvedValue(null); - const req = { query: { symbol: 'AAPL' } }; - const res = mockResponse(); + const res = await api.get(`${BASE}/chart`).query({ symbol: 'AAPL' }).expect(200); - await routeHandlers['get:/chart'](req, res); - - expect(res.json).toHaveBeenCalledWith({ ok: true, data: { items: [], total: 0 } }); + expect(res.body).toEqual(okEnvelope({ items: [], total: 0 })); }); it('maps chart points and filters null closes', async () => { @@ -291,12 +264,9 @@ describe('Market Lookup Routes', () => { ], }); - const req = { query: { symbol: 'AAPL', range: '5y', interval: '1wk' } }; - const res = mockResponse(); - - await routeHandlers['get:/chart'](req, res); + const res = await api.get(`${BASE}/chart`).query({ symbol: 'AAPL', range: '5y', interval: '1wk' }).expect(200); - const payload = res.json.mock.calls[0][0].data; + const payload = res.body.data; expect(payload.symbol).toBe('AAPL'); expect(payload.currency).toBe('USD'); expect(payload.items).toHaveLength(1); @@ -305,10 +275,9 @@ describe('Market Lookup Routes', () => { it('throws AppError (502) when chart request crashes', async () => { mockYahooChart.mockRejectedValue(new Error('chart error')); - const req = { query: { symbol: 'AAPL', range: 'max' } }; - const res = mockResponse(); - await expect(routeHandlers['get:/chart'](req, res)).rejects.toBeInstanceOf(AppError); + const res = await api.get(`${BASE}/chart`).query({ symbol: 'AAPL', range: 'max' }).expect(502); + expect(res.body).toEqual(errEnvelope({ code: 'BAD_GATEWAY' })); }); it('passes validateResult:false so incomplete Yahoo meta degrades instead of 502', async () => { @@ -319,18 +288,14 @@ describe('Market Lookup Routes', () => { ], }); - const req = { query: { symbol: 'KAU_EUR' } }; - const res = mockResponse(); - - await routeHandlers['get:/chart'](req, res); + const res = await api.get(`${BASE}/chart`).query({ symbol: 'KAU_EUR' }).expect(200); expect(mockYahooChart).toHaveBeenCalledWith( 'KAU_EUR', expect.any(Object), { validateResult: false }, ); - const payload = res.json.mock.calls[0][0].data; - expect(payload.items).toHaveLength(1); + expect(res.body.data.items).toHaveLength(1); }); }); @@ -367,12 +332,9 @@ describe('Market Lookup Routes', () => { ], }); - const req = { query: { symbols: 'AAPL,MSFT', count: '5' } }; - const res = mockResponse(); - - await routeHandlers['get:/news'](req, res); + const res = await api.get(`${BASE}/news`).query({ symbols: 'AAPL,MSFT', count: '5' }).expect(200); - const body = res.json.mock.calls[0][0].data; + const body = res.body.data; expect(body.items).toHaveLength(2); expect(body.items[0].title).toBe('Unique headline'); expect(body.items[0].thumbnail).toBe('https://img.example.com/c.jpg'); @@ -383,13 +345,9 @@ describe('Market Lookup Routes', () => { it('should tolerate yahoo search failures and return empty results', async () => { mockYahooSearch.mockRejectedValue(new Error('search exploded')); - const req = { query: { symbols: 'AAPL' } }; - const res = mockResponse(); - - await routeHandlers['get:/news'](req, res); + const res = await api.get(`${BASE}/news`).query({ symbols: 'AAPL' }).expect(200); - expect(res.status).not.toHaveBeenCalled(); - expect(res.json).toHaveBeenCalledWith({ ok: true, data: { items: [], total: 0 } }); + expect(res.body).toEqual(okEnvelope({ items: [], total: 0 })); }); it('uses default symbols and caps count at 50', async () => { @@ -397,10 +355,7 @@ describe('Market Lookup Routes', () => { news: [{ title: 'One', link: 'l', publisher: 'p', providerPublishTime: 1, thumbnail: {} }], }); - const req = { query: { count: '500' } }; - const res = mockResponse(); - - await routeHandlers['get:/news'](req, res); + const res = await api.get(`${BASE}/news`).query({ count: '500' }).expect(200); expect(mockYahooSearch).toHaveBeenCalledTimes(3); expect(mockYahooSearch).toHaveBeenCalledWith( @@ -408,19 +363,15 @@ describe('Market Lookup Routes', () => { { quotesCount: 0, newsCount: 50 }, { validateResult: false }, ); - expect(res.json).toHaveBeenCalled(); }); it('coerces a repeated (array) symbols param instead of throwing', async () => { // Express parses `?symbols=AAPL&symbols=MSFT` as an array — the route // now joins it to a string rather than letting `.split` throw a 502. mockYahooSearch.mockResolvedValue({ news: [] }); - const req = { query: { symbols: ['AAPL', 'MSFT'] } }; - const res = mockResponse(); - await routeHandlers['get:/news'](req, res); + await api.get(`${BASE}/news`).query('symbols=AAPL&symbols=MSFT').expect(200); - expect(res.json).toHaveBeenCalled(); expect(mockYahooSearch).toHaveBeenCalledWith('AAPL', expect.anything(), expect.anything()); expect(mockYahooSearch).toHaveBeenCalledWith('MSFT', expect.anything(), expect.anything()); }); @@ -430,12 +381,8 @@ describe('Market Lookup Routes', () => { it('coerces a repeated (array) symbols param instead of throwing', async () => { mockYahooQuote.mockResolvedValue({ symbol: 'AAPL', regularMarketPrice: 1 }); mockYahooQuoteSummary.mockResolvedValue({}); - const req = { query: { symbols: ['AAPL', 'MSFT'] } }; - const res = mockResponse(); - - await routeHandlers['get:/quote'](req, res); - expect(res.json).toHaveBeenCalled(); + await api.get(`${BASE}/quote`).query('symbols=AAPL&symbols=MSFT').expect(200); }); it('uses summary fallback buckets and quote defaults', async () => { @@ -445,12 +392,9 @@ describe('Market Lookup Routes', () => { upgradeDowngradeHistory: { history: Array.from({ length: 12 }, (_, i) => ({ epochGradeDate: i, firm: `F${i}`, toGrade: 'Buy', action: 'up' })) }, }); - const req = { query: { symbols: 'MSFT' } }; - const res = mockResponse(); - - await routeHandlers['get:/quote'](req, res); + const res = await api.get(`${BASE}/quote`).query({ symbols: 'MSFT' }).expect(200); - const quote = res.json.mock.calls[0][0].data.items[0]; + const quote = res.body.data.items[0]; expect(quote.name).toBe('MSFT'); expect(quote.currency).toBe('USD'); expect(quote.analystConsensus).toEqual({ strongBuy: 1, buy: 2, hold: 3, sell: 4, strongSell: 5 }); @@ -466,12 +410,9 @@ describe('Market Lookup Routes', () => { }); mockYahooQuoteSummary.mockRejectedValue(new Error('summary unavailable')); - const req = { query: { symbols: 'NVDA' } }; - const res = mockResponse(); + const res = await api.get(`${BASE}/quote`).query({ symbols: 'NVDA' }).expect(200); - await routeHandlers['get:/quote'](req, res); - - const quote = res.json.mock.calls[0][0].data.items[0]; + const quote = res.body.data.items[0]; expect(quote.symbol).toBe('NVDA'); expect(quote.price).toBe(900); expect(quote.analystConsensus).toBeNull(); @@ -485,10 +426,7 @@ describe('Market Lookup Routes', () => { const ranges = ['1d', '5d', '1mo', '3mo', '6mo', '1y', '2y', '5y', 'max', 'unknown']; for (const range of ranges) { - const req = { query: { symbol: 'AAPL', range, interval: '1d' } }; - const res = mockResponse(); - await routeHandlers['get:/chart'](req, res); - expect(res.status).not.toHaveBeenCalled(); + await api.get(`${BASE}/chart`).query({ symbol: 'AAPL', range, interval: '1d' }).expect(200); } expect(mockYahooChart).toHaveBeenCalledTimes(ranges.length); @@ -509,12 +447,9 @@ describe('Market Lookup Routes', () => { ], }); - const req = { query: { symbols: 'AAPL', count: '1' } }; - const res = mockResponse(); - - await routeHandlers['get:/news'](req, res); + const res = await api.get(`${BASE}/news`).query({ symbols: 'AAPL', count: '1' }).expect(200); - const article = res.json.mock.calls[0][0].data.items[0]; + const article = res.body.data.items[0]; expect(article.thumbnail).toBeNull(); }); }); diff --git a/apps/node-backend/tests/routes/plannedTransactions.test.js b/apps/node-backend/tests/routes/plannedTransactions.test.js index acdd6882..34ccf406 100644 --- a/apps/node-backend/tests/routes/plannedTransactions.test.js +++ b/apps/node-backend/tests/routes/plannedTransactions.test.js @@ -1,17 +1,15 @@ /** * Planned transaction route tests. * Mirrors: apps/backend/tests/test_planned_transactions.py + * + * Runs against the REAL router mounted on a throwaway Express app (see + * tests/helpers/routeApp.js). Notably `validateIdParam` is no longer stubbed — + * the guard that the old mock-router harness dropped from the chain now runs on + * every `/:id` route here. */ import { describe, it, expect, vi, beforeEach } from 'vitest'; import { mockLogger } from '../helpers/mockLogger.js'; -import { createMockRouter, createMockResponse } from '../helpers/routeHarness.js'; - -const { router: mockRouter, handlers: routeHandlers } = createMockRouter(); - -vi.mock('express', () => ({ - default: { Router: () => mockRouter }, - Router: () => mockRouter, -})); +import { routeAgent } from '../helpers/routeApp.js'; vi.mock('../../src/repositories/plannedTransactionRepository.js', () => ({ default: { @@ -31,56 +29,30 @@ vi.mock('../../src/database/connection.js', () => ({ query: vi.fn(), })); -vi.mock('../../src/services/loanRepaymentService.js', () => ({ - generateLoanRepaymentSchedule: vi.fn(() => ({ - regular_payment_amount: 850, - first_due_date: '2026-04-01', - schedule: [ - { - installment_number: 1, - due_date: '2026-04-01', - payment_amount: 850, - principal_amount: 700, - interest_amount: 150, - remaining_principal: 9300, - }, - ], - })), -})); - -vi.mock('../../src/middleware/validation.js', async (importOriginal) => ({ - // Keep the real helpers (assertYmd, validateId, …); only stub the middleware. - ...(await importOriginal()), - validateIdParam: (_req, _res, next) => next(), -})); - +// The per-route PATCH limiter (routes/plannedTransactions.js:417) is stubbed +// here on purpose: this suite issues ~28 PATCHes against one in-memory counter +// keyed by IP, so the real 30/min ceiling would make the file self-throttling +// and flaky as tests are added. The transactions suites exercise the real +// limiter chain. Every OTHER middleware on the chain is real. vi.mock('../../src/middleware/rateLimiter.js', () => ({ rateLimiter: () => (_req, _res, next) => next(), })); -vi.mock('../../src/services/recurrenceService.js', () => ({ - calculateNextDate: vi.fn((base, pattern) => { - if (pattern === 'monthly') { - const d = new Date(base); - d.setMonth(d.getMonth() + 1); - return d; - } - return null; - }), -})); - vi.mock('../../src/config/logger.js', () => ({ logger: mockLogger(), })); import plannedTransactionRepository from '../../src/repositories/plannedTransactionRepository.js'; import { query as dbQuery } from '../../src/database/connection.js'; -import { ValidationError, NotFoundError } from '../../src/middleware/errorHandler.js'; -await import('../../src/routes/plannedTransactions.js'); -function mockResponse() { - return createMockResponse({ set: vi.fn() }); -} +const { default: plannedRouter } = await import('../../src/routes/plannedTransactions.js'); + +const api = routeAgent(plannedRouter, { mountPath: '/api/planned-transactions' }); + +const BASE = '/api/planned-transactions'; +const post = (body) => api.post(`${BASE}/`).send(body); +const patch = (id, body) => api.patch(`${BASE}/${id}`).send(body); +const execute = (id, body) => api.post(`${BASE}/${id}/execute`).send(body); describe('Planned Transaction Routes', () => { beforeEach(() => vi.clearAllMocks()); @@ -89,13 +61,12 @@ describe('Planned Transaction Routes', () => { it('should return empty list', async () => { plannedTransactionRepository.getAll.mockResolvedValue({ items: [], total: 0 }); - const req = { query: {} }; - const res = mockResponse(); - await routeHandlers['get:/'](req, res); + const res = await api.get(`${BASE}/`).expect(200); - const result = res.json.mock.calls[0][0]; - expect(result.data.items).toEqual([]); - expect(result.data.total).toBe(0); + expect(res.body.ok).toBe(true); + expect(res.body.data.items).toEqual([]); + expect(res.body.data.total).toBe(0); + expect(res.body.meta.requestId).toEqual(expect.any(String)); }); it('should return planned transactions', async () => { @@ -104,31 +75,24 @@ describe('Planned Transaction Routes', () => { total: 1, }); - const req = { query: {} }; - const res = mockResponse(); - await routeHandlers['get:/'](req, res); + const res = await api.get(`${BASE}/`).expect(200); - expect(res.json.mock.calls[0][0].data.total).toBe(1); + expect(res.body.data.total).toBe(1); }); it('should respect pagination', async () => { plannedTransactionRepository.getAll.mockResolvedValue({ items: [], total: 10 }); - const req = { query: { limit: '5', offset: '2' } }; - const res = mockResponse(); - await routeHandlers['get:/'](req, res); + const res = await api.get(`${BASE}/?limit=5&offset=2`).expect(200); - const data = res.json.mock.calls[0][0].data; - expect(data.limit).toBe(5); - expect(data.offset).toBe(2); + expect(res.body.data.limit).toBe(5); + expect(res.body.data.offset).toBe(2); }); it('should filter by is_recurring', async () => { plannedTransactionRepository.getAll.mockResolvedValue({ items: [], total: 0 }); - const req = { query: { is_recurring: 'true' } }; - const res = mockResponse(); - await routeHandlers['get:/'](req, res); + await api.get(`${BASE}/?is_recurring=true`).expect(200); expect(plannedTransactionRepository.getAll).toHaveBeenCalledWith( expect.objectContaining({ isRecurring: true }) @@ -138,9 +102,7 @@ describe('Planned Transaction Routes', () => { it('should filter by is_executed', async () => { plannedTransactionRepository.getAll.mockResolvedValue({ items: [], total: 0 }); - const req = { query: { is_executed: 'false' } }; - const res = mockResponse(); - await routeHandlers['get:/'](req, res); + await api.get(`${BASE}/?is_executed=false`).expect(200); expect(plannedTransactionRepository.getAll).toHaveBeenCalledWith( expect.objectContaining({ isExecuted: false }) @@ -155,39 +117,32 @@ describe('Planned Transaction Routes', () => { is_recurring: false, is_executed: false, }); - const req = { body: { planned_date: '2026-03-15', bank_account: 'Chase', amount: 50 } }; - const res = mockResponse(); - await routeHandlers['post:/'](req, res); + const res = await post({ planned_date: '2026-03-15', bank_account: 'Chase', amount: 50 }) + .expect(201); - expect(res.status).toHaveBeenCalledWith(201); + expect(res.body.ok).toBe(true); + expect(res.body.data.id).toBe(1); }); - it('should throw ValidationError for missing fields', async () => { - const req = { body: { amount: 50 } }; - const res = mockResponse(); - await expect(routeHandlers['post:/'](req, res)).rejects.toBeInstanceOf(ValidationError); + it('should return a 400 VALIDATION_ERROR envelope for missing fields', async () => { + const res = await post({ amount: 50 }).expect(400); + expect(res.body.error.code).toBe('VALIDATION_ERROR'); }); it('rejects a zero amount (meaningless, never auto-matches)', async () => { - const req = { body: { planned_date: '2026-03-15', bank_account: 'Chase', amount: 0 } }; - const res = mockResponse(); - await expect(routeHandlers['post:/'](req, res)).rejects.toBeInstanceOf(ValidationError); + await post({ planned_date: '2026-03-15', bank_account: 'Chase', amount: 0 }).expect(400); expect(plannedTransactionRepository.create).not.toHaveBeenCalled(); }); it('rejects an absurd amount above the money-column ceiling', async () => { - const req = { body: { planned_date: '2026-03-15', bank_account: 'Chase', amount: 1e15 } }; - const res = mockResponse(); - await expect(routeHandlers['post:/'](req, res)).rejects.toBeInstanceOf(ValidationError); + await post({ planned_date: '2026-03-15', bank_account: 'Chase', amount: 1e15 }).expect(400); expect(plannedTransactionRepository.create).not.toHaveBeenCalled(); }); it('rejects a negative reminder_days_before', async () => { - const req = { - body: { planned_date: '2026-03-15', bank_account: 'Chase', amount: 50, reminder_days_before: -1 }, - }; - const res = mockResponse(); - await expect(routeHandlers['post:/'](req, res)).rejects.toBeInstanceOf(ValidationError); + await post({ + planned_date: '2026-03-15', bank_account: 'Chase', amount: 50, reminder_days_before: -1, + }).expect(400); expect(plannedTransactionRepository.create).not.toHaveBeenCalled(); }); @@ -202,22 +157,17 @@ describe('Planned Transaction Routes', () => { is_executed: false, }); - const req = { - body: { - bank_account: 'Mortgage', - is_loan: true, - loan_type: 'amortizing', - loan_principal: 10000, - loan_annual_interest_rate: 6, - loan_term_months: 12, - loan_start_date: '2026-04-01', - loan_payment_day: 1, - }, - }; - const res = mockResponse(); - await routeHandlers['post:/'](req, res); - - expect(res.status).toHaveBeenCalledWith(201); + await post({ + bank_account: 'Mortgage', + is_loan: true, + loan_type: 'amortizing', + loan_principal: 10000, + loan_annual_interest_rate: 6, + loan_term_months: 12, + loan_start_date: '2026-04-01', + loan_payment_day: 1, + }).expect(201); + expect(plannedTransactionRepository.create).toHaveBeenCalledWith( expect.objectContaining({ is_loan: true, @@ -227,27 +177,19 @@ describe('Planned Transaction Routes', () => { ); }); - it('rejects an invalid recurrence_pattern (fortnightly) with ValidationError', async () => { - const req = { - body: { - planned_date: '2026-03-15', bank_account: 'Chase', amount: 50, - is_recurring: true, recurrence_pattern: 'fortnightly', - }, - }; - const res = mockResponse(); - await expect(routeHandlers['post:/'](req, res)).rejects.toBeInstanceOf(ValidationError); + it('rejects an invalid recurrence_pattern (fortnightly) with a 400', async () => { + await post({ + planned_date: '2026-03-15', bank_account: 'Chase', amount: 50, + is_recurring: true, recurrence_pattern: 'fortnightly', + }).expect(400); expect(plannedTransactionRepository.create).not.toHaveBeenCalled(); }); it('rejects is_recurring:true with no recurrence_pattern (would be perpetually due)', async () => { - const req = { - body: { - planned_date: '2026-03-15', bank_account: 'Chase', amount: 50, - is_recurring: true, - }, - }; - const res = mockResponse(); - await expect(routeHandlers['post:/'](req, res)).rejects.toBeInstanceOf(ValidationError); + await post({ + planned_date: '2026-03-15', bank_account: 'Chase', amount: 50, + is_recurring: true, + }).expect(400); expect(plannedTransactionRepository.create).not.toHaveBeenCalled(); }); @@ -256,15 +198,11 @@ describe('Planned Transaction Routes', () => { id: 9, planned_date: '2026-03-15', amount: '50.00', bank_account: 'Chase', is_recurring: true, is_executed: false, }); - const req = { - body: { - planned_date: '2026-03-15', bank_account: 'Chase', amount: 50, - is_recurring: true, recurrence_pattern: 'every 10 days', - }, - }; - const res = mockResponse(); - await routeHandlers['post:/'](req, res); - expect(res.status).toHaveBeenCalledWith(201); + + await post({ + planned_date: '2026-03-15', bank_account: 'Chase', amount: 50, + is_recurring: true, recurrence_pattern: 'every 10 days', + }).expect(201); }); it('stores a loan as a monthly recurrence so /execute advances it', async () => { @@ -273,34 +211,25 @@ describe('Planned Transaction Routes', () => { is_loan: true, is_recurring: true, recurrence_pattern: 'monthly', is_executed: false, }); - const req = { - body: { - bank_account: 'Mortgage', is_loan: true, loan_type: 'amortizing', - loan_principal: 10000, loan_annual_interest_rate: 6, loan_term_months: 12, - loan_start_date: '2026-04-01', loan_payment_day: 1, - // The frontend may inject a display string; the route must replace it. - recurrence_pattern: 'loan(12 months)', - }, - }; - const res = mockResponse(); - await routeHandlers['post:/'](req, res); - - expect(res.status).toHaveBeenCalledWith(201); + await post({ + bank_account: 'Mortgage', is_loan: true, loan_type: 'amortizing', + loan_principal: 10000, loan_annual_interest_rate: 6, loan_term_months: 12, + loan_start_date: '2026-04-01', loan_payment_day: 1, + // The frontend may inject a display string; the route must replace it. + recurrence_pattern: 'loan(12 months)', + }).expect(201); + expect(plannedTransactionRepository.create).toHaveBeenCalledWith( expect.objectContaining({ is_loan: true, is_recurring: true, recurrence_pattern: 'monthly' }), ); }); - it('should throw ValidationError when loan_term_months is out of bounds', async () => { - const req = { - body: { - bank_account: 'Mortgage', - is_loan: true, - loan_term_months: 601, - }, - }; - const res = mockResponse(); - await expect(routeHandlers['post:/'](req, res)).rejects.toBeInstanceOf(ValidationError); + it('should return 400 when loan_term_months is out of bounds', async () => { + await post({ + bank_account: 'Mortgage', + is_loan: true, + loan_term_months: 601, + }).expect(400); expect(plannedTransactionRepository.create).not.toHaveBeenCalled(); }); }); @@ -318,35 +247,31 @@ describe('Planned Transaction Routes', () => { }); it('coerces a string amount to a number before the repository', async () => { - const req = { body: { ...validBody, amount: '50.5' } }; - await routeHandlers['post:/'](req, mockResponse()); + await post({ ...validBody, amount: '50.5' }).expect(201); expect(plannedTransactionRepository.create).toHaveBeenCalledWith( expect.objectContaining({ amount: 50.5 }), ); }); it('rejects an empty-string amount (coerces to 0, which is meaningless)', async () => { - const req = { body: { ...validBody, amount: '' } }; - await expect(routeHandlers['post:/'](req, mockResponse())).rejects.toBeInstanceOf(ValidationError); + await post({ ...validBody, amount: '' }).expect(400); }); it('rejects non-array tags (including null)', async () => { - await expect(routeHandlers['post:/']({ body: { ...validBody, tags: 'groceries' } }, mockResponse())) - .rejects.toBeInstanceOf(ValidationError); - await expect(routeHandlers['post:/']({ body: { ...validBody, tags: null } }, mockResponse())) - .rejects.toBeInstanceOf(ValidationError); + await post({ ...validBody, tags: 'groceries' }).expect(400); + await post({ ...validBody, tags: null }).expect(400); }); it('coerces reminder_days_before and accepts the 0/365 boundaries', async () => { - await routeHandlers['post:/']({ body: { ...validBody, reminder_days_before: '5' } }, mockResponse()); + await post({ ...validBody, reminder_days_before: '5' }).expect(201); expect(plannedTransactionRepository.create).toHaveBeenCalledWith( expect.objectContaining({ reminder_days_before: 5 }), ); - await routeHandlers['post:/']({ body: { ...validBody, reminder_days_before: 0 } }, mockResponse()); + await post({ ...validBody, reminder_days_before: 0 }).expect(201); expect(plannedTransactionRepository.create).toHaveBeenLastCalledWith( expect.objectContaining({ reminder_days_before: 0 }), ); - await routeHandlers['post:/']({ body: { ...validBody, reminder_days_before: 365 } }, mockResponse()); + await post({ ...validBody, reminder_days_before: 365 }).expect(201); expect(plannedTransactionRepository.create).toHaveBeenLastCalledWith( expect.objectContaining({ reminder_days_before: 365 }), ); @@ -354,17 +279,16 @@ describe('Planned Transaction Routes', () => { it('rejects out-of-range or fractional reminder_days_before', async () => { for (const bad of [366, 2.5, 'abc']) { - await expect(routeHandlers['post:/']({ body: { ...validBody, reminder_days_before: bad } }, mockResponse())) - .rejects.toBeInstanceOf(ValidationError); + await post({ ...validBody, reminder_days_before: bad }).expect(400); } }); it('coerces max_occurrences to an integer and accepts the 1 boundary', async () => { - await routeHandlers['post:/']({ body: { ...validBody, max_occurrences: '3' } }, mockResponse()); + await post({ ...validBody, max_occurrences: '3' }).expect(201); expect(plannedTransactionRepository.create).toHaveBeenCalledWith( expect.objectContaining({ max_occurrences: 3 }), ); - await routeHandlers['post:/']({ body: { ...validBody, max_occurrences: 1 } }, mockResponse()); + await post({ ...validBody, max_occurrences: 1 }).expect(201); expect(plannedTransactionRepository.create).toHaveBeenLastCalledWith( expect.objectContaining({ max_occurrences: 1 }), ); @@ -372,54 +296,47 @@ describe('Planned Transaction Routes', () => { it('rejects non-positive or non-numeric max_occurrences', async () => { for (const bad of [0, -1, 1.5, 'abc']) { - await expect(routeHandlers['post:/']({ body: { ...validBody, max_occurrences: bad } }, mockResponse())) - .rejects.toBeInstanceOf(ValidationError); + await post({ ...validBody, max_occurrences: bad }).expect(400); } }); it('accepts a valid recurrence_end_date unchanged and rejects a malformed one', async () => { - await routeHandlers['post:/']({ body: { ...validBody, recurrence_end_date: '2026-12-31' } }, mockResponse()); + await post({ ...validBody, recurrence_end_date: '2026-12-31' }).expect(201); expect(plannedTransactionRepository.create).toHaveBeenCalledWith( expect.objectContaining({ recurrence_end_date: '2026-12-31' }), ); - await expect(routeHandlers['post:/']({ body: { ...validBody, recurrence_end_date: 'banana' } }, mockResponse())) - .rejects.toBeInstanceOf(ValidationError); + await post({ ...validBody, recurrence_end_date: 'banana' }).expect(400); }); it('normalises currency to uppercase ISO and rejects free text', async () => { // Free-typed "euro" used to be uppercased to "EURO" by the repository and // then violate the 0046 ISO CHECK as a raw 500. - const badReq = { body: { ...validBody, currency: 'euro' } }; - await expect(routeHandlers['post:/'](badReq, mockResponse())).rejects.toBeInstanceOf(ValidationError); + await post({ ...validBody, currency: 'euro' }).expect(400); expect(plannedTransactionRepository.create).not.toHaveBeenCalled(); - const okReq = { body: { ...validBody, currency: 'usd' } }; - await routeHandlers['post:/'](okReq, mockResponse()); + await post({ ...validBody, currency: 'usd' }).expect(201); expect(plannedTransactionRepository.create).toHaveBeenCalledWith( expect.objectContaining({ currency: 'USD' }), ); }); it('maps an absent/empty currency to undefined so the repository default (EUR) applies', async () => { - const req = { body: { ...validBody, currency: '' } }; - await routeHandlers['post:/'](req, mockResponse()); + await post({ ...validBody, currency: '' }).expect(201); expect(plannedTransactionRepository.create).toHaveBeenCalledWith( expect.objectContaining({ currency: undefined }), ); }); it('drops truthy recurrence bounds on a loan instead of validating them', async () => { - const req = { - body: { - bank_account: 'Mortgage', is_loan: true, loan_type: 'amortizing', - loan_principal: 10000, loan_annual_interest_rate: 6, loan_term_months: 12, - loan_start_date: '2026-04-01', loan_payment_day: 1, - // Garbage that would 400 on a non-loan — the loan branch deletes it. - max_occurrences: 'abc', recurrence_end_date: 'banana', - frequency: 'x', custom_interval_days: 3, end_date: 'y', - }, - }; - await routeHandlers['post:/'](req, mockResponse()); + await post({ + bank_account: 'Mortgage', is_loan: true, loan_type: 'amortizing', + loan_principal: 10000, loan_annual_interest_rate: 6, loan_term_months: 12, + loan_start_date: '2026-04-01', loan_payment_day: 1, + // Garbage that would 400 on a non-loan — the loan branch deletes it. + max_occurrences: 'abc', recurrence_end_date: 'banana', + frequency: 'x', custom_interval_days: 3, end_date: 'y', + }).expect(201); + const arg = plannedTransactionRepository.create.mock.calls[0][0]; expect('max_occurrences' in arg).toBe(false); expect('recurrence_end_date' in arg).toBe(false); @@ -429,15 +346,12 @@ describe('Planned Transaction Routes', () => { }); it('still rejects a falsy-but-present max_occurrences on a loan (not dropped)', async () => { - const req = { - body: { - bank_account: 'Mortgage', is_loan: true, loan_type: 'amortizing', - loan_principal: 10000, loan_annual_interest_rate: 6, loan_term_months: 12, - loan_start_date: '2026-04-01', loan_payment_day: 1, - max_occurrences: 0, - }, - }; - await expect(routeHandlers['post:/'](req, mockResponse())).rejects.toBeInstanceOf(ValidationError); + await post({ + bank_account: 'Mortgage', is_loan: true, loan_type: 'amortizing', + loan_principal: 10000, loan_annual_interest_rate: 6, loan_term_months: 12, + loan_start_date: '2026-04-01', loan_payment_day: 1, + max_occurrences: 0, + }).expect(400); expect(plannedTransactionRepository.create).not.toHaveBeenCalled(); }); }); @@ -449,8 +363,7 @@ describe('Planned Transaction Routes', () => { }); it('coerces reminder_days_before and max_occurrences on update', async () => { - const req = { params: { id: '1' }, body: { reminder_days_before: '7', max_occurrences: '4' } }; - await routeHandlers['patch:/:id'](req, mockResponse()); + await patch(1, { reminder_days_before: '7', max_occurrences: '4' }).expect(200); expect(plannedTransactionRepository.update).toHaveBeenCalledWith( 1, expect.objectContaining({ reminder_days_before: 7, max_occurrences: 4 }), @@ -465,8 +378,7 @@ describe('Planned Transaction Routes', () => { { tags: 'nope' }, { recurrence_pattern: 'fortnightly' }, ]) { - await expect(routeHandlers['patch:/:id']({ params: { id: '1' }, body }, mockResponse())) - .rejects.toBeInstanceOf(ValidationError); + await patch(1, body).expect(400); } expect(plannedTransactionRepository.update).not.toHaveBeenCalled(); }); @@ -475,12 +387,11 @@ describe('Planned Transaction Routes', () => { // PATCH forwarded the raw value to the SET builder, so "euro" hit the // 0046 ISO CHECK as a raw 500 and null hit the NOT NULL constraint. for (const body of [{ currency: 'euro' }, { currency: null }, { currency: '' }]) { - await expect(routeHandlers['patch:/:id']({ params: { id: '1' }, body }, mockResponse())) - .rejects.toBeInstanceOf(ValidationError); + await patch(1, body).expect(400); } expect(plannedTransactionRepository.update).not.toHaveBeenCalled(); - await routeHandlers['patch:/:id']({ params: { id: '1' }, body: { currency: 'usd' } }, mockResponse()); + await patch(1, { currency: 'usd' }).expect(200); expect(plannedTransactionRepository.update).toHaveBeenCalledWith( 1, expect.objectContaining({ currency: 'USD' }), @@ -495,12 +406,11 @@ describe('Planned Transaction Routes', () => { { amount: null }, { amount: '' }, ]) { - await expect(routeHandlers['patch:/:id']({ params: { id: '1' }, body }, mockResponse())) - .rejects.toBeInstanceOf(ValidationError); + await patch(1, body).expect(400); } expect(plannedTransactionRepository.update).not.toHaveBeenCalled(); - await routeHandlers['patch:/:id']({ params: { id: '1' }, body: { amount: '-42.50' } }, mockResponse()); + await patch(1, { amount: '-42.50' }).expect(200); expect(plannedTransactionRepository.update).toHaveBeenCalledWith( 1, expect.objectContaining({ amount: -42.5 }), @@ -511,17 +421,15 @@ describe('Planned Transaction Routes', () => { // is_recurring:true on a row without a pattern used to store and leave // the row perpetually due after /execute (the POST guard has an exact // sibling); clearing the pattern on a recurring row recreated it. - await expect(routeHandlers['patch:/:id']({ params: { id: '1' }, body: { is_recurring: true } }, mockResponse())) - .rejects.toBeInstanceOf(ValidationError); + await patch(1, { is_recurring: true }).expect(400); plannedTransactionRepository.getById.mockResolvedValue({ id: 1, is_loan: false, is_recurring: true, recurrence_pattern: 'monthly' }); - await expect(routeHandlers['patch:/:id']({ params: { id: '1' }, body: { recurrence_pattern: null } }, mockResponse())) - .rejects.toBeInstanceOf(ValidationError); + await patch(1, { recurrence_pattern: null }).expect(400); expect(plannedTransactionRepository.update).not.toHaveBeenCalled(); // Turning recurrence on WITH a valid pattern still passes. plannedTransactionRepository.getById.mockResolvedValue({ id: 1, is_loan: false }); - await routeHandlers['patch:/:id']({ params: { id: '1' }, body: { is_recurring: true, recurrence_pattern: 'monthly' } }, mockResponse()); + await patch(1, { is_recurring: true, recurrence_pattern: 'monthly' }).expect(200); expect(plannedTransactionRepository.update).toHaveBeenCalledWith( 1, expect.objectContaining({ is_recurring: true, recurrence_pattern: 'monthly' }), @@ -529,7 +437,7 @@ describe('Planned Transaction Routes', () => { // An unrelated edit to a legacy broken row (recurring, no pattern) is NOT blocked. plannedTransactionRepository.getById.mockResolvedValue({ id: 1, is_loan: false, is_recurring: true, recurrence_pattern: null }); - await routeHandlers['patch:/:id']({ params: { id: '1' }, body: { memo: 'still editable' } }, mockResponse()); + await patch(1, { memo: 'still editable' }).expect(200); expect(plannedTransactionRepository.update).toHaveBeenCalledWith( 1, expect.objectContaining({ memo: 'still editable' }), @@ -537,11 +445,9 @@ describe('Planned Transaction Routes', () => { }); it('passes explicit nulls through to clear recurrence bounds and pattern', async () => { - const req = { - params: { id: '1' }, - body: { recurrence_end_date: null, max_occurrences: null, recurrence_pattern: null }, - }; - await routeHandlers['patch:/:id'](req, mockResponse()); + await patch(1, { + recurrence_end_date: null, max_occurrences: null, recurrence_pattern: null, + }).expect(200); expect(plannedTransactionRepository.update).toHaveBeenCalledWith( 1, expect.objectContaining({ @@ -559,19 +465,26 @@ describe('Planned Transaction Routes', () => { id: 1, planned_date: '2026-03-15', amount: '50.00', }); - const req = { params: { id: '1' } }; - const res = mockResponse(); - await routeHandlers['get:/:id'](req, res); + const res = await api.get(`${BASE}/1`).expect(200); - expect(res.json).toHaveBeenCalled(); + expect(res.body.data.id).toBe(1); }); - it('should throw NotFoundError for non-existent', async () => { + it('should return a 404 envelope for non-existent', async () => { plannedTransactionRepository.getById.mockResolvedValue(null); - const req = { params: { id: '99999' } }; - const res = mockResponse(); - await expect(routeHandlers['get:/:id'](req, res)).rejects.toBeInstanceOf(NotFoundError); + const res = await api.get(`${BASE}/99999`).expect(404); + + expect(res.body.error.code).toBe('NOT_FOUND'); + }); + + it('rejects a non-integer :id via the real validateIdParam guard', async () => { + // Previously `vi.mock('.../middleware/validation.js')` replaced + // validateIdParam with a pass-through, so this guard was never tested. + const res = await api.get(`${BASE}/abc`).expect(400); + + expect(res.body.error.code).toBe('VALIDATION_ERROR'); + expect(plannedTransactionRepository.getById).not.toHaveBeenCalled(); }); }); @@ -580,19 +493,15 @@ describe('Planned Transaction Routes', () => { plannedTransactionRepository.getById.mockResolvedValue({ id: 1 }); plannedTransactionRepository.update.mockResolvedValue({ id: 1, amount: '75.00' }); - const req = { params: { id: '1' }, body: { amount: 75 } }; - const res = mockResponse(); - await routeHandlers['patch:/:id'](req, res); + const res = await patch(1, { amount: 75 }).expect(200); - expect(res.json).toHaveBeenCalled(); + expect(res.body.ok).toBe(true); }); - it('should throw NotFoundError for non-existent', async () => { + it('should return 404 for non-existent', async () => { plannedTransactionRepository.getById.mockResolvedValue(null); - const req = { params: { id: '99999' }, body: { amount: 75 } }; - const res = mockResponse(); - await expect(routeHandlers['patch:/:id'](req, res)).rejects.toBeInstanceOf(NotFoundError); + await patch(99999, { amount: 75 }).expect(404); }); it('should resolve recipient_name and category_name to IDs', async () => { @@ -606,15 +515,10 @@ describe('Planned Transaction Routes', () => { .mockResolvedValueOnce({ rows: [{ id: 11 }] }) .mockResolvedValueOnce({ rows: [{ id: 22 }] }); - const req = { - params: { id: '1' }, - body: { - recipient_name: 'John', - category_name: 'FOOD:GROCERIES', - }, - }; - const res = mockResponse(); - await routeHandlers['patch:/:id'](req, res); + await patch(1, { + recipient_name: 'John', + category_name: 'FOOD:GROCERIES', + }).expect(200); expect(plannedTransactionRepository.update).toHaveBeenCalledWith( 1, @@ -623,7 +527,6 @@ describe('Planned Transaction Routes', () => { category_id: 22, }) ); - expect(res.json).toHaveBeenCalled(); }); // Sign/amount derivation pins: whenever the repayment schedule is @@ -647,17 +550,13 @@ describe('Planned Transaction Routes', () => { plannedTransactionRepository.getById.mockResolvedValueOnce(existingLoan); plannedTransactionRepository.updateWithLoanSchedule.mockResolvedValue({ id: 1, is_loan: true }); - const req = { - params: { id: '1' }, - // Client edits the principal but sends its stale (pre-regeneration) amount. - body: { - loan_principal: 10000, - loan_annual_interest_rate: 6, - loan_term_months: 12, - amount: -214.03, - }, - }; - await routeHandlers['patch:/:id'](req, mockResponse()); + // Client edits the principal but sends its stale (pre-regeneration) amount. + await patch(1, { + loan_principal: 10000, + loan_annual_interest_rate: 6, + loan_term_months: 12, + amount: -214.03, + }).expect(200); expect(plannedTransactionRepository.updateWithLoanSchedule).toHaveBeenCalledWith( 1, @@ -673,20 +572,16 @@ describe('Planned Transaction Routes', () => { plannedTransactionRepository.getById.mockResolvedValueOnce({ id: 1, is_loan: false }); plannedTransactionRepository.updateWithLoanSchedule.mockResolvedValue({ id: 1, is_loan: true }); - const req = { - params: { id: '1' }, - body: { - is_loan: true, - loan_type: 'amortizing', - loan_principal: 10000, - loan_annual_interest_rate: 6, - loan_term_months: 12, - loan_start_date: '2026-04-01', - loan_payment_day: 1, - amount: 500, // stale client value, wrong sign — must be ignored - }, - }; - await routeHandlers['patch:/:id'](req, mockResponse()); + await patch(1, { + is_loan: true, + loan_type: 'amortizing', + loan_principal: 10000, + loan_annual_interest_rate: 6, + loan_term_months: 12, + loan_start_date: '2026-04-01', + loan_payment_day: 1, + amount: 500, // stale client value, wrong sign — must be ignored + }).expect(200); expect(plannedTransactionRepository.updateWithLoanSchedule).toHaveBeenCalledWith( 1, @@ -702,8 +597,7 @@ describe('Planned Transaction Routes', () => { plannedTransactionRepository.getById.mockResolvedValueOnce(existingLoan); plannedTransactionRepository.update.mockResolvedValue({ id: 1, is_loan: true }); - const req = { params: { id: '1' }, body: { memo: 'note', amount: -123.45 } }; - await routeHandlers['patch:/:id'](req, mockResponse()); + await patch(1, { memo: 'note', amount: -123.45 }).expect(200); expect(plannedTransactionRepository.updateWithLoanSchedule).not.toHaveBeenCalled(); expect(plannedTransactionRepository.update).toHaveBeenCalledWith( @@ -725,12 +619,7 @@ describe('Planned Transaction Routes', () => { }); plannedTransactionRepository.updateWithLoanSchedule.mockResolvedValue({ id: 1, is_loan: false, loan_schedule: [] }); - const req = { - params: { id: '1' }, - body: { is_loan: false }, - }; - const res = mockResponse(); - await routeHandlers['patch:/:id'](req, res); + await patch(1, { is_loan: false }).expect(200); // Field update + schedule clear must go through ONE atomic method ([] clears // the schedule) — not a separate update() + replaceLoanSchedule() pair. @@ -750,7 +639,6 @@ describe('Planned Transaction Routes', () => { [], ); expect(plannedTransactionRepository.update).not.toHaveBeenCalled(); - expect(res.json).toHaveBeenCalled(); }); }); @@ -761,14 +649,10 @@ describe('Planned Transaction Routes', () => { .mockResolvedValueOnce({ id: 1, is_executed: true, last_executed_date: '2026-03-15' }); plannedTransactionRepository.executeAndAdvance.mockResolvedValue({ duplicate: false }); - const req = { - params: { id: '1' }, - body: { executed_transaction_id: 10, execution_date: '2026-03-15' }, - }; - const res = mockResponse(); - await routeHandlers['post:/:id/execute'](req, res); + const res = await execute(1, { executed_transaction_id: 10, execution_date: '2026-03-15' }) + .expect(200); - expect(res.json).toHaveBeenCalled(); + expect(res.body.ok).toBe(true); }); it('should execute recurring and advance date', async () => { @@ -780,9 +664,7 @@ describe('Planned Transaction Routes', () => { .mockResolvedValueOnce({ id: 1, is_executed: false, planned_date: '2026-04-15' }); plannedTransactionRepository.executeAndAdvance.mockResolvedValue({ duplicate: false }); - const req = { params: { id: '1' }, body: { executed_transaction_id: 10 } }; - const res = mockResponse(); - await routeHandlers['post:/:id/execute'](req, res); + await execute(1, { executed_transaction_id: 10 }).expect(200); const call = plannedTransactionRepository.executeAndAdvance.mock.calls[0]; expect(call[3].is_executed).toBe(false); @@ -799,8 +681,7 @@ describe('Planned Transaction Routes', () => { .mockResolvedValueOnce({ id: 1 }); plannedTransactionRepository.executeAndAdvance.mockResolvedValue({ duplicate: false }); - const req = { params: { id: '1' }, body: { executed_transaction_id: 10 } }; - await routeHandlers['post:/:id/execute'](req, mockResponse()); + await execute(1, { executed_transaction_id: 10 }).expect(200); const updateFields = plannedTransactionRepository.executeAndAdvance.mock.calls[0][3]; expect(updateFields.planned_date).toBe('2026-02-28'); // not 2026-02-27 (the UTC day) @@ -815,25 +696,22 @@ describe('Planned Transaction Routes', () => { .mockResolvedValueOnce({ id: 1 }); plannedTransactionRepository.executeAndAdvance.mockResolvedValue({ duplicate: false }); - const req = { params: { id: '1' }, body: { executed_transaction_id: 11 } }; - await routeHandlers['post:/:id/execute'](req, mockResponse()); + await execute(1, { executed_transaction_id: 11 }).expect(200); const updateFields = plannedTransactionRepository.executeAndAdvance.mock.calls[0][3]; expect(updateFields.planned_date).toBe('2026-03-28'); }); - it('should throw ValidationError without executed_transaction_id', async () => { - const req = { params: { id: '1' }, body: {} }; - const res = mockResponse(); - await expect(routeHandlers['post:/:id/execute'](req, res)).rejects.toBeInstanceOf(ValidationError); + it('should return 400 without executed_transaction_id', async () => { + const res = await execute(1, {}).expect(400); + expect(res.body.error.code).toBe('VALIDATION_ERROR'); }); - it('should throw NotFoundError for non-existent', async () => { + it('should return 404 for non-existent', async () => { plannedTransactionRepository.getById.mockResolvedValue(null); - const req = { params: { id: '99999' }, body: { executed_transaction_id: 10 } }; - const res = mockResponse(); - await expect(routeHandlers['post:/:id/execute'](req, res)).rejects.toBeInstanceOf(NotFoundError); + const res = await execute(99999, { executed_transaction_id: 10 }).expect(404); + expect(res.body.error.code).toBe('NOT_FOUND'); }); }); @@ -841,21 +719,16 @@ describe('Planned Transaction Routes', () => { it('should delete and return 204 with no body', async () => { plannedTransactionRepository.hardDelete.mockResolvedValue(true); - const req = { params: { id: '1' } }; - const res = mockResponse(); - await routeHandlers['delete:/:id'](req, res); + const res = await api.delete(`${BASE}/1`).expect(204); - expect(res.status).toHaveBeenCalledWith(204); - expect(res.send).toHaveBeenCalledWith(); - expect(res.json).not.toHaveBeenCalled(); + expect(res.text).toBe(''); }); - it('should throw NotFoundError for non-existent', async () => { + it('should return 404 for non-existent', async () => { plannedTransactionRepository.hardDelete.mockResolvedValue(false); - const req = { params: { id: '99999' } }; - const res = mockResponse(); - await expect(routeHandlers['delete:/:id'](req, res)).rejects.toBeInstanceOf(NotFoundError); + const res = await api.delete(`${BASE}/99999`).expect(404); + expect(res.body.error.code).toBe('NOT_FOUND'); }); }); }); diff --git a/apps/node-backend/tests/routes/portfolioImport.test.js b/apps/node-backend/tests/routes/portfolioImport.test.js index 214f4206..eddc34fa 100644 --- a/apps/node-backend/tests/routes/portfolioImport.test.js +++ b/apps/node-backend/tests/routes/portfolioImport.test.js @@ -4,18 +4,20 @@ * Focused on the batch/row id guards: ids are positive bigserial PKs, so * non-positive or trailing-garbage values must 400 before touching the DB * (previously "-1"/"0"/"12abc" slipped through as NaN-tolerant parseInt). + * + * Runs against the REAL router mounted on a throwaway Express app (see + * tests/helpers/routeApp.js), which also puts the router's own trailing error + * middleware (`router.use(csvUploadErrorTranslator)`, routes/ + * portfolioImportRoutes.js:494) on the tested path — the mock-router harness + * dropped it entirely. multer is still stubbed to a pass-through (no real + * multipart parsing); this suite doesn't exercise the upload routes, so no + * `req.file` injection is needed (see portfolioImportValidationPins.test.js + * for that, mirroring importValidationPins.test.js's pattern). */ import { describe, it, expect, vi, beforeEach } from 'vitest'; import { mockConnection } from '../helpers/repoMocks.js'; import { mockLogger } from '../helpers/mockLogger.js'; -import { createMockRouter, createMockResponse } from '../helpers/routeHarness.js'; - -const { router: mockRouter, handlers: routeHandlers } = createMockRouter(); - -vi.mock('express', () => ({ - default: { Router: () => mockRouter }, - Router: () => mockRouter, -})); +import { routeAgent, okEnvelope, errEnvelope } from '../helpers/routeApp.js'; vi.mock('multer', () => { const multer = vi.fn(() => ({ @@ -75,37 +77,31 @@ vi.mock('../../src/config/logger.js', () => ({ })); import { getBatch, getPreviewRows, listBatches } from '../../src/services/portfolioImportBatchService.js'; -import { ValidationError } from '../../src/middleware/errorHandler.js'; -await import('../../src/routes/portfolioImportRoutes.js'); + +const { default: portfolioImportRouter } = await import('../../src/routes/portfolioImportRoutes.js'); + +const BASE = '/api/portfolio/import'; +const api = routeAgent(portfolioImportRouter, { mountPath: BASE }); describe('Portfolio Import Routes — batch/row id guards', () => { beforeEach(() => vi.clearAllMocks()); it('GET /batches/:id/preview rejects a negative batch id with a 400', async () => { - const req = { params: { id: '-1' } }; - const res = createMockResponse(); - - await expect(routeHandlers['get:/batches/:id/preview'](req, res)) - .rejects.toBeInstanceOf(ValidationError); + const res = await api.get(`${BASE}/batches/-1/preview`).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); expect(getBatch).not.toHaveBeenCalled(); expect(getPreviewRows).not.toHaveBeenCalled(); }); it('GET /batches/:id rejects a trailing-garbage id like "12abc"', async () => { - const req = { params: { id: '12abc' } }; - const res = createMockResponse(); - - await expect(routeHandlers['get:/batches/:id'](req, res)) - .rejects.toBeInstanceOf(ValidationError); + const res = await api.get(`${BASE}/batches/12abc`).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); expect(getBatch).not.toHaveBeenCalled(); }); it('POST /batches/:id/rows/:rowId/investment-override rejects a zero rowId', async () => { - const req = { params: { id: '5', rowId: '0' }, body: {} }; - const res = createMockResponse(); - - await expect(routeHandlers['post:/batches/:id/rows/:rowId/investment-override'](req, res)) - .rejects.toBeInstanceOf(ValidationError); + const res = await api.post(`${BASE}/batches/5/rows/0/investment-override`).send({}).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); }); }); @@ -114,19 +110,16 @@ describe('Portfolio Import Routes — collection response shape', () => { it('GET /batches returns the canonical { items, total, limit, offset } body', async () => { listBatches.mockResolvedValue({ batches: [{ id: 1, status: 'complete' }], total: 1 }); - const res = createMockResponse(); - await routeHandlers['get:/batches']({ query: {} }, res); + const res = await api.get(`${BASE}/batches`).expect(200); - const body = res.json.mock.calls[0][0]; - expect(body.ok).toBe(true); // The old `batches` wire key is gone; the service still speaks `batches`. - expect(body.data.batches).toBeUndefined(); - expect(body.data).toEqual({ + expect(res.body.data.batches).toBeUndefined(); + expect(res.body).toEqual(okEnvelope({ items: [{ id: 1, status: 'complete' }], total: 1, limit: 50, offset: 0, - }); + })); }); }); diff --git a/apps/node-backend/tests/routes/portfolioImportValidationPins.test.js b/apps/node-backend/tests/routes/portfolioImportValidationPins.test.js index df94a015..b60a066e 100644 --- a/apps/node-backend/tests/routes/portfolioImportValidationPins.test.js +++ b/apps/node-backend/tests/routes/portfolioImportValidationPins.test.js @@ -6,18 +6,17 @@ * parseBrokerageParams' multipart-string coercion, buildPortfolioConfig's * required/enum/trim/default build, and normalizePortfolioParserConfig's * pass-through semantics — so the swap cannot change the wire. + * + * Driven over HTTP against the real router (tests/helpers/routeApp.js), + * mirroring importValidationPins.test.js: multer is stubbed to a pass-through + * (no real multipart parsing) and the uploaded file is injected by a `before` + * middleware, the same per-mount slot main.js uses (main.js:326 mounts + * importRateLimiter there for this router). */ import { describe, it, expect, vi, beforeEach } from 'vitest'; import { mockConnection } from '../helpers/repoMocks.js'; import { mockLogger } from '../helpers/mockLogger.js'; -import { createMockRouter, createMockResponse } from '../helpers/routeHarness.js'; - -const { router: mockRouter, handlers: routeHandlers } = createMockRouter(); - -vi.mock('express', () => ({ - default: { Router: () => mockRouter }, - Router: () => mockRouter, -})); +import { routeAgent } from '../helpers/routeApp.js'; vi.mock('multer', () => { const multer = vi.fn(() => ({ @@ -76,19 +75,32 @@ vi.mock('../../src/config/logger.js', () => ({ logger: mockLogger(), })); -import { runPortfolioImportPipeline } from '../../src/services/portfolioImportPipeline/index.js'; +import { runPortfolioImportPipeline, commitPortfolioImport } from '../../src/services/portfolioImportPipeline/index.js'; +// NOT mocked: only .../portfolioImportPipeline/index.js is. This is the real +// boundary function, run here over the mocked pg connection. +import { createBatch } from '../../src/services/portfolioImportPipeline/stage.js'; +import { query as dbQuery } from '../../src/database/connection.js'; import { getBatch, overrideInvestment } from '../../src/services/portfolioImportBatchService.js'; import customParserConfigRepository from '../../src/repositories/customParserConfigRepository.js'; -import { ValidationError } from '../../src/middleware/errorHandler.js'; -await import('../../src/routes/portfolioImportRoutes.js'); -const mockResponse = () => createMockResponse(); -const file = () => ({ path: '/tmp/pin.csv', originalname: 'pin.csv', size: 10 }); +const { default: portfolioImportRouter } = await import('../../src/routes/portfolioImportRoutes.js'); + +const UPLOAD = { path: '/tmp/pin.csv', originalname: 'pin.csv', size: 10 }; +const BASE = '/api/portfolio/import'; +// multer is stubbed, so nothing populates req.file — inject it in the same +// per-mount slot main.js uses. +const api = routeAgent(portfolioImportRouter, { + mountPath: BASE, + before: [(req, _res, next) => { req.file = { ...UPLOAD }; next(); }], +}); + +/** Encode a path segment so ids with spaces survive the URL round-trip. */ +const seg = (v) => encodeURIComponent(String(v)); const minimalQuery = { date_column: 'D', name_column: 'N', default_asset_class: 'etf' }; const runCustom = (query, body = {}) => - routeHandlers['post:/csv/custom']({ file: file(), query, body }, mockResponse()); + api.post(`${BASE}/csv/custom`).query(query).send(body); beforeEach(() => { vi.clearAllMocks(); @@ -101,51 +113,55 @@ beforeEach(() => { describe('batch-id coercion pins', () => { it("accepts '12.0' / ' 12 ' via Number() coercion", async () => { getBatch.mockResolvedValue({ id: 12, status: 'complete' }); - await routeHandlers['get:/batches/:id']({ params: { id: '12.0' } }, mockResponse()); + + await api.get(`${BASE}/batches/${seg('12.0')}`).expect(200); expect(getBatch).toHaveBeenLastCalledWith(12); - await routeHandlers['get:/batches/:id']({ params: { id: ' 12 ' } }, mockResponse()); + + await api.get(`${BASE}/batches/${seg(' 12 ')}`).expect(200); expect(getBatch).toHaveBeenLastCalledWith(12); }); it('rejects fractional/garbage ids on the commit and override sites', async () => { - await expect(routeHandlers['post:/batches/:id/commit']({ params: { id: '2.5' }, body: {} }, mockResponse())) - .rejects.toBeInstanceOf(ValidationError); - await expect(routeHandlers['post:/batches/:id/rows/:rowId/investment-override']( - { params: { id: '3.5', rowId: '1' }, body: {} }, mockResponse(), - )).rejects.toBeInstanceOf(ValidationError); + const res1 = await api.post(`${BASE}/batches/${seg('2.5')}/commit`).send({}).expect(400); + expect(res1.body.error.code).toBe('VALIDATION_ERROR'); + + const res2 = await api.post(`${BASE}/batches/${seg('3.5')}/rows/1/investment-override`).send({}).expect(400); + expect(res2.body.error.code).toBe('VALIDATION_ERROR'); + expect(getBatch).not.toHaveBeenCalled(); }); it('coerces both ids on the investment-override site', async () => { overrideInvestment.mockResolvedValue(1); - await routeHandlers['post:/batches/:id/rows/:rowId/investment-override']( - { params: { id: '5.0', rowId: ' 6 ' }, body: { investment_id: null } }, - mockResponse(), - ); + + await api.post(`${BASE}/batches/${seg('5.0')}/rows/${seg(' 6 ')}/investment-override`) + .send({ investment_id: null }) + .expect(200); + expect(overrideInvestment).toHaveBeenCalledWith({ batchId: 5, rowId: 6, investmentId: null }); }); }); describe('parseBrokerageParams pins (POST /csv/custom)', () => { it("coerces multipart strings: is_brokerage 'true'/'false', account_id '7'", async () => { - await runCustom({ ...minimalQuery, is_brokerage: 'true', account_id: '7' }); + await runCustom({ ...minimalQuery, is_brokerage: 'true', account_id: '7' }).expect(201); expect(runPortfolioImportPipeline).toHaveBeenLastCalledWith( expect.objectContaining({ isBrokerage: true, accountId: 7 }), ); - await runCustom({ ...minimalQuery, is_brokerage: 'false', account_id: '7' }); + await runCustom({ ...minimalQuery, is_brokerage: 'false', account_id: '7' }).expect(201); expect(runPortfolioImportPipeline).toHaveBeenLastCalledWith( expect.objectContaining({ isBrokerage: false, accountId: 7 }), ); - await runCustom({ ...minimalQuery, is_brokerage: true, account_id: 7 }); + await runCustom({ ...minimalQuery, is_brokerage: true, account_id: 7 }).expect(201); expect(runPortfolioImportPipeline).toHaveBeenLastCalledWith( expect.objectContaining({ isBrokerage: true, accountId: 7 }), ); }); it('treats an empty account_id as absent and defaults is_brokerage to false', async () => { - await runCustom({ ...minimalQuery, account_id: '' }); + await runCustom({ ...minimalQuery, account_id: '' }).expect(201); expect(runPortfolioImportPipeline).toHaveBeenLastCalledWith( expect.objectContaining({ isBrokerage: false, accountId: undefined }), ); @@ -153,11 +169,11 @@ describe('parseBrokerageParams pins (POST /csv/custom)', () => { it('rejects non-integer account ids and a brokerage import without an account', async () => { for (const account_id of ['abc', '7.5', '-1', '0']) { - await expect(runCustom({ ...minimalQuery, account_id })) - .rejects.toThrow('account_id must be a positive integer'); + const res = await runCustom({ ...minimalQuery, account_id }).expect(400); + expect(res.body.error.message).toContain('account_id must be a positive integer'); } - await expect(runCustom({ ...minimalQuery, is_brokerage: 'true' })) - .rejects.toThrow(/brokerage import requires account_id/); + const res = await runCustom({ ...minimalQuery, is_brokerage: 'true' }).expect(400); + expect(res.body.error.message).toMatch(/brokerage import requires account_id/); expect(runPortfolioImportPipeline).not.toHaveBeenCalled(); }); }); @@ -169,7 +185,7 @@ describe('buildPortfolioConfig pins (POST /csv/custom)', () => { default_asset_class: 'stock', default_type: 'sell', date_format: ' %d-%m-%Y ', separator: ';', encoding: ' latin1 ', skip_rows: '2.9', type_mapping: '{"K":"buy"}', adapter_name: ' custom ', - }); + }).expect(201); expect(runPortfolioImportPipeline).toHaveBeenCalledWith(expect.objectContaining({ adapterName: 'custom', defaultAssetClass: 'stock', @@ -191,7 +207,7 @@ describe('buildPortfolioConfig pins (POST /csv/custom)', () => { }); it('applies defaults for a minimal config', async () => { - await runCustom(minimalQuery); + await runCustom(minimalQuery).expect(201); expect(runPortfolioImportPipeline).toHaveBeenCalledWith(expect.objectContaining({ adapterName: 'portfolio_generic', defaultType: 'buy', @@ -203,58 +219,121 @@ describe('buildPortfolioConfig pins (POST /csv/custom)', () => { }); it('empty separator falls back to ","; malformed type_mapping falls back to {}', async () => { - await runCustom({ ...minimalQuery, separator: '', type_mapping: 'not-json' }); + await runCustom({ ...minimalQuery, separator: '', type_mapping: 'not-json' }).expect(201); expect(runPortfolioImportPipeline).toHaveBeenCalledWith(expect.objectContaining({ customConfig: expect.objectContaining({ separator: ',', type_mapping: {} }), })); }); it('rejects each invalid field with its message', async () => { - await expect(runCustom({ name_column: 'N', default_asset_class: 'etf' })) - .rejects.toThrow('date_column is required'); - await expect(runCustom({ date_column: 'D', default_asset_class: 'etf' })) - .rejects.toThrow('map at least one of symbol_column or name_column'); - await expect(runCustom({ date_column: 'D', name_column: 'N' })) - .rejects.toThrow('default_asset_class is required and must be a valid asset class'); - await expect(runCustom({ ...minimalQuery, default_asset_class: 'house' })) - .rejects.toThrow('default_asset_class is required and must be a valid asset class'); - await expect(runCustom({ ...minimalQuery, default_type: 'yolo' })) - .rejects.toThrow('default_type "yolo" is not a valid transaction type'); - await expect(runCustom({ ...minimalQuery, separator: ';;' })) - .rejects.toThrow('separator must be a single character'); - await expect(runCustom({ ...minimalQuery, skip_rows: '-1' })) - .rejects.toThrow('skip_rows must be zero or a positive integer'); + let res = await runCustom({ name_column: 'N', default_asset_class: 'etf' }).expect(400); + expect(res.body.error.message).toContain('date_column is required'); + + res = await runCustom({ date_column: 'D', default_asset_class: 'etf' }).expect(400); + expect(res.body.error.message).toContain('map at least one of symbol_column or name_column'); + + res = await runCustom({ date_column: 'D', name_column: 'N' }).expect(400); + expect(res.body.error.message).toContain('default_asset_class is required and must be a valid asset class'); + + res = await runCustom({ ...minimalQuery, default_asset_class: 'house' }).expect(400); + expect(res.body.error.message).toContain('default_asset_class is required and must be a valid asset class'); + + res = await runCustom({ ...minimalQuery, default_type: 'yolo' }).expect(400); + expect(res.body.error.message).toContain('default_type "yolo" is not a valid transaction type'); + + res = await runCustom({ ...minimalQuery, separator: ';;' }).expect(400); + expect(res.body.error.message).toContain('separator must be a single character'); + + res = await runCustom({ ...minimalQuery, skip_rows: '-1' }).expect(400); + expect(res.body.error.message).toContain('skip_rows must be zero or a positive integer'); + expect(runPortfolioImportPipeline).not.toHaveBeenCalled(); }); }); describe('normalizePortfolioParserConfig pins (POST /parsers)', () => { - const create = (config) => - routeHandlers['post:/parsers']({ body: { name: 'P', config } }, mockResponse()); + const create = (config) => api.post(`${BASE}/parsers`).send({ name: 'P', config }); it('passes a valid config through UNCHANGED, unknown keys and all', async () => { const config = { dateColumn: ' Date ', symbolColumn: 'Sym', defaultAssetClass: 'stock', memoColumn: 42, separator: ';;', custom_extra: { nested: true }, }; - await create(config); + await create(config).expect(201); expect(customParserConfigRepository.create).toHaveBeenCalledWith( expect.objectContaining({ config, kind: 'portfolio' }), ); }); it('rejects missing dateColumn / symbol-or-name / bad asset class', async () => { - await expect(create({ symbolColumn: 'S', defaultAssetClass: 'stock' })) - .rejects.toThrow('config.dateColumn is required'); - await expect(create({ dateColumn: 'D', defaultAssetClass: 'stock' })) - .rejects.toThrow('config requires symbolColumn or nameColumn'); - await expect(create({ dateColumn: 'D', nameColumn: 'N', defaultAssetClass: 'house' })) - .rejects.toThrow('config.defaultAssetClass must be a valid asset class'); + let res = await create({ symbolColumn: 'S', defaultAssetClass: 'stock' }).expect(400); + expect(res.body.error.message).toContain('config.dateColumn is required'); + + res = await create({ dateColumn: 'D', defaultAssetClass: 'stock' }).expect(400); + expect(res.body.error.message).toContain('config requires symbolColumn or nameColumn'); + + res = await create({ dateColumn: 'D', nameColumn: 'N', defaultAssetClass: 'house' }).expect(400); + expect(res.body.error.message).toContain('config.defaultAssetClass must be a valid asset class'); }); it('rejects a non-object or array config', async () => { for (const config of [null, 'str', [1]]) { - await expect(create(config)).rejects.toThrow('Missing or invalid "config"'); + const res = await create(config).expect(400); + expect(res.body.error.message).toContain('Missing or invalid "config"'); } }); }); + +/** + * `batch_id` wire type — the portfolio half of the same split. + * + * `POST /csv/custom` relays `result.batchId` from `createBatch` + * (services/portfolioImportPipeline/stage.js), while `POST /batches/:id/commit` + * re-reads the id off the URL through `coercedIdSchema`. `createBatch` used to + * hand back node-postgres's BIGSERIAL STRING, so the two responses typed the + * same JSON field differently. NUMBER is now the single wire type (normalized at + * the stage boundary; pinned by tests/importPipeline.stage.test.js). + */ +describe('batch_id wire type', () => { + const BATCH_ID = 12; + + // The pipeline is mocked, but its `batchId` comes from the REAL `createBatch` + // running over the mocked pg connection primed with what node-postgres + // actually returns for a BIGSERIAL: the STRING '12'. Before the stage-boundary + // fix these pins failed with `batch_id: "12"`. + const realBatchId = async () => { + dbQuery.mockResolvedValueOnce({ rows: [{ id: String(BATCH_ID) }] }); + return createBatch({ adapterName: 'generic' }); + }; + + it('POST /csv/custom emits a numeric batch_id on both the 201 and the 202', async () => { + runPortfolioImportPipeline.mockImplementation(async () => ({ + requiresReview: false, batchId: await realBatchId(), total: 1, skipped: 0, imported: 1, duplicates: 0, errors: 0, + })); + const committed = await runCustom(minimalQuery, {}).expect(201); + expect(typeof committed.body.data.batch_id).toBe('number'); + expect(committed.body.data.batch_id).toBe(BATCH_ID); + + runPortfolioImportPipeline.mockImplementation(async () => ({ + requiresReview: true, batchId: await realBatchId(), matchSourceCounts: { symbol: 1 }, + })); + const review = await runCustom(minimalQuery, {}).expect(202); + expect(typeof review.body.data.batch_id).toBe('number'); + expect(review.body.data.batch_id).toStrictEqual(committed.body.data.batch_id); + }); + + it('POST /batches/:id/commit emits the SAME type and value for the same batch', async () => { + runPortfolioImportPipeline.mockImplementation(async () => ({ + requiresReview: true, batchId: await realBatchId(), matchSourceCounts: {}, + })); + const started = await runCustom(minimalQuery, {}).expect(202); + + getBatch.mockResolvedValue({ id: BATCH_ID, status: 'awaiting_review' }); + commitPortfolioImport.mockResolvedValue({ imported: 1, duplicates: 0, errors: 0 }); + const committed = await api.post(`${BASE}/batches/${BATCH_ID}/commit`).send({}).expect(200); + + expect(typeof committed.body.data.batch_id).toBe('number'); + // The whole point of the finding: strict equality across the two responses. + expect(committed.body.data.batch_id).toStrictEqual(started.body.data.batch_id); + }); +}); diff --git a/apps/node-backend/tests/routes/recipientBankAccounts.test.js b/apps/node-backend/tests/routes/recipientBankAccounts.test.js index 71e61160..e596a5a7 100644 --- a/apps/node-backend/tests/routes/recipientBankAccounts.test.js +++ b/apps/node-backend/tests/routes/recipientBankAccounts.test.js @@ -1,18 +1,17 @@ /** * Recipient Bank Account route tests. * Mirrors: apps/backend/tests/test_recipient_bank_accounts.py + * + * Runs against the REAL router mounted on a throwaway Express app (see + * tests/helpers/routeApp.js) — validateIdParam is no longer stubbed. */ import { describe, it, expect, vi, beforeEach } from 'vitest'; import { mockLogger } from '../helpers/mockLogger.js'; -import { createMockRouter, createMockResponse } from '../helpers/routeHarness.js'; - -const { router: mockRouter, handlers: routeHandlers } = createMockRouter(); - -vi.mock('express', () => ({ - default: { Router: () => mockRouter }, - Router: () => mockRouter, -})); +import { routeAgent, errEnvelope } from '../helpers/routeApp.js'; +// The route imports its repository through services/recipientBankAccountService.js, +// which re-exports the default from this module — mocking the repository here +// intercepts that same binding. vi.mock('../../src/repositories/recipientBankAccountRepository.js', () => ({ default: { getByRecipientId: vi.fn(), @@ -25,23 +24,16 @@ vi.mock('../../src/repositories/recipientBankAccountRepository.js', () => ({ }, })); -vi.mock('../../src/middleware/validation.js', async (importOriginal) => ({ - // Keep the real helpers (assertMaxLength, …); only stub the middleware. - ...(await importOriginal()), - validateIdParam: (req, res, next) => next(), -})); - vi.mock('../../src/config/logger.js', () => ({ logger: mockLogger(), })); import bankAccountRepo from '../../src/repositories/recipientBankAccountRepository.js'; -import { ValidationError, NotFoundError } from '../../src/middleware/errorHandler.js'; -await import('../../src/routes/recipientBankAccounts.js'); -function mockResponse() { - return createMockResponse(); -} +const { default: recipientBankAccountsRouter } = await import('../../src/routes/recipientBankAccounts.js'); + +const api = routeAgent(recipientBankAccountsRouter, { mountPath: '/api/recipients' }); +const BASE = '/api/recipients'; describe('Recipient Bank Account Routes', () => { beforeEach(() => vi.clearAllMocks()); @@ -52,23 +44,18 @@ describe('Recipient Bank Account Routes', () => { { id: 1, recipient_id: 1, account_number: 'BE61734041478017', bank_name: 'BELFIUS', is_primary: true }, ]); - const req = { params: { id: '1' }, query: {} }; - const res = mockResponse(); - await routeHandlers['get:/:id/bank-accounts'](req, res); + const res = await api.get(`${BASE}/1/bank-accounts`).expect(200); - const data = res.json.mock.calls[0][0]; - expect(data.data.items).toHaveLength(1); - expect(data.data.items[0].account_number).toBe('BE61734041478017'); + expect(res.body.data.items).toHaveLength(1); + expect(res.body.data.items[0].account_number).toBe('BE61734041478017'); }); it('should return empty list', async () => { bankAccountRepo.getByRecipientId.mockResolvedValue([]); - const req = { params: { id: '999' }, query: {} }; - const res = mockResponse(); - await routeHandlers['get:/:id/bank-accounts'](req, res); + const res = await api.get(`${BASE}/999/bank-accounts`).expect(200); - expect(res.json.mock.calls[0][0].data.items).toEqual([]); + expect(res.body.data.items).toEqual([]); }); }); @@ -79,14 +66,9 @@ describe('Recipient Bank Account Routes', () => { created: true, }); - const req = { - params: { id: '1' }, - body: { account_number: 'BE61734041478017', bank_name: 'Belfius' }, - }; - const res = mockResponse(); - await routeHandlers['post:/:id/bank-accounts'](req, res); - - expect(res.status).toHaveBeenCalledWith(201); + await api.post(`${BASE}/1/bank-accounts`) + .send({ account_number: 'BE61734041478017', bank_name: 'Belfius' }) + .expect(201); }); it('should return 200 for existing account', async () => { @@ -95,26 +77,21 @@ describe('Recipient Bank Account Routes', () => { created: false, }); - const req = { - params: { id: '1' }, - body: { account_number: 'BE61734041478017' }, - }; - const res = mockResponse(); - await routeHandlers['post:/:id/bank-accounts'](req, res); - - expect(res.status).toHaveBeenCalledWith(200); + await api.post(`${BASE}/1/bank-accounts`) + .send({ account_number: 'BE61734041478017' }) + .expect(200); }); - it('should throw ValidationError for missing account_number', async () => { - const req = { params: { id: '1' }, body: {} }; - const res = mockResponse(); - await expect(routeHandlers['post:/:id/bank-accounts'](req, res)).rejects.toBeInstanceOf(ValidationError); + it('should return a 400 VALIDATION_ERROR envelope for missing account_number', async () => { + const res = await api.post(`${BASE}/1/bank-accounts`).send({}).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); }); it('rejects an account_number longer than the VARCHAR(34) column (was a raw 22001 500)', async () => { - const req = { params: { id: '1' }, body: { account_number: 'X'.repeat(35) } }; - const res = mockResponse(); - await expect(routeHandlers['post:/:id/bank-accounts'](req, res)).rejects.toBeInstanceOf(ValidationError); + const res = await api.post(`${BASE}/1/bank-accounts`) + .send({ account_number: 'X'.repeat(35) }) + .expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); expect(bankAccountRepo.createOrGet).not.toHaveBeenCalled(); }); @@ -125,34 +102,38 @@ describe('Recipient Bank Account Routes', () => { created: true, }); - const req = { params: { id: '1' }, body: { account_number: acct } }; - const res = mockResponse(); - await routeHandlers['post:/:id/bank-accounts'](req, res); + await api.post(`${BASE}/1/bank-accounts`).send({ account_number: acct }).expect(201); - expect(res.status).toHaveBeenCalledWith(201); expect(bankAccountRepo.createOrGet).toHaveBeenCalledWith( expect.objectContaining({ accountNumber: acct }), ); }); + + it('rejects a non-integer :id via the real validateIdParam guard', async () => { + // Previously `vi.mock('.../middleware/validation.js')` replaced + // validateIdParam with a pass-through, so this guard was never tested. + const res = await api.post(`${BASE}/abc/bank-accounts`) + .send({ account_number: 'BE61734041478017' }) + .expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); + expect(bankAccountRepo.createOrGet).not.toHaveBeenCalled(); + }); }); describe('PATCH /:recipientId/bank-accounts/:accountId', () => { - it('should throw ValidationError for invalid account ID', async () => { - const req = { params: { id: '1', accountId: '0' }, body: {} }; - const res = mockResponse(); - await expect(routeHandlers['patch:/:id/bank-accounts/:accountId'](req, res)).rejects.toBeInstanceOf(ValidationError); + it('should return a 400 VALIDATION_ERROR envelope for invalid account ID', async () => { + const res = await api.patch(`${BASE}/1/bank-accounts/0`).send({}).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); expect(bankAccountRepo.update).not.toHaveBeenCalled(); }); - it('should throw NotFoundError when account does not exist', async () => { + it('should return a 404 NOT_FOUND envelope when account does not exist', async () => { bankAccountRepo.update.mockResolvedValue(null); - const req = { - params: { id: '1', accountId: '99' }, - body: { bank_name: 'Belfius' }, - }; - const res = mockResponse(); - await expect(routeHandlers['patch:/:id/bank-accounts/:accountId'](req, res)).rejects.toBeInstanceOf(NotFoundError); + const res = await api.patch(`${BASE}/1/bank-accounts/99`) + .send({ bank_name: 'Belfius' }) + .expect(404); + expect(res.body).toEqual(errEnvelope({ code: 'NOT_FOUND' })); expect(bankAccountRepo.update).toHaveBeenCalledWith(99, { bankName: 'Belfius', @@ -164,47 +145,40 @@ describe('Recipient Bank Account Routes', () => { it('should return updated account with links when update succeeds', async () => { bankAccountRepo.update.mockResolvedValue({ id: 5, bank_name: 'Updated Bank' }); - const req = { - params: { id: '1', accountId: '5' }, - body: { bank_name: 'Updated Bank', address: 'Main Street 1', account_label: 'Primary' }, - }; - const res = mockResponse(); - await routeHandlers['patch:/:id/bank-accounts/:accountId'](req, res); + const res = await api.patch(`${BASE}/1/bank-accounts/5`) + .send({ bank_name: 'Updated Bank', address: 'Main Street 1', account_label: 'Primary' }) + .expect(200); expect(bankAccountRepo.update).toHaveBeenCalledWith(5, { bankName: 'Updated Bank', address: 'Main Street 1', accountLabel: 'Primary', }); - expect(res.json).toHaveBeenCalledWith({ ok: true, data: { id: 5, bank_name: 'Updated Bank', links: [] } }); + expect(res.body.data).toEqual({ id: 5, bank_name: 'Updated Bank', links: [] }); }); - it('should propagate thrown error when update throws', async () => { + it('should answer a 500 when update throws', async () => { bankAccountRepo.update.mockRejectedValue(new Error('boom')); - const req = { - params: { id: '1', accountId: '5' }, - body: { bank_name: 'Updated Bank' }, - }; - const res = mockResponse(); - await expect(routeHandlers['patch:/:id/bank-accounts/:accountId'](req, res)).rejects.toThrow('boom'); + const res = await api.patch(`${BASE}/1/bank-accounts/5`) + .send({ bank_name: 'Updated Bank' }) + .expect(500); + expect(res.body.error.message).toBe('boom'); }); }); describe('DELETE /:recipientId/bank-accounts/:accountId', () => { - it('should throw ValidationError for invalid account ID', async () => { - const req = { params: { id: '1', accountId: '0' } }; - const res = mockResponse(); - await expect(routeHandlers['delete:/:id/bank-accounts/:accountId'](req, res)).rejects.toBeInstanceOf(ValidationError); + it('should return a 400 VALIDATION_ERROR envelope for invalid account ID', async () => { + const res = await api.delete(`${BASE}/1/bank-accounts/0`).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); expect(bankAccountRepo.softDelete).not.toHaveBeenCalled(); }); - it('should throw NotFoundError when delete target does not exist', async () => { + it('should return a 404 NOT_FOUND envelope when delete target does not exist', async () => { bankAccountRepo.softDelete.mockResolvedValue(false); - const req = { params: { id: '1', accountId: '15' } }; - const res = mockResponse(); - await expect(routeHandlers['delete:/:id/bank-accounts/:accountId'](req, res)).rejects.toBeInstanceOf(NotFoundError); + const res = await api.delete(`${BASE}/1/bank-accounts/15`).expect(404); + expect(res.body).toEqual(errEnvelope({ code: 'NOT_FOUND' })); }); // Soft delete, so 200 + the deactivated entity rather than 204 (see @@ -213,39 +187,31 @@ describe('Recipient Bank Account Routes', () => { bankAccountRepo.softDelete.mockResolvedValue(true); bankAccountRepo.getById.mockResolvedValue({ id: 15, account_number: 'BE01', is_active: false }); - const req = { params: { id: '1', accountId: '15' } }; - const res = mockResponse(); - await routeHandlers['delete:/:id/bank-accounts/:accountId'](req, res); + const res = await api.delete(`${BASE}/1/bank-accounts/15`).expect(200); - expect(res.json).toHaveBeenCalledWith({ - ok: true, - data: { id: 15, account_number: 'BE01', is_active: false, links: [] }, - }); + expect(res.body.data).toEqual({ id: 15, account_number: 'BE01', is_active: false, links: [] }); }); - it('should propagate thrown error when delete throws', async () => { + it('should answer a 500 when delete throws', async () => { bankAccountRepo.softDelete.mockRejectedValue(new Error('boom')); - const req = { params: { id: '1', accountId: '15' } }; - const res = mockResponse(); - await expect(routeHandlers['delete:/:id/bank-accounts/:accountId'](req, res)).rejects.toThrow('boom'); + const res = await api.delete(`${BASE}/1/bank-accounts/15`).expect(500); + expect(res.body.error.message).toBe('boom'); }); }); describe('POST /:recipientId/bank-accounts/:accountId/set-primary', () => { - it('should throw NotFoundError when setPrimary fails', async () => { + it('should return a 404 NOT_FOUND envelope when setPrimary fails', async () => { bankAccountRepo.setPrimary.mockResolvedValue(false); - const req = { params: { id: '1', accountId: '99' } }; - const res = mockResponse(); - await expect(routeHandlers['post:/:id/bank-accounts/:accountId/set-primary'](req, res)).rejects.toBeInstanceOf(NotFoundError); + const res = await api.post(`${BASE}/1/bank-accounts/99/set-primary`).expect(404); + expect(res.body).toEqual(errEnvelope({ code: 'NOT_FOUND' })); expect(bankAccountRepo.setPrimary).toHaveBeenCalledWith(99, 1); }); - it('should throw ValidationError when account ID is invalid', async () => { - const req = { params: { id: '1', accountId: 'invalid' } }; - const res = mockResponse(); - await expect(routeHandlers['post:/:id/bank-accounts/:accountId/set-primary'](req, res)).rejects.toBeInstanceOf(ValidationError); + it('should return a 400 VALIDATION_ERROR envelope when account ID is invalid', async () => { + const res = await api.post(`${BASE}/1/bank-accounts/invalid/set-primary`).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); expect(bankAccountRepo.setPrimary).not.toHaveBeenCalled(); }); @@ -253,21 +219,18 @@ describe('Recipient Bank Account Routes', () => { bankAccountRepo.setPrimary.mockResolvedValue(true); bankAccountRepo.getById.mockResolvedValue({ id: 2, recipient_id: 1, is_primary: true }); - const req = { params: { id: '1', accountId: '2' } }; - const res = mockResponse(); - await routeHandlers['post:/:id/bank-accounts/:accountId/set-primary'](req, res); + const res = await api.post(`${BASE}/1/bank-accounts/2/set-primary`).expect(200); expect(bankAccountRepo.setPrimary).toHaveBeenCalledWith(2, 1); expect(bankAccountRepo.getById).toHaveBeenCalledWith(2); - expect(res.json).toHaveBeenCalledWith({ ok: true, data: { id: 2, recipient_id: 1, is_primary: true, links: [] } }); + expect(res.body.data).toEqual({ id: 2, recipient_id: 1, is_primary: true, links: [] }); }); - it('should propagate thrown error when setPrimary throws', async () => { + it('should answer a 500 when setPrimary throws', async () => { bankAccountRepo.setPrimary.mockRejectedValue(new Error('boom')); - const req = { params: { id: '1', accountId: '2' } }; - const res = mockResponse(); - await expect(routeHandlers['post:/:id/bank-accounts/:accountId/set-primary'](req, res)).rejects.toThrow('boom'); + const res = await api.post(`${BASE}/1/bank-accounts/2/set-primary`).expect(500); + expect(res.body.error.message).toBe('boom'); }); }); }); diff --git a/apps/node-backend/tests/routes/recipients.test.js b/apps/node-backend/tests/routes/recipients.test.js index 5e80a5c3..d85e3e73 100644 --- a/apps/node-backend/tests/routes/recipients.test.js +++ b/apps/node-backend/tests/routes/recipients.test.js @@ -1,18 +1,17 @@ /** * Recipient route tests. * Mirrors: apps/backend/tests/test_recipients.py + * + * Runs against the REAL router mounted on a throwaway Express app (see + * tests/helpers/routeApp.js) — validateIdParam is no longer stubbed. */ import { describe, it, expect, vi, beforeEach } from 'vitest'; import { mockLogger } from '../helpers/mockLogger.js'; -import { createMockRouter, createMockResponse } from '../helpers/routeHarness.js'; - -const { router: mockRouter, handlers: routeHandlers } = createMockRouter(); - -vi.mock('express', () => ({ - default: { Router: () => mockRouter }, - Router: () => mockRouter, -})); +import { routeAgent, errEnvelope } from '../helpers/routeApp.js'; +// The route imports its repository through services/recipientService.js, which +// re-exports the default from this module — mocking the repository here +// intercepts that same binding. vi.mock('../../src/repositories/recipientRepository.js', () => ({ default: { getAll: vi.fn(), @@ -44,6 +43,10 @@ vi.mock('../../src/services/recipientClusterService.js', () => ({ findRecipientClusters: vi.fn(), })); +vi.mock('../../src/services/materializedViewService.js', () => ({ + scheduleRefresh: vi.fn(), +})); + vi.mock('../../src/config/logger.js', () => ({ logger: mockLogger(), })); @@ -51,8 +54,11 @@ vi.mock('../../src/config/logger.js', () => ({ import recipientRepository from '../../src/repositories/recipientRepository.js'; import { mergeRecipients as mergeRecipientsAtomic } from '../../src/services/recipientMergeService.js'; import { updatePattern, deletePattern } from '../../src/services/recipientPatternService.js'; -import { ValidationError, NotFoundError } from '../../src/middleware/errorHandler.js'; -await import('../../src/routes/recipients.js'); + +const { default: recipientsRouter } = await import('../../src/routes/recipients.js'); + +const api = routeAgent(recipientsRouter, { mountPath: '/api/recipients' }); +const BASE = '/api/recipients'; describe('Recipient Routes', () => { beforeEach(() => vi.resetAllMocks()); @@ -62,14 +68,11 @@ describe('Recipient Routes', () => { recipientRepository.getAll.mockResolvedValue([]); recipientRepository.getCount.mockResolvedValue(0); - const req = { query: {} }; - const res = mockResponse(); - await routeHandlers['get:/'](req, res); + const res = await api.get(BASE).expect(200); - const result = res.json.mock.calls[0][0]; - expect(result.ok).toBe(true); - expect(result.data.items).toEqual([]); - expect(result.data.total).toBe(0); + expect(res.body.ok).toBe(true); + expect(res.body.data.items).toEqual([]); + expect(res.body.data.total).toBe(0); }); it('should return recipients with data', async () => { @@ -79,11 +82,9 @@ describe('Recipient Routes', () => { ]); recipientRepository.getCount.mockResolvedValue(2); - const req = { query: {} }; - const res = mockResponse(); - await routeHandlers['get:/'](req, res); + const res = await api.get(BASE).expect(200); - expect(res.json.mock.calls[0][0].data.total).toBe(2); + expect(res.body.data.total).toBe(2); }); }); @@ -94,11 +95,7 @@ describe('Recipient Routes', () => { created: true, }); - const req = { body: { name: 'John Doe' } }; - const res = mockResponse(); - await routeHandlers['post:/'](req, res); - - expect(res.status).toHaveBeenCalledWith(201); + await api.post(BASE).send({ name: 'John Doe' }).expect(201); }); it('should return 200 for duplicate', async () => { @@ -107,17 +104,12 @@ describe('Recipient Routes', () => { created: false, }); - const req = { body: { name: 'John Doe' } }; - const res = mockResponse(); - await routeHandlers['post:/'](req, res); - - expect(res.status).toHaveBeenCalledWith(200); + await api.post(BASE).send({ name: 'John Doe' }).expect(200); }); - it('should throw ValidationError for missing name', async () => { - const req = { body: {} }; - const res = mockResponse(); - await expect(routeHandlers['post:/'](req, res)).rejects.toBeInstanceOf(ValidationError); + it('should return a 400 VALIDATION_ERROR envelope for missing name', async () => { + const res = await api.post(BASE).send({}).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); }); }); @@ -125,19 +117,23 @@ describe('Recipient Routes', () => { it('should return recipient by id', async () => { recipientRepository.getById.mockResolvedValue({ id: 1, name: 'JOHN DOE' }); - const req = { params: { id: '1' } }; - const res = mockResponse(); - await routeHandlers['get:/:id'](req, res); - - expect(res.json).toHaveBeenCalled(); + const res = await api.get(`${BASE}/1`).expect(200); + expect(res.body.data.id).toBe(1); }); - it('should throw NotFoundError for non-existent', async () => { + it('should return a 404 NOT_FOUND envelope for non-existent', async () => { recipientRepository.getById.mockResolvedValue(null); - const req = { params: { id: '99999' } }; - const res = mockResponse(); - await expect(routeHandlers['get:/:id'](req, res)).rejects.toBeInstanceOf(NotFoundError); + const res = await api.get(`${BASE}/99999`).expect(404); + expect(res.body).toEqual(errEnvelope({ code: 'NOT_FOUND' })); + }); + + it('rejects a non-integer :id via the real validateIdParam guard', async () => { + // Previously `vi.mock('.../middleware/validation.js')` replaced + // validateIdParam with a pass-through, so this guard was never tested. + const res = await api.get(`${BASE}/abc`).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); + expect(recipientRepository.getById).not.toHaveBeenCalled(); }); }); @@ -145,19 +141,15 @@ describe('Recipient Routes', () => { it('should update recipient', async () => { recipientRepository.update.mockResolvedValue({ id: 1, name: 'UPDATED' }); - const req = { params: { id: '1' }, body: { notes: 'new' } }; - const res = mockResponse(); - await routeHandlers['patch:/:id'](req, res); - - expect(res.json).toHaveBeenCalled(); + const res = await api.patch(`${BASE}/1`).send({ notes: 'new' }).expect(200); + expect(res.body.data.name).toBe('UPDATED'); }); - it('should throw NotFoundError for non-existent', async () => { + it('should return a 404 NOT_FOUND envelope for non-existent', async () => { recipientRepository.update.mockResolvedValue(null); - const req = { params: { id: '99999' }, body: { notes: 'x' } }; - const res = mockResponse(); - await expect(routeHandlers['patch:/:id'](req, res)).rejects.toBeInstanceOf(NotFoundError); + const res = await api.patch(`${BASE}/99999`).send({ notes: 'x' }).expect(404); + expect(res.body).toEqual(errEnvelope({ code: 'NOT_FOUND' })); }); }); @@ -165,37 +157,29 @@ describe('Recipient Routes', () => { it('should delete recipient and return 204 with no body', async () => { recipientRepository.hardDelete.mockResolvedValue(true); - const req = { params: { id: '1' } }; - const res = mockResponse(); - await routeHandlers['delete:/:id'](req, res); - - expect(res.status).toHaveBeenCalledWith(204); - expect(res.send).toHaveBeenCalledWith(); - expect(res.json).not.toHaveBeenCalled(); + const res = await api.delete(`${BASE}/1`).expect(204); + expect(res.text).toBe(''); }); - it('should throw NotFoundError for non-existent', async () => { + it('should return a 404 NOT_FOUND envelope for non-existent', async () => { recipientRepository.hardDelete.mockResolvedValue(false); - const req = { params: { id: '99999' } }; - const res = mockResponse(); - await expect(routeHandlers['delete:/:id'](req, res)).rejects.toBeInstanceOf(NotFoundError); + const res = await api.delete(`${BASE}/99999`).expect(404); + expect(res.body).toEqual(errEnvelope({ code: 'NOT_FOUND' })); }); }); describe('POST /:id/merge', () => { - it('should throw ValidationError when alias_ids is missing', async () => { - const req = { params: { id: '1' }, body: {} }; - const res = mockResponse(); - await expect(routeHandlers['post:/:id/merge'](req, res)).rejects.toBeInstanceOf(ValidationError); + it('should return a 400 VALIDATION_ERROR envelope when alias_ids is missing', async () => { + const res = await api.post(`${BASE}/1/merge`).send({}).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); }); - it('should throw ValidationError when primary recipient is itself an alias', async () => { + it('should return a 400 VALIDATION_ERROR envelope when primary recipient is itself an alias', async () => { recipientRepository.getById.mockResolvedValue({ id: 1, primary_recipient_id: 2 }); - const req = { params: { id: '1' }, body: { alias_ids: [3] } }; - const res = mockResponse(); - await expect(routeHandlers['post:/:id/merge'](req, res)).rejects.toBeInstanceOf(ValidationError); + const res = await api.post(`${BASE}/1/merge`).send({ alias_ids: [3] }).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); }); it('should merge aliases and return primary plus aliases', async () => { @@ -211,58 +195,46 @@ describe('Recipient Routes', () => { { id: 4, name: 'ALIAS B' }, ]); - const req = { params: { id: '1' }, body: { alias_ids: ['3', '4'] } }; - const res = mockResponse(); - await routeHandlers['post:/:id/merge'](req, res); + const res = await api.post(`${BASE}/1/merge`).send({ alias_ids: ['3', '4'] }).expect(200); expect(mergeRecipientsAtomic).toHaveBeenCalledWith(1, [3, 4]); - expect(res.json).toHaveBeenCalledWith({ - ok: true, - data: { - primary: { id: 1, name: 'PRIMARY', primary_recipient_id: null, links: [] }, - merged_ids: [3, 4], - reassigned: { transactions: 7, splits: 0, planned: 0, bankAccounts: 1 }, - aliases: [ - { id: 3, name: 'ALIAS A' }, - { id: 4, name: 'ALIAS B' }, - ], - patternSuggestion: null, - }, + expect(res.body.data).toEqual({ + primary: { id: 1, name: 'PRIMARY', primary_recipient_id: null, links: [] }, + merged_ids: [3, 4], + reassigned: { transactions: 7, splits: 0, planned: 0, bankAccounts: 1 }, + aliases: [ + { id: 3, name: 'ALIAS A' }, + { id: 4, name: 'ALIAS B' }, + ], + patternSuggestion: null, }); }); - it('should throw NotFoundError when primary recipient does not exist', async () => { + it('should return a 404 NOT_FOUND envelope when primary recipient does not exist', async () => { recipientRepository.getById.mockResolvedValue(null); - const req = { params: { id: '123' }, body: { alias_ids: [5] } }; - const res = mockResponse(); - await expect(routeHandlers['post:/:id/merge'](req, res)).rejects.toBeInstanceOf(NotFoundError); + const res = await api.post(`${BASE}/123/merge`).send({ alias_ids: [5] }).expect(404); + expect(res.body).toEqual(errEnvelope({ code: 'NOT_FOUND' })); }); }); describe('POST /:id/unmerge', () => { - it('should throw NotFoundError when recipient cannot be unmerged', async () => { + it('should return a 404 NOT_FOUND envelope when recipient cannot be unmerged', async () => { recipientRepository.unmergeRecipient.mockResolvedValue(false); - const req = { params: { id: '44' } }; - const res = mockResponse(); - await expect(routeHandlers['post:/:id/unmerge'](req, res)).rejects.toBeInstanceOf(NotFoundError); + const res = await api.post(`${BASE}/44/unmerge`).expect(404); + expect(res.body).toEqual(errEnvelope({ code: 'NOT_FOUND' })); }); it('should return updated recipient when unmerge succeeds', async () => { recipientRepository.unmergeRecipient.mockResolvedValue(true); recipientRepository.getById.mockResolvedValue({ id: 44, name: 'UNMERGED', primary_recipient_id: null }); - const req = { params: { id: '44' } }; - const res = mockResponse(); - await routeHandlers['post:/:id/unmerge'](req, res); + const res = await api.post(`${BASE}/44/unmerge`).expect(200); expect(recipientRepository.unmergeRecipient).toHaveBeenCalledWith(44); expect(recipientRepository.getById).toHaveBeenCalledWith(44); - expect(res.json).toHaveBeenCalledWith({ - ok: true, - data: { id: 44, name: 'UNMERGED', primary_recipient_id: null, links: [] }, - }); + expect(res.body.data).toEqual({ id: 44, name: 'UNMERGED', primary_recipient_id: null, links: [] }); }); }); @@ -273,57 +245,37 @@ describe('Recipient Routes', () => { { id: 11, name: 'Alias Two', primary_recipient_id: 1 }, ]); - const req = { params: { id: '1' } }; - const res = mockResponse(); - await routeHandlers['get:/:id/aliases'](req, res); + const res = await api.get(`${BASE}/1/aliases`).expect(200); expect(recipientRepository.getAliases).toHaveBeenCalledWith(1); - expect(res.json).toHaveBeenCalledWith({ - ok: true, - data: { - items: [ - { id: 10, name: 'Alias One', primary_recipient_id: 1, links: [] }, - { id: 11, name: 'Alias Two', primary_recipient_id: 1, links: [] }, - ], - total: 2, - }, + expect(res.body.data).toEqual({ + items: [ + { id: 10, name: 'Alias One', primary_recipient_id: 1, links: [] }, + { id: 11, name: 'Alias Two', primary_recipient_id: 1, links: [] }, + ], + total: 2, }); }); }); describe('pattern sub-route id guards', () => { it('PATCH /:id/patterns/:patternId rejects a negative patternId', async () => { - const req = { params: { id: '1', patternId: '-3' }, body: {} }; - const res = mockResponse(); - - await expect(routeHandlers['patch:/:id/patterns/:patternId'](req, res)) - .rejects.toBeInstanceOf(ValidationError); + const res = await api.patch(`${BASE}/1/patterns/-3`).send({}).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); expect(updatePattern).not.toHaveBeenCalled(); }); it('DELETE /:id/patterns/:patternId rejects a zero patternId', async () => { - const req = { params: { id: '1', patternId: '0' } }; - const res = mockResponse(); - - await expect(routeHandlers['delete:/:id/patterns/:patternId'](req, res)) - .rejects.toBeInstanceOf(ValidationError); + const res = await api.delete(`${BASE}/1/patterns/0`).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); expect(deletePattern).not.toHaveBeenCalled(); }); it('DELETE /:id/patterns/:patternId returns 204 with no body', async () => { - const req = { params: { id: '1', patternId: '77' } }; - const res = mockResponse(); - - await routeHandlers['delete:/:id/patterns/:patternId'](req, res); + const res = await api.delete(`${BASE}/1/patterns/77`).expect(204); expect(deletePattern).toHaveBeenCalledWith(77); - expect(res.status).toHaveBeenCalledWith(204); - expect(res.send).toHaveBeenCalledWith(); - expect(res.json).not.toHaveBeenCalled(); + expect(res.text).toBe(''); }); }); }); - -function mockResponse() { - return createMockResponse(); -} diff --git a/apps/node-backend/tests/routes/research.test.js b/apps/node-backend/tests/routes/research.test.js index 455fe88d..9fe250a1 100644 --- a/apps/node-backend/tests/routes/research.test.js +++ b/apps/node-backend/tests/routes/research.test.js @@ -5,16 +5,14 @@ * scalar normalization, per-endpoint symbol requireds, key_type set + default, * instrument_key/query requireds, positiveInt coercion, and the macro * provider/series_id guards. + * + * Runs against the REAL router mounted on a throwaway Express app (see + * tests/helpers/routeApp.js). main.js:330 also mounts `marketRateLimiter` + * before this router — deliberately not reproduced here (module-level counter + * shared across the whole worker; see routeApp.js's fidelity map). */ import { beforeEach, describe, expect, it, vi } from 'vitest'; -import { createMockRouter, createMockResponse } from '../helpers/routeHarness.js'; - -const { router: mockRouter, handlers: routeHandlers } = createMockRouter(); - -vi.mock('express', () => ({ - default: { Router: () => mockRouter }, - Router: () => mockRouter, -})); +import { routeAgent, errEnvelope } from '../helpers/routeApp.js'; vi.mock('../../src/services/research/researchAggregator.js', () => ({ researchAggregator: { @@ -53,8 +51,11 @@ import { researchAggregator } from '../../src/services/research/researchAggregat import { researchMappingService } from '../../src/services/research/researchMappingService.js'; import { clearKey, listKeyStatuses } from '../../src/services/research/researchProviderKeyService.js'; import { runPortfolioForecast } from '../../src/services/research/projection/portfolioProjection.js'; -import { ValidationError } from '../../src/middleware/errorHandler.js'; -await import('../../src/routes/research.js'); + +const { default: researchRouter } = await import('../../src/routes/research.js'); + +const api = routeAgent(researchRouter, { mountPath: '/api/research' }); +const BASE = '/api/research'; describe('Research route parameter guards', () => { beforeEach(() => { @@ -67,40 +68,42 @@ describe('Research route parameter guards', () => { }); describe('symbol requireds', () => { - it.each(['get:/quote', 'get:/chart', 'get:/analyst', 'get:/news'])( - '%s rejects a missing/empty symbol', - async (key) => { - await expect(routeHandlers[key]({ query: {} }, createMockResponse())) - .rejects.toThrow('symbol parameter required'); - await expect(routeHandlers[key]({ query: { symbol: ' ' } }, createMockResponse())) - .rejects.toBeInstanceOf(ValidationError); + it.each(['/quote', '/chart', '/analyst', '/news'])( + 'GET %s rejects a missing/empty symbol', + async (path) => { + const res1 = await api.get(`${BASE}${path}`).expect(400); + expect(res1.body.error.message).toBe('symbol parameter required'); + + const res2 = await api.get(`${BASE}${path}`).query({ symbol: ' ' }).expect(400); + expect(res2.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); }, ); it('uses the first entry of an array-valued symbol and trims scalars', async () => { - await routeHandlers['get:/quote']({ query: { symbol: ['AAPL', 'MSFT'] } }, createMockResponse()); + await api.get(`${BASE}/quote?symbol=AAPL&symbol=MSFT`).expect(200); expect(researchAggregator.fetch).toHaveBeenCalledWith('quote', expect.objectContaining({ symbol: 'AAPL' })); - await routeHandlers['get:/news']({ query: { symbol: ' TSLA ' } }, createMockResponse()); + await api.get(`${BASE}/news`).query({ symbol: ' TSLA ' }).expect(200); expect(researchAggregator.fetch).toHaveBeenLastCalledWith('news', { symbol: 'TSLA' }); }); }); describe('macro series guards', () => { it('rejects an unknown provider with the provider list', async () => { - await expect(routeHandlers['get:/macro/series']({ query: { provider: 'nope', series_id: 'CPIAUCSL' } }, createMockResponse())) - .rejects.toThrow('provider must be one of: fred, eurostat, dbnomics'); + const res = await api.get(`${BASE}/macro/series`).query({ provider: 'nope', series_id: 'CPIAUCSL' }).expect(400); + expect(res.body.error.message).toBe('provider must be one of: fred, eurostat, dbnomics'); }); it('rejects a series_id that fails the provider shape', async () => { - await expect(routeHandlers['get:/macro/series']({ query: { provider: 'fred', series_id: 'a/b/c' } }, createMockResponse())) - .rejects.toThrow('valid series_id required for the given provider'); - await expect(routeHandlers['get:/macro/series']({ query: { provider: 'fred' } }, createMockResponse())) - .rejects.toThrow('valid series_id required for the given provider'); + const res1 = await api.get(`${BASE}/macro/series`).query({ provider: 'fred', series_id: 'a/b/c' }).expect(400); + expect(res1.body.error.message).toBe('valid series_id required for the given provider'); + + const res2 = await api.get(`${BASE}/macro/series`).query({ provider: 'fred' }).expect(400); + expect(res2.body.error.message).toBe('valid series_id required for the given provider'); }); it('fetches with defaulted range for a valid provider/series pair', async () => { - await routeHandlers['get:/macro/series']({ query: { provider: 'fred', series_id: 'CPIAUCSL' } }, createMockResponse()); + await api.get(`${BASE}/macro/series`).query({ provider: 'fred', series_id: 'CPIAUCSL' }).expect(200); expect(researchAggregator.fetchMacroSeries).toHaveBeenCalledWith({ provider: 'fred', seriesId: 'CPIAUCSL', range: '5y', }); @@ -109,36 +112,32 @@ describe('Research route parameter guards', () => { describe('mapping guards', () => { it('GET /mappings requires instrument_key and defaults key_type to isin', async () => { - await expect(routeHandlers['get:/mappings']({ query: {} }, createMockResponse())) - .rejects.toThrow('instrument_key required'); + const res = await api.get(`${BASE}/mappings`).expect(400); + expect(res.body.error.message).toBe('instrument_key required'); - await routeHandlers['get:/mappings']({ query: { instrument_key: 'US0378331005' } }, createMockResponse()); + await api.get(`${BASE}/mappings`).query({ instrument_key: 'US0378331005' }).expect(200); expect(researchMappingService.list).toHaveBeenCalledWith('US0378331005', 'isin'); }); it('rejects an unknown key_type', async () => { - await expect( - routeHandlers['get:/mappings']({ query: { instrument_key: 'X', key_type: 'weird' } }, createMockResponse()), - ).rejects.toThrow("key_type must be 'isin' or 'internal'"); + const res = await api.get(`${BASE}/mappings`).query({ instrument_key: 'X', key_type: 'weird' }).expect(400); + expect(res.body.error.message).toBe("key_type must be 'isin' or 'internal'"); }); it('POST /mappings/resolve requires query and coerces investment_id via parseInt', async () => { - await expect( - routeHandlers['post:/mappings/resolve']({ body: { instrument_key: 'X', query: '' } }, createMockResponse()), - ).rejects.toThrow('query required'); + const res = await api.post(`${BASE}/mappings/resolve`).send({ instrument_key: 'X', query: '' }).expect(400); + expect(res.body.error.message).toBe('query required'); - await routeHandlers['post:/mappings/resolve']( - { body: { instrument_key: 'X', key_type: 'internal', query: 'apple', investment_id: '5' } }, - createMockResponse(), - ); + await api.post(`${BASE}/mappings/resolve`) + .send({ instrument_key: 'X', key_type: 'internal', query: 'apple', investment_id: '5' }) + .expect(200); expect(researchMappingService.resolve).toHaveBeenCalledWith(expect.objectContaining({ instrumentKey: 'X', keyType: 'internal', query: 'apple', investmentId: 5, })); - await routeHandlers['post:/mappings/resolve']( - { body: { instrument_key: 'X', query: 'apple', investment_id: 'abc' } }, - createMockResponse(), - ); + await api.post(`${BASE}/mappings/resolve`) + .send({ instrument_key: 'X', query: 'apple', investment_id: 'abc' }) + .expect(200); expect(researchMappingService.resolve).toHaveBeenLastCalledWith( expect.objectContaining({ investmentId: undefined }), ); @@ -146,46 +145,38 @@ describe('Research route parameter guards', () => { it('POST /mappings rejects a missing or empty mappings array', async () => { for (const mappings of [undefined, 'x', []]) { - await expect( - routeHandlers['post:/mappings']({ body: { instrument_key: 'X', mappings } }, createMockResponse()), - ).rejects.toThrow('mappings must be a non-empty array'); + const res = await api.post(`${BASE}/mappings`).send({ instrument_key: 'X', mappings }).expect(400); + expect(res.body.error.message).toBe('mappings must be a non-empty array'); } }); it('DELETE /mappings/:id keeps parseInt id coercion and answers 204', async () => { researchMappingService.remove.mockResolvedValue(true); - const res = createMockResponse(); - await routeHandlers['delete:/mappings/:id']({ params: { id: '12abc' } }, res); + const res = await api.delete(`${BASE}/mappings/12abc`).expect(204); expect(researchMappingService.remove).toHaveBeenCalledWith(12); - expect(res.status).toHaveBeenCalledWith(204); - expect(res.send).toHaveBeenCalledWith(); - expect(res.json).not.toHaveBeenCalled(); - await expect(routeHandlers['delete:/mappings/:id']({ params: { id: 'abc' } }, createMockResponse())) - .rejects.toThrow('valid mapping id required'); + expect(res.text).toBe(''); + + const res2 = await api.delete(`${BASE}/mappings/abc`).expect(400); + expect(res2.body.error.message).toBe('valid mapping id required'); }); // Idempotent: an already-removed mapping is still 204, not 404. it('DELETE /mappings/:id answers 204 when nothing was removed', async () => { researchMappingService.remove.mockResolvedValue(false); - const res = createMockResponse(); - await routeHandlers['delete:/mappings/:id']({ params: { id: '5' } }, res); - expect(res.status).toHaveBeenCalledWith(204); - expect(res.json).not.toHaveBeenCalled(); + const res = await api.delete(`${BASE}/mappings/5`).expect(204); + expect(res.text).toBe(''); }); }); describe('DELETE /provider-keys/:provider', () => { it('clears the key and answers 204 with no body', async () => { clearKey.mockResolvedValue(true); - const res = createMockResponse(); - await routeHandlers['delete:/provider-keys/:provider']({ params: { provider: 'finnhub' } }, res); + const res = await api.delete(`${BASE}/provider-keys/finnhub`).expect(204); expect(clearKey).toHaveBeenCalledWith('finnhub'); // The statuses are refetched by the caller — the delete must not re-read them. expect(listKeyStatuses).not.toHaveBeenCalled(); - expect(res.status).toHaveBeenCalledWith(204); - expect(res.send).toHaveBeenCalledWith(); - expect(res.json).not.toHaveBeenCalled(); + expect(res.text).toBe(''); }); }); @@ -195,16 +186,13 @@ describe('Research route parameter guards', () => { describe('POST /portfolio-forecast body casing', () => { it('reads the snake_case spellings', async () => { runPortfolioForecast.mockResolvedValue({ bands: [] }); - const res = createMockResponse(); - await routeHandlers['post:/portfolio-forecast']({ - body: { - horizon_months: 24, - monthly_contribution: 250, - paths: 500, - forward_blend: 0.5, - target_value: 100000, - }, - }, res); + const res = await api.post(`${BASE}/portfolio-forecast`).send({ + horizon_months: 24, + monthly_contribution: 250, + paths: 500, + forward_blend: 0.5, + target_value: 100000, + }).expect(200); expect(runPortfolioForecast).toHaveBeenCalledWith(expect.objectContaining({ horizonMonths: 24, @@ -213,19 +201,17 @@ describe('Research route parameter guards', () => { forwardBlend: 0.5, targetValue: 100000, })); - expect(res.json.mock.calls[0][0].data).toEqual({ bands: [] }); + expect(res.body.data).toEqual({ bands: [] }); }); it('no longer accepts the camelCase spellings', async () => { runPortfolioForecast.mockResolvedValue({ bands: [] }); - await routeHandlers['post:/portfolio-forecast']({ - body: { - horizonMonths: 24, - monthlyContribution: 250, - forwardBlend: 0.5, - targetValue: 100000, - }, - }, createMockResponse()); + await api.post(`${BASE}/portfolio-forecast`).send({ + horizonMonths: 24, + monthlyContribution: 250, + forwardBlend: 0.5, + targetValue: 100000, + }).expect(200); expect(runPortfolioForecast).toHaveBeenCalledWith(expect.objectContaining({ horizonMonths: undefined, @@ -237,9 +223,9 @@ describe('Research route parameter guards', () => { it('does not fall back to camelCase when the snake_case key is absent', async () => { runPortfolioForecast.mockResolvedValue({ bands: [] }); - await routeHandlers['post:/portfolio-forecast']({ - body: { horizon_months: 12, monthlyContribution: 999 }, - }, createMockResponse()); + await api.post(`${BASE}/portfolio-forecast`).send({ + horizon_months: 12, monthlyContribution: 999, + }).expect(200); expect(runPortfolioForecast).toHaveBeenCalledWith(expect.objectContaining({ horizonMonths: 12, @@ -250,10 +236,9 @@ describe('Research route parameter guards', () => { describe('search empty-q short-circuit', () => { it('returns an empty payload without calling the aggregator', async () => { - const res = createMockResponse(); - await routeHandlers['get:/search']({ query: {} }, res); + const res = await api.get(`${BASE}/search`).expect(200); expect(researchAggregator.fetch).not.toHaveBeenCalled(); - expect(res.json.mock.calls[0][0].data).toEqual({ items: [] }); + expect(res.body.data).toEqual({ items: [] }); }); }); }); diff --git a/apps/node-backend/tests/routes/savedCharts.test.js b/apps/node-backend/tests/routes/savedCharts.test.js index ace19a4c..f7fa238c 100644 --- a/apps/node-backend/tests/routes/savedCharts.test.js +++ b/apps/node-backend/tests/routes/savedCharts.test.js @@ -1,13 +1,16 @@ +/** + * Saved Charts route tests. + * + * Runs against the REAL router mounted on a throwaway Express app (see + * tests/helpers/routeApp.js). All validation in savedCharts.js is inline in + * the handlers (no route-level guard middleware — `:id` is parsed with a bare + * `parseInt`), so this migration is mechanical: `routeHandlers[...]` calls + * become supertest requests and `.rejects.toBeInstanceOf(...)` assertions + * become status-code + envelope assertions against the real error handler. + */ import { beforeEach, describe, expect, it, vi } from 'vitest'; import { mockLogger } from '../helpers/mockLogger.js'; -import { createMockRouter, createMockResponse } from '../helpers/routeHarness.js'; - -const { router: mockRouter, handlers: routeHandlers } = createMockRouter(); - -vi.mock('express', () => ({ - default: { Router: () => mockRouter }, - Router: () => mockRouter, -})); +import { routeAgent, okEnvelope, errEnvelope } from '../helpers/routeApp.js'; vi.mock('../../src/repositories/savedChartsRepository.js', () => ({ default: { @@ -24,9 +27,11 @@ vi.mock('../../src/config/logger.js', () => ({ })); import savedChartsRepository from '../../src/repositories/savedChartsRepository.js'; -import { ValidationError, NotFoundError } from '../../src/middleware/errorHandler.js'; -await import('../../src/routes/savedCharts.js'); +const { default: savedChartsRouter } = await import('../../src/routes/savedCharts.js'); + +const api = routeAgent(savedChartsRouter, { mountPath: '/api/saved-charts' }); +const BASE = '/api/saved-charts'; describe('Saved Charts Routes', () => { beforeEach(() => { @@ -37,73 +42,51 @@ describe('Saved Charts Routes', () => { it('returns all saved charts', async () => { savedChartsRepository.getAll.mockResolvedValue([{ id: 1 }]); - const req = {}; - const res = mockResponse(); - - await routeHandlers['get:/'](req, res); + const res = await api.get(BASE).expect(200); // No limit/offset on the request → unbounded query, canonical collection // shape { items, total } with total = row count and no COUNT round-trip. expect(savedChartsRepository.getAll).toHaveBeenCalledWith({}); expect(savedChartsRepository.getCount).not.toHaveBeenCalled(); - expect(res.json).toHaveBeenCalledWith({ ok: true, data: { items: [{ id: 1 }], total: 1 } }); + expect(res.body).toEqual(okEnvelope({ items: [{ id: 1 }], total: 1 })); }); it('pages and reports the full total when limit/offset are supplied', async () => { savedChartsRepository.getAll.mockResolvedValue([{ id: 2 }]); savedChartsRepository.getCount.mockResolvedValue(5); - const req = { query: { limit: '1', offset: '1' } }; - const res = mockResponse(); - - await routeHandlers['get:/'](req, res); + const res = await api.get(BASE).query({ limit: '1', offset: '1' }).expect(200); expect(savedChartsRepository.getAll).toHaveBeenCalledWith({ limit: 1, offset: 1 }); - expect(res.json).toHaveBeenCalledWith({ - ok: true, - data: { items: [{ id: 2 }], total: 5, limit: 1, offset: 1 }, - }); + expect(res.body).toEqual(okEnvelope({ items: [{ id: 2 }], total: 5, limit: 1, offset: 1 })); }); it('propagates error when repository fails', async () => { savedChartsRepository.getAll.mockRejectedValue(new Error('db down')); - const req = {}; - const res = mockResponse(); - - await expect(routeHandlers['get:/'](req, res)).rejects.toThrow('db down'); + const res = await api.get(BASE).expect(500); + expect(res.body.error.message).toBe('db down'); }); }); describe('POST /', () => { it('throws ValidationError when name is missing', async () => { - const req = { body: { chartType: 'line', categoryIds: [1] } }; - const res = mockResponse(); - - await expect(routeHandlers['post:/'](req, res)).rejects.toBeInstanceOf(ValidationError); + const res = await api.post(BASE).send({ chartType: 'line', categoryIds: [1] }).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); }); it('throws ValidationError when chartType is invalid', async () => { - const req = { body: { name: 'Main', chartType: 'pie', categoryIds: [1] } }; - const res = mockResponse(); - - await expect(routeHandlers['post:/'](req, res)).rejects.toBeInstanceOf(ValidationError); + await api.post(BASE).send({ name: 'Main', chartType: 'pie', categoryIds: [1] }).expect(400); }); it('throws ValidationError when categoryIds has invalid entries', async () => { - const req = { body: { name: 'Main', chartType: 'line', categoryIds: [1, 'nope'] } }; - const res = mockResponse(); - - await expect(routeHandlers['post:/'](req, res)).rejects.toBeInstanceOf(ValidationError); + await api.post(BASE).send({ name: 'Main', chartType: 'line', categoryIds: [1, 'nope'] }).expect(400); }); it('creates chart with trimmed name and default chart type', async () => { savedChartsRepository.create.mockResolvedValue({ id: 4, name: 'Main', chart_type: 'line', category_ids: [1, 2] }); - const req = { body: { name: ' Main ', categoryIds: ['1', 2] } }; - const res = mockResponse(); - - await routeHandlers['post:/'](req, res); + const res = await api.post(BASE).send({ name: ' Main ', categoryIds: ['1', 2] }).expect(201); expect(savedChartsRepository.create).toHaveBeenCalledWith({ name: 'Main', @@ -118,17 +101,13 @@ describe('Saved Charts Routes', () => { dateRangeStart: undefined, dateRangeEnd: undefined, }); - expect(res.status).toHaveBeenCalledWith(201); - expect(res.json).toHaveBeenCalledWith({ ok: true, data: { id: 4, name: 'Main', chart_type: 'line', category_ids: [1, 2] } }); + expect(res.body).toEqual(okEnvelope({ id: 4, name: 'Main', chart_type: 'line', category_ids: [1, 2] })); }); it('normalizes tagIds when provided', async () => { savedChartsRepository.create.mockResolvedValue({ id: 5, name: 'Tagged', tag_ids: [5, 6] }); - const req = { body: { name: 'Tagged', categoryIds: [], tagIds: ['5', 6] } }; - const res = mockResponse(); - - await routeHandlers['post:/'](req, res); + await api.post(BASE).send({ name: 'Tagged', categoryIds: [], tagIds: ['5', 6] }).expect(201); expect(savedChartsRepository.create).toHaveBeenCalledWith( expect.objectContaining({ tagIds: [5, 6] }), @@ -136,19 +115,13 @@ describe('Saved Charts Routes', () => { }); it('throws ValidationError when tagIds has invalid entries', async () => { - const req = { body: { name: 'Main', categoryIds: [], tagIds: [1, 'nope'] } }; - const res = mockResponse(); - - await expect(routeHandlers['post:/'](req, res)).rejects.toBeInstanceOf(ValidationError); + await api.post(BASE).send({ name: 'Main', categoryIds: [], tagIds: [1, 'nope'] }).expect(400); }); it('passes all-source flags through to the repository', async () => { savedChartsRepository.create.mockResolvedValue({ id: 6, name: 'AllTags' }); - const req = { body: { name: 'AllTags', categoryIds: [], allTags: true } }; - const res = mockResponse(); - - await routeHandlers['post:/'](req, res); + await api.post(BASE).send({ name: 'AllTags', categoryIds: [], allTags: true }).expect(201); expect(savedChartsRepository.create).toHaveBeenCalledWith( expect.objectContaining({ allTags: true, allCategories: false, allRecipients: false }), @@ -156,19 +129,13 @@ describe('Saved Charts Routes', () => { }); it('throws ValidationError when an all-source flag is not a boolean', async () => { - const req = { body: { name: 'Main', categoryIds: [], allTags: 'yes' } }; - const res = mockResponse(); - - await expect(routeHandlers['post:/'](req, res)).rejects.toBeInstanceOf(ValidationError); + await api.post(BASE).send({ name: 'Main', categoryIds: [], allTags: 'yes' }).expect(400); }); it('accepts the ranked variant on a bar chart', async () => { savedChartsRepository.create.mockResolvedValue({ id: 7, name: 'Ranked' }); - const req = { body: { name: 'Ranked', categoryIds: [1], chartType: 'bar', chartVariant: 'ranked' } }; - const res = mockResponse(); - - await routeHandlers['post:/'](req, res); + await api.post(BASE).send({ name: 'Ranked', categoryIds: [1], chartType: 'bar', chartVariant: 'ranked' }).expect(201); expect(savedChartsRepository.create).toHaveBeenCalledWith( expect.objectContaining({ chartType: 'bar', chartVariant: 'ranked' }), @@ -176,37 +143,27 @@ describe('Saved Charts Routes', () => { }); it('rejects the ranked variant on a line chart', async () => { - const req = { body: { name: 'Bad', categoryIds: [1], chartType: 'line', chartVariant: 'ranked' } }; - const res = mockResponse(); - - await expect(routeHandlers['post:/'](req, res)).rejects.toBeInstanceOf(ValidationError); + await api.post(BASE).send({ name: 'Bad', categoryIds: [1], chartType: 'line', chartVariant: 'ranked' }).expect(400); }); it('rejects a whitespace-only name', async () => { - const req = { body: { name: ' ', categoryIds: [1] } }; - - await expect(routeHandlers['post:/'](req, mockResponse())).rejects.toBeInstanceOf(ValidationError); + await api.post(BASE).send({ name: ' ', categoryIds: [1] }).expect(400); expect(savedChartsRepository.create).not.toHaveBeenCalled(); }); it('rejects a non-string name', async () => { - const req = { body: { name: 123, categoryIds: [1] } }; - - await expect(routeHandlers['post:/'](req, mockResponse())).rejects.toBeInstanceOf(ValidationError); + await api.post(BASE).send({ name: 123, categoryIds: [1] }).expect(400); }); it('rejects a missing categoryIds field', async () => { - const req = { body: { name: 'Main' } }; - - await expect(routeHandlers['post:/'](req, mockResponse())).rejects.toBeInstanceOf(ValidationError); + await api.post(BASE).send({ name: 'Main' }).expect(400); expect(savedChartsRepository.create).not.toHaveBeenCalled(); }); it('wraps a scalar categoryIds into a one-element int array', async () => { savedChartsRepository.create.mockResolvedValue({ id: 10 }); - const req = { body: { name: 'Scalar', categoryIds: '5' } }; - await routeHandlers['post:/'](req, mockResponse()); + await api.post(BASE).send({ name: 'Scalar', categoryIds: '5' }).expect(201); expect(savedChartsRepository.create).toHaveBeenCalledWith( expect.objectContaining({ categoryIds: [5] }), @@ -216,39 +173,30 @@ describe('Saved Charts Routes', () => { it('normalizes recipientIds and rejects invalid entries', async () => { savedChartsRepository.create.mockResolvedValue({ id: 11 }); - await routeHandlers['post:/']({ body: { name: 'R', categoryIds: [], recipientIds: ['2', 3] } }, mockResponse()); + await api.post(BASE).send({ name: 'R', categoryIds: [], recipientIds: ['2', 3] }).expect(201); expect(savedChartsRepository.create).toHaveBeenCalledWith( expect.objectContaining({ recipientIds: [2, 3] }), ); - await expect( - routeHandlers['post:/']({ body: { name: 'R', categoryIds: [], recipientIds: [0] } }, mockResponse()), - ).rejects.toBeInstanceOf(ValidationError); + await api.post(BASE).send({ name: 'R', categoryIds: [], recipientIds: [0] }).expect(400); }); it('rejects an unknown chartVariant', async () => { - const req = { body: { name: 'Main', categoryIds: [1], chartVariant: 'wavy' } }; - - await expect(routeHandlers['post:/'](req, mockResponse())).rejects.toBeInstanceOf(ValidationError); + await api.post(BASE).send({ name: 'Main', categoryIds: [1], chartVariant: 'wavy' }).expect(400); }); it('rejects an unknown timeBucket', async () => { - const req = { body: { name: 'Main', categoryIds: [1], timeBucket: 'weekly' } }; - - await expect(routeHandlers['post:/'](req, mockResponse())).rejects.toBeInstanceOf(ValidationError); + await api.post(BASE).send({ name: 'Main', categoryIds: [1], timeBucket: 'weekly' }).expect(400); }); it('rejects a null chartType (only undefined may fall back to the default)', async () => { - const req = { body: { name: 'Main', categoryIds: [1], chartType: null } }; - - await expect(routeHandlers['post:/'](req, mockResponse())).rejects.toBeInstanceOf(ValidationError); + await api.post(BASE).send({ name: 'Main', categoryIds: [1], chartType: null }).expect(400); }); it('passes a yearly timeBucket through', async () => { savedChartsRepository.create.mockResolvedValue({ id: 12 }); - const req = { body: { name: 'Yearly', categoryIds: [1], timeBucket: 'yearly' } }; - await routeHandlers['post:/'](req, mockResponse()); + await api.post(BASE).send({ name: 'Yearly', categoryIds: [1], timeBucket: 'yearly' }).expect(201); expect(savedChartsRepository.create).toHaveBeenCalledWith( expect.objectContaining({ timeBucket: 'yearly' }), @@ -264,18 +212,14 @@ describe('Saved Charts Routes', () => { ['area', 'grouped'], ['area', 'ranked'], ])('rejects the illegal combination %s:%s', async (chartType, chartVariant) => { - const req = { body: { name: 'Combo', categoryIds: [1], chartType, chartVariant } }; - - await expect(routeHandlers['post:/'](req, mockResponse())).rejects.toBeInstanceOf(ValidationError); + await api.post(BASE).send({ name: 'Combo', categoryIds: [1], chartType, chartVariant }).expect(400); expect(savedChartsRepository.create).not.toHaveBeenCalled(); }); it('rejects a variant that is illegal against the DEFAULT line chartType', async () => { // No chartType in the body: the default 'line' must still participate in // the combination rule, so a bare stacked variant is rejected. - const req = { body: { name: 'Combo', categoryIds: [1], chartVariant: 'stacked' } }; - - await expect(routeHandlers['post:/'](req, mockResponse())).rejects.toBeInstanceOf(ValidationError); + await api.post(BASE).send({ name: 'Combo', categoryIds: [1], chartVariant: 'stacked' }).expect(400); }); it.each([ @@ -285,8 +229,7 @@ describe('Saved Charts Routes', () => { ])('accepts the legal combination %s:%s', async (chartType, chartVariant) => { savedChartsRepository.create.mockResolvedValue({ id: 13 }); - const req = { body: { name: 'Combo', categoryIds: [1], chartType, chartVariant } }; - await routeHandlers['post:/'](req, mockResponse()); + await api.post(BASE).send({ name: 'Combo', categoryIds: [1], chartType, chartVariant }).expect(201); expect(savedChartsRepository.create).toHaveBeenCalledWith( expect.objectContaining({ chartType, chartVariant }), @@ -294,16 +237,13 @@ describe('Saved Charts Routes', () => { }); it('rejects an unparsable dateRangeStart', async () => { - const req = { body: { name: 'Dated', categoryIds: [1], dateRangeStart: 'not-a-date' } }; - - await expect(routeHandlers['post:/'](req, mockResponse())).rejects.toBeInstanceOf(ValidationError); + await api.post(BASE).send({ name: 'Dated', categoryIds: [1], dateRangeStart: 'not-a-date' }).expect(400); }); it('passes a valid dateRangeStart through unchanged and maps empty string to null', async () => { savedChartsRepository.create.mockResolvedValue({ id: 14 }); - const req = { body: { name: 'Dated', categoryIds: [1], dateRangeStart: '2025-01-01', dateRangeEnd: '' } }; - await routeHandlers['post:/'](req, mockResponse()); + await api.post(BASE).send({ name: 'Dated', categoryIds: [1], dateRangeStart: '2025-01-01', dateRangeEnd: '' }).expect(201); expect(savedChartsRepository.create).toHaveBeenCalledWith( expect.objectContaining({ dateRangeStart: '2025-01-01', dateRangeEnd: null }), @@ -313,58 +253,41 @@ describe('Saved Charts Routes', () => { describe('PATCH /:id', () => { it('throws ValidationError when id is not a number', async () => { - const req = { params: { id: 'abc' }, body: {} }; - const res = mockResponse(); - - await expect(routeHandlers['patch:/:id'](req, res)).rejects.toBeInstanceOf(ValidationError); + await api.patch(`${BASE}/abc`).send({}).expect(400); }); it('throws ValidationError when name is blank after trimming', async () => { - const req = { params: { id: '1' }, body: { name: ' ' } }; - const res = mockResponse(); - - await expect(routeHandlers['patch:/:id'](req, res)).rejects.toBeInstanceOf(ValidationError); + await api.patch(`${BASE}/1`).send({ name: ' ' }).expect(400); }); it('throws ValidationError when categoryIds is invalid', async () => { - const req = { params: { id: '1' }, body: { categoryIds: [1, 'bad-id'] } }; - const res = mockResponse(); - - await expect(routeHandlers['patch:/:id'](req, res)).rejects.toBeInstanceOf(ValidationError); + await api.patch(`${BASE}/1`).send({ categoryIds: [1, 'bad-id'] }).expect(400); }); it('throws NotFoundError when chart does not exist', async () => { savedChartsRepository.update.mockResolvedValue(null); - const req = { params: { id: '9' }, body: { name: 'Updated' } }; - const res = mockResponse(); - - await expect(routeHandlers['patch:/:id'](req, res)).rejects.toBeInstanceOf(NotFoundError); + const res = await api.patch(`${BASE}/9`).send({ name: 'Updated' }).expect(404); + expect(res.body).toEqual(errEnvelope({ code: 'NOT_FOUND' })); }); it('updates chart and normalizes categoryIds when provided', async () => { savedChartsRepository.update.mockResolvedValue({ id: 9, name: 'Updated' }); - const req = { params: { id: '9' }, body: { categoryIds: ['3', 4], chartType: 'bar' } }; - const res = mockResponse(); - - await routeHandlers['patch:/:id'](req, res); + const res = await api.patch(`${BASE}/9`).send({ categoryIds: ['3', 4], chartType: 'bar' }).expect(200); expect(savedChartsRepository.update).toHaveBeenCalledWith(9, { name: undefined, chartType: 'bar', categoryIds: [3, 4], }); - expect(res.json).toHaveBeenCalledWith({ ok: true, data: { id: 9, name: 'Updated' } }); + expect(res.body).toEqual(okEnvelope({ id: 9, name: 'Updated' })); }); it('normalizes tagIds when provided', async () => { savedChartsRepository.update.mockResolvedValue({ id: 9, name: 'Updated' }); - const req = { params: { id: '9' }, body: { tagIds: ['7', 8] } }; - const res = mockResponse(); - - await routeHandlers['patch:/:id'](req, res); + await api.patch(`${BASE}/9`).send({ tagIds: ['7', 8] }).expect(200); expect(savedChartsRepository.update).toHaveBeenCalledWith( 9, @@ -373,18 +296,13 @@ describe('Saved Charts Routes', () => { }); it('throws ValidationError when tagIds is invalid', async () => { - const req = { params: { id: '1' }, body: { tagIds: [1, 'bad-id'] } }; - const res = mockResponse(); - - await expect(routeHandlers['patch:/:id'](req, res)).rejects.toBeInstanceOf(ValidationError); + await api.patch(`${BASE}/1`).send({ tagIds: [1, 'bad-id'] }).expect(400); }); it('updates all-source flags when provided', async () => { savedChartsRepository.update.mockResolvedValue({ id: 9 }); - const req = { params: { id: '9' }, body: { allRecipients: true } }; - const res = mockResponse(); - await routeHandlers['patch:/:id'](req, res); + await api.patch(`${BASE}/9`).send({ allRecipients: true }).expect(200); expect(savedChartsRepository.update).toHaveBeenCalledWith( 9, @@ -393,17 +311,13 @@ describe('Saved Charts Routes', () => { }); it('throws ValidationError when an all-source flag is not a boolean', async () => { - const req = { params: { id: '1' }, body: { allCategories: 1 } }; - const res = mockResponse(); - - await expect(routeHandlers['patch:/:id'](req, res)).rejects.toBeInstanceOf(ValidationError); + await api.patch(`${BASE}/1`).send({ allCategories: 1 }).expect(400); }); it('trims the name before updating', async () => { savedChartsRepository.update.mockResolvedValue({ id: 9 }); - const req = { params: { id: '9' }, body: { name: ' Renamed ' } }; - await routeHandlers['patch:/:id'](req, mockResponse()); + await api.patch(`${BASE}/9`).send({ name: ' Renamed ' }).expect(200); expect(savedChartsRepository.update).toHaveBeenCalledWith( 9, @@ -412,27 +326,19 @@ describe('Saved Charts Routes', () => { }); it('rejects a non-string name', async () => { - const req = { params: { id: '1' }, body: { name: 42 } }; - - await expect(routeHandlers['patch:/:id'](req, mockResponse())).rejects.toBeInstanceOf(ValidationError); + await api.patch(`${BASE}/1`).send({ name: 42 }).expect(400); }); it('rejects an unknown chartVariant', async () => { - const req = { params: { id: '1' }, body: { chartVariant: 'wavy' } }; - - await expect(routeHandlers['patch:/:id'](req, mockResponse())).rejects.toBeInstanceOf(ValidationError); + await api.patch(`${BASE}/1`).send({ chartVariant: 'wavy' }).expect(400); }); it('rejects an unknown timeBucket', async () => { - const req = { params: { id: '1' }, body: { timeBucket: 'daily' } }; - - await expect(routeHandlers['patch:/:id'](req, mockResponse())).rejects.toBeInstanceOf(ValidationError); + await api.patch(`${BASE}/1`).send({ timeBucket: 'daily' }).expect(400); }); it('rejects an illegal chartType/chartVariant pair when both are provided', async () => { - const req = { params: { id: '1' }, body: { chartType: 'line', chartVariant: 'stacked' } }; - - await expect(routeHandlers['patch:/:id'](req, mockResponse())).rejects.toBeInstanceOf(ValidationError); + await api.patch(`${BASE}/1`).send({ chartType: 'line', chartVariant: 'stacked' }).expect(400); expect(savedChartsRepository.update).not.toHaveBeenCalled(); }); @@ -441,8 +347,7 @@ describe('Saved Charts Routes', () => { // a lone chartType change never consults the persisted variant. savedChartsRepository.update.mockResolvedValue({ id: 9 }); - const req = { params: { id: '9' }, body: { chartType: 'line' } }; - await routeHandlers['patch:/:id'](req, mockResponse()); + await api.patch(`${BASE}/9`).send({ chartType: 'line' }).expect(200); expect(savedChartsRepository.update).toHaveBeenCalledWith( 9, @@ -453,8 +358,7 @@ describe('Saved Charts Routes', () => { it('allows chartVariant alone without cross-checking the stored type', async () => { savedChartsRepository.update.mockResolvedValue({ id: 9 }); - const req = { params: { id: '9' }, body: { chartVariant: 'ranked' } }; - await routeHandlers['patch:/:id'](req, mockResponse()); + await api.patch(`${BASE}/9`).send({ chartVariant: 'ranked' }).expect(200); expect(savedChartsRepository.update).toHaveBeenCalledWith( 9, @@ -463,16 +367,13 @@ describe('Saved Charts Routes', () => { }); it('rejects an unparsable dateRangeEnd', async () => { - const req = { params: { id: '1' }, body: { dateRangeEnd: 'yesterdayish' } }; - - await expect(routeHandlers['patch:/:id'](req, mockResponse())).rejects.toBeInstanceOf(ValidationError); + await api.patch(`${BASE}/1`).send({ dateRangeEnd: 'yesterdayish' }).expect(400); }); it('maps an empty-string date to null (clear) on update', async () => { savedChartsRepository.update.mockResolvedValue({ id: 9 }); - const req = { params: { id: '9' }, body: { dateRangeEnd: '' } }; - await routeHandlers['patch:/:id'](req, mockResponse()); + await api.patch(`${BASE}/9`).send({ dateRangeEnd: '' }).expect(200); expect(savedChartsRepository.update).toHaveBeenCalledWith( 9, @@ -481,17 +382,13 @@ describe('Saved Charts Routes', () => { }); it('rejects a zero chart id', async () => { - const req = { params: { id: '0' }, body: { name: 'x' } }; - - await expect(routeHandlers['patch:/:id'](req, mockResponse())).rejects.toBeInstanceOf(ValidationError); + await api.patch(`${BASE}/0`).send({ name: 'x' }).expect(400); }); it('passes null through to CLEAR a date range (was silently coerced to undefined)', async () => { savedChartsRepository.update.mockResolvedValue({ id: 9 }); - const req = { params: { id: '9' }, body: { dateRangeStart: null } }; - const res = mockResponse(); - await routeHandlers['patch:/:id'](req, res); + await api.patch(`${BASE}/9`).send({ dateRangeStart: null }).expect(200); expect(savedChartsRepository.update).toHaveBeenCalledWith( 9, @@ -502,43 +399,26 @@ describe('Saved Charts Routes', () => { describe('DELETE /:id', () => { it('throws ValidationError for invalid chart id', async () => { - const req = { params: { id: 'bad' } }; - const res = mockResponse(); - - await expect(routeHandlers['delete:/:id'](req, res)).rejects.toBeInstanceOf(ValidationError); + await api.delete(`${BASE}/bad`).expect(400); }); it('throws ValidationError for a negative chart id (was accepted as -5)', async () => { - const req = { params: { id: '-5' } }; - const res = mockResponse(); - - await expect(routeHandlers['delete:/:id'](req, res)).rejects.toBeInstanceOf(ValidationError); + await api.delete(`${BASE}/-5`).expect(400); expect(savedChartsRepository.delete).not.toHaveBeenCalled(); }); it('throws NotFoundError when delete misses', async () => { savedChartsRepository.delete.mockResolvedValue(false); - const req = { params: { id: '8' } }; - const res = mockResponse(); - - await expect(routeHandlers['delete:/:id'](req, res)).rejects.toBeInstanceOf(NotFoundError); + await api.delete(`${BASE}/8`).expect(404); }); it('returns 204 when delete succeeds', async () => { savedChartsRepository.delete.mockResolvedValue(true); - const req = { params: { id: '8' } }; - const res = mockResponse(); + const res = await api.delete(`${BASE}/8`).expect(204); - await routeHandlers['delete:/:id'](req, res); - - expect(res.status).toHaveBeenCalledWith(204); - expect(res.send).toHaveBeenCalled(); + expect(res.text).toBe(''); }); }); }); - -function mockResponse() { - return createMockResponse(); -} diff --git a/apps/node-backend/tests/routes/settings.test.js b/apps/node-backend/tests/routes/settings.test.js index 0d83c48f..342966d0 100644 --- a/apps/node-backend/tests/routes/settings.test.js +++ b/apps/node-backend/tests/routes/settings.test.js @@ -1,14 +1,16 @@ +/** + * Settings route tests. + * + * Runs against the REAL router mounted on a throwaway Express app (see + * tests/helpers/routeApp.js). + */ import { beforeEach, describe, expect, it, vi } from 'vitest'; import { mockLogger } from '../helpers/mockLogger.js'; -import { createMockRouter, createMockResponse } from '../helpers/routeHarness.js'; - -const { router: mockRouter, handlers: routeHandlers } = createMockRouter(); - -vi.mock('express', () => ({ - default: { Router: () => mockRouter }, - Router: () => mockRouter, -})); +import { routeAgent, errEnvelope } from '../helpers/routeApp.js'; +// The route imports its repository through services/settingsService.js, which +// re-exports the default from this module — mocking the repository here +// intercepts that same binding. vi.mock('../../src/repositories/settingsRepository.js', () => ({ default: { getAll: vi.fn(), @@ -24,9 +26,11 @@ vi.mock('../../src/config/logger.js', () => ({ })); import settingsRepository from '../../src/repositories/settingsRepository.js'; -import { ValidationError, NotFoundError } from '../../src/middleware/errorHandler.js'; -await import('../../src/routes/settings.js'); +const { default: settingsRouter } = await import('../../src/routes/settings.js'); + +const api = routeAgent(settingsRouter, { mountPath: '/api/settings' }); +const BASE = '/api/settings'; describe('Settings Routes', () => { beforeEach(() => { @@ -37,19 +41,16 @@ describe('Settings Routes', () => { it('returns all settings', async () => { settingsRepository.getAll.mockResolvedValue({ app_settings: { defaultCurrency: 'EUR' } }); - const req = { params: {}, query: {} }; - const res = mockResponse(); - await routeHandlers['get:/'](req, res); + const res = await api.get(BASE).expect(200); - expect(res.json).toHaveBeenCalledWith({ ok: true, data: { app_settings: { defaultCurrency: 'EUR' } } }); + expect(res.body.data).toEqual({ app_settings: { defaultCurrency: 'EUR' } }); }); - it('propagates error when fetching all settings fails', async () => { + it('answers a 500 when fetching all settings fails', async () => { settingsRepository.getAll.mockRejectedValue(new Error('boom')); - const req = { params: {}, query: {} }; - const res = mockResponse(); - await expect(routeHandlers['get:/'](req, res)).rejects.toThrow('boom'); + const res = await api.get(BASE).expect(500); + expect(res.body.error.message).toBe('boom'); }); }); @@ -57,31 +58,25 @@ describe('Settings Routes', () => { it('returns stored setting value when present', async () => { settingsRepository.get.mockResolvedValue({ defaultCurrency: 'USD' }); - const req = { params: { key: 'app_settings' } }; - const res = mockResponse(); - await routeHandlers['get:/:key'](req, res); + const res = await api.get(`${BASE}/app_settings`).expect(200); - expect(res.json).toHaveBeenCalledWith({ ok: true, data: { key: 'app_settings', value: { defaultCurrency: 'USD' } } }); + expect(res.body.data).toEqual({ key: 'app_settings', value: { defaultCurrency: 'USD' } }); }); it('returns default for known key when missing', async () => { settingsRepository.get.mockResolvedValue(null); - const req = { params: { key: 'onboarding_complete' } }; - const res = mockResponse(); - await routeHandlers['get:/:key'](req, res); + const res = await api.get(`${BASE}/onboarding_complete`).expect(200); - expect(res.json).toHaveBeenCalledWith({ ok: true, data: { key: 'onboarding_complete', value: false } }); + expect(res.body.data).toEqual({ key: 'onboarding_complete', value: false }); }); it('app_settings default mirrors the frontend store (no default-copy drift)', async () => { settingsRepository.get.mockResolvedValue(null); - const req = { params: { key: 'app_settings' } }; - const res = mockResponse(); - await routeHandlers['get:/:key'](req, res); + const res = await api.get(`${BASE}/app_settings`).expect(200); - const { value } = res.json.mock.calls[0][0].data; + const { value } = res.body.data; // Keys that had drifted from DEFAULT_APP_SETTINGS. expect(value).toMatchObject({ costBasisMethod: 'weighted_avg', @@ -96,134 +91,102 @@ describe('Settings Routes', () => { it('dashboard_settings default includes exclusionScope', async () => { settingsRepository.get.mockResolvedValue(null); - const req = { params: { key: 'dashboard_settings' } }; - const res = mockResponse(); - await routeHandlers['get:/:key'](req, res); + const res = await api.get(`${BASE}/dashboard_settings`).expect(200); - const { value } = res.json.mock.calls[0][0].data; - expect(value.exclusionScope).toBe('everywhere'); + expect(res.body.data.value.exclusionScope).toBe('everywhere'); }); it('returns false default for includeTransfers when unset', async () => { // Missing from SETTING_DEFAULTS this GET 404'd until the first toggle. settingsRepository.get.mockResolvedValue(null); - const req = { params: { key: 'includeTransfers' } }; - const res = mockResponse(); - await routeHandlers['get:/:key'](req, res); + const res = await api.get(`${BASE}/includeTransfers`).expect(200); - expect(res.json).toHaveBeenCalledWith({ ok: true, data: { key: 'includeTransfers', value: false } }); + expect(res.body.data).toEqual({ key: 'includeTransfers', value: false }); }); - it('throws NotFoundError for unknown missing key', async () => { + it('returns a 404 NOT_FOUND envelope for unknown missing key', async () => { settingsRepository.get.mockResolvedValue(null); - const req = { params: { key: 'unknown_key' } }; - const res = mockResponse(); - await expect(routeHandlers['get:/:key'](req, res)).rejects.toBeInstanceOf(NotFoundError); + const res = await api.get(`${BASE}/unknown_key`).expect(404); + expect(res.body).toEqual(errEnvelope({ code: 'NOT_FOUND' })); }); - it('propagates error when fetching setting fails', async () => { + it('answers a 500 when fetching setting fails', async () => { settingsRepository.get.mockRejectedValue(new Error('boom')); - const req = { params: { key: 'app_settings' } }; - const res = mockResponse(); - await expect(routeHandlers['get:/:key'](req, res)).rejects.toThrow('boom'); + const res = await api.get(`${BASE}/app_settings`).expect(500); + expect(res.body.error.message).toBe('boom'); }); }); describe('PUT /:key', () => { - it('throws ValidationError when key length exceeds maximum', async () => { - const req = { params: { key: 'k'.repeat(101) }, body: { value: true } }; - const res = mockResponse(); - - await expect(routeHandlers['put:/:key'](req, res)).rejects.toBeInstanceOf(ValidationError); + it('returns a 400 VALIDATION_ERROR envelope when key length exceeds maximum', async () => { + const res = await api.put(`${BASE}/${'k'.repeat(101)}`).send({ value: true }).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); }); - it('throws ValidationError when value is missing from request body', async () => { - const req = { params: { key: 'dashboard_settings' }, body: {} }; - const res = mockResponse(); - - await expect(routeHandlers['put:/:key'](req, res)).rejects.toBeInstanceOf(ValidationError); + it('returns a 400 VALIDATION_ERROR envelope when value is missing from request body', async () => { + const res = await api.put(`${BASE}/dashboard_settings`).send({}).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); }); it.each(['__proto__', 'constructor', 'prototype'])( - 'throws ValidationError for forbidden key %s', + 'returns a 400 VALIDATION_ERROR envelope for forbidden key %s', async (key) => { - const req = { params: { key }, body: { value: { polluted: true } } }; - const res = mockResponse(); - - await expect(routeHandlers['put:/:key'](req, res)).rejects.toBeInstanceOf(ValidationError); + const res = await api.put(`${BASE}/${key}`).send({ value: { polluted: true } }).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); } ); - it('throws ValidationError (not TypeError) for dashboard_settings with value null', async () => { + it('returns a 400 VALIDATION_ERROR (not a 500) for dashboard_settings with value null', async () => { // typeof null === 'object' — a missing null check made this a 500. - const req = { params: { key: 'dashboard_settings' }, body: { value: null } }; - const res = mockResponse(); - - await expect(routeHandlers['put:/:key'](req, res)).rejects.toBeInstanceOf(ValidationError); + const res = await api.put(`${BASE}/dashboard_settings`).send({ value: null }).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); }); - it('throws ValidationError for dashboard_settings with invalid exclusionScope', async () => { - const req = { - params: { key: 'dashboard_settings' }, - body: { value: { exclusionScope: 'invalid-scope' } }, - }; - const res = mockResponse(); - - await expect(routeHandlers['put:/:key'](req, res)).rejects.toBeInstanceOf(ValidationError); + it('returns a 400 VALIDATION_ERROR envelope for dashboard_settings with invalid exclusionScope', async () => { + const res = await api.put(`${BASE}/dashboard_settings`) + .send({ value: { exclusionScope: 'invalid-scope' } }) + .expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); }); - it('throws ValidationError for dashboard_settings when excludedCategoryIds contains invalid value', async () => { - const req = { - params: { key: 'dashboard_settings' }, - body: { value: { excludedCategoryIds: [1, 'abc'] } }, - }; - const res = mockResponse(); - - await expect(routeHandlers['put:/:key'](req, res)).rejects.toBeInstanceOf(ValidationError); + it('returns a 400 VALIDATION_ERROR envelope for dashboard_settings when excludedCategoryIds contains invalid value', async () => { + const res = await api.put(`${BASE}/dashboard_settings`) + .send({ value: { excludedCategoryIds: [1, 'abc'] } }) + .expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); }); it('saves setting when payload is valid', async () => { settingsRepository.set.mockResolvedValue({ key: 'theme_settings', value: { theme: 'dark' } }); - const req = { params: { key: 'theme_settings' }, body: { value: { theme: 'dark' } } }; - const res = mockResponse(); - await routeHandlers['put:/:key'](req, res); + const res = await api.put(`${BASE}/theme_settings`).send({ value: { theme: 'dark' } }).expect(200); expect(settingsRepository.set).toHaveBeenCalledWith('theme_settings', { theme: 'dark' }); - expect(res.json).toHaveBeenCalledWith({ ok: true, data: { key: 'theme_settings', value: { theme: 'dark' } } }); + expect(res.body.data).toEqual({ key: 'theme_settings', value: { theme: 'dark' } }); }); - it('throws ValidationError for theme_settings with unknown variant', async () => { - const req = { - params: { key: 'theme_settings' }, - body: { value: { variant: 'matrix-green' } }, - }; - const res = mockResponse(); - - await expect(routeHandlers['put:/:key'](req, res)).rejects.toBeInstanceOf(ValidationError); + it('returns a 400 VALIDATION_ERROR envelope for theme_settings with unknown variant', async () => { + const res = await api.put(`${BASE}/theme_settings`) + .send({ value: { variant: 'matrix-green' } }) + .expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); }); - it('throws ValidationError for theme_settings with unknown mode', async () => { - const req = { - params: { key: 'theme_settings' }, - body: { value: { mode: 'sepia' } }, - }; - const res = mockResponse(); - - await expect(routeHandlers['put:/:key'](req, res)).rejects.toBeInstanceOf(ValidationError); + it('returns a 400 VALIDATION_ERROR envelope for theme_settings with unknown mode', async () => { + const res = await api.put(`${BASE}/theme_settings`) + .send({ value: { mode: 'sepia' } }) + .expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); }); - it('throws ValidationError for theme_settings with malformed schedule time', async () => { - const req = { - params: { key: 'theme_settings' }, - body: { value: { schedule: { lightFrom: '25:00', darkFrom: '20:00' } } }, - }; - const res = mockResponse(); - - await expect(routeHandlers['put:/:key'](req, res)).rejects.toBeInstanceOf(ValidationError); + it('returns a 400 VALIDATION_ERROR envelope for theme_settings with malformed schedule time', async () => { + const res = await api.put(`${BASE}/theme_settings`) + .send({ value: { schedule: { lightFrom: '25:00', darkFrom: '20:00' } } }) + .expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); }); it('accepts theme_settings with known variant, mode, and schedule', async () => { @@ -232,14 +195,9 @@ describe('Settings Routes', () => { value: { mode: 'schedule', schedule: { lightFrom: '07:00', darkFrom: '20:00' }, variant: 'dracula' }, }); - const req = { - params: { key: 'theme_settings' }, - body: { - value: { mode: 'schedule', schedule: { lightFrom: '07:00', darkFrom: '20:00' }, variant: 'dracula' }, - }, - }; - const res = mockResponse(); - await routeHandlers['put:/:key'](req, res); + await api.put(`${BASE}/theme_settings`) + .send({ value: { mode: 'schedule', schedule: { lightFrom: '07:00', darkFrom: '20:00' }, variant: 'dracula' } }) + .expect(200); expect(settingsRepository.set).toHaveBeenCalledWith('theme_settings', { mode: 'schedule', @@ -249,166 +207,125 @@ describe('Settings Routes', () => { }); it('rejects an unknown setting key with a 400 naming the known keys', async () => { - const req = { params: { key: 'totally_unknown_key' }, body: { value: { any: 'json' } } }; - const res = mockResponse(); - - await expect(routeHandlers['put:/:key'](req, res)).rejects.toBeInstanceOf(ValidationError); - await expect(routeHandlers['put:/:key'](req, res)).rejects.toThrow(/Unknown setting key 'totally_unknown_key'.*Known keys:/); + const res = await api.put(`${BASE}/totally_unknown_key`).send({ value: { any: 'json' } }).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); + expect(res.body.error.message).toMatch(/Unknown setting key 'totally_unknown_key'.*Known keys:/); expect(settingsRepository.set).not.toHaveBeenCalled(); }); it('accepts dismissed_recurring_patterns as an array (RecurringDetectionPanel payload)', async () => { settingsRepository.set.mockResolvedValue({ key: 'dismissed_recurring_patterns', value: [3, 7] }); - const req = { params: { key: 'dismissed_recurring_patterns' }, body: { value: [3, 7] } }; - const res = mockResponse(); - await routeHandlers['put:/:key'](req, res); + await api.put(`${BASE}/dismissed_recurring_patterns`).send({ value: [3, 7] }).expect(200); expect(settingsRepository.set).toHaveBeenCalledWith('dismissed_recurring_patterns', [3, 7]); }); it('rejects a non-array dismissed_recurring_patterns', async () => { - const req = { params: { key: 'dismissed_recurring_patterns' }, body: { value: 'weekly' } }; - const res = mockResponse(); - - await expect(routeHandlers['put:/:key'](req, res)).rejects.toBeInstanceOf(ValidationError); + const res = await api.put(`${BASE}/dismissed_recurring_patterns`).send({ value: 'weekly' }).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); }); it('accepts a portfolio_tax_adjustments_v1 entry map (usePortfolioTaxAdjustments payload)', async () => { const value = { '2026:4': { taxes: 12.5, fees: 3 } }; settingsRepository.set.mockResolvedValue({ key: 'portfolio_tax_adjustments_v1', value }); - const req = { params: { key: 'portfolio_tax_adjustments_v1' }, body: { value } }; - const res = mockResponse(); - await routeHandlers['put:/:key'](req, res); + await api.put(`${BASE}/portfolio_tax_adjustments_v1`).send({ value }).expect(200); expect(settingsRepository.set).toHaveBeenCalledWith('portfolio_tax_adjustments_v1', value); }); - it('propagates error when single setting save fails', async () => { + it('answers a 500 when single setting save fails', async () => { settingsRepository.set.mockRejectedValue(new Error('boom')); - const req = { params: { key: 'theme_settings' }, body: { value: { theme: 'dark' } } }; - const res = mockResponse(); - await expect(routeHandlers['put:/:key'](req, res)).rejects.toThrow('boom'); + const res = await api.put(`${BASE}/theme_settings`).send({ value: { theme: 'dark' } }).expect(500); + expect(res.body.error.message).toBe('boom'); }); }); describe('PUT /', () => { - it('throws ValidationError when body is an array', async () => { - const req = { body: [] }; - const res = mockResponse(); - - await expect(routeHandlers['put:/'](req, res)).rejects.toBeInstanceOf(ValidationError); + it('returns a 400 VALIDATION_ERROR envelope when body is an array', async () => { + const res = await api.put(BASE).send([]).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); }); - it('throws ValidationError when body is not an object', async () => { - const req = { body: 'invalid' }; - const res = mockResponse(); - - await expect(routeHandlers['put:/'](req, res)).rejects.toBeInstanceOf(ValidationError); + it('returns a 400 VALIDATION_ERROR envelope when body is not an object', async () => { + const res = await api.put(BASE).set('Content-Type', 'application/json').send('"invalid"').expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); }); - it('throws ValidationError when a key exceeds max length', async () => { + it('returns a 400 VALIDATION_ERROR envelope when a key exceeds max length', async () => { const longKey = 'x'.repeat(101); - const req = { body: { [longKey]: true } }; - const res = mockResponse(); - - await expect(routeHandlers['put:/'](req, res)).rejects.toBeInstanceOf(ValidationError); + const res = await api.put(BASE).send({ [longKey]: true }).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); }); - it('throws ValidationError when dashboard_settings payload is not an object', async () => { - const req = { body: { dashboard_settings: 'invalid' } }; - const res = mockResponse(); - - await expect(routeHandlers['put:/'](req, res)).rejects.toBeInstanceOf(ValidationError); + it('returns a 400 VALIDATION_ERROR envelope when dashboard_settings payload is not an object', async () => { + const res = await api.put(BASE).send({ dashboard_settings: 'invalid' }).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); }); it('bulk saves settings when payload is valid', async () => { settingsRepository.setMany.mockResolvedValue(undefined); - const req = { - body: { - onboarding_complete: true, - dashboard_settings: { excludedCategoryIds: [1, 2] }, - }, - }; - const res = mockResponse(); - - await routeHandlers['put:/'](req, res); + const res = await api.put(BASE).send({ + onboarding_complete: true, + dashboard_settings: { excludedCategoryIds: [1, 2] }, + }).expect(200); expect(settingsRepository.setMany).toHaveBeenCalledWith({ onboarding_complete: true, dashboard_settings: { excludedCategoryIds: [1, 2] }, }); - expect(res.json).toHaveBeenCalledWith({ ok: true, data: { saved: 2 } }); + expect(res.body.data).toEqual({ saved: 2 }); }); - it('propagates error when bulk save fails', async () => { + it('answers a 500 when bulk save fails', async () => { settingsRepository.setMany.mockRejectedValue(new Error('boom')); - const req = { body: { onboarding_complete: true } }; - const res = mockResponse(); - await expect(routeHandlers['put:/'](req, res)).rejects.toThrow('boom'); + const res = await api.put(BASE).send({ onboarding_complete: true }).expect(500); + expect(res.body.error.message).toBe('boom'); }); it('rejects an unknown key via bulk (no unknown-key bypass)', async () => { - const req = { body: { onboarding_complete: true, mystery_key: { any: 'json' } } }; - const res = mockResponse(); - await expect(routeHandlers['put:/'](req, res)).rejects.toBeInstanceOf(ValidationError); + const res = await api.put(BASE).send({ onboarding_complete: true, mystery_key: { any: 'json' } }).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); expect(settingsRepository.setMany).not.toHaveBeenCalled(); }); it('rejects an invalid cost_basis_method via bulk (no validation bypass)', async () => { - const req = { body: { cost_basis_method: 'bogus' } }; - const res = mockResponse(); - await expect(routeHandlers['put:/'](req, res)).rejects.toBeInstanceOf(ValidationError); + const res = await api.put(BASE).send({ cost_basis_method: 'bogus' }).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); expect(settingsRepository.setMany).not.toHaveBeenCalled(); }); it('accepts a valid cost_basis_method via bulk', async () => { settingsRepository.setMany.mockResolvedValue(undefined); - const req = { body: { cost_basis_method: 'fifo' } }; - const res = mockResponse(); - await routeHandlers['put:/'](req, res); + await api.put(BASE).send({ cost_basis_method: 'fifo' }).expect(200); expect(settingsRepository.setMany).toHaveBeenCalledWith({ cost_basis_method: 'fifo' }); }); }); describe('DELETE /:key', () => { - it('throws NotFoundError when setting does not exist', async () => { + it('returns a 404 NOT_FOUND envelope when setting does not exist', async () => { settingsRepository.delete.mockResolvedValue(false); - const req = { params: { key: 'missing_key' } }; - const res = mockResponse(); - - await expect(routeHandlers['delete:/:key'](req, res)).rejects.toBeInstanceOf(NotFoundError); + const res = await api.delete(`${BASE}/missing_key`).expect(404); + expect(res.body).toEqual(errEnvelope({ code: 'NOT_FOUND' })); }); it('returns 204 with no body when setting exists', async () => { settingsRepository.delete.mockResolvedValue(true); - const req = { params: { key: 'theme_settings' } }; - const res = mockResponse(); - - await routeHandlers['delete:/:key'](req, res); - - expect(res.status).toHaveBeenCalledWith(204); - expect(res.send).toHaveBeenCalledWith(); - expect(res.json).not.toHaveBeenCalled(); + const res = await api.delete(`${BASE}/theme_settings`).expect(204); + expect(res.text).toBe(''); }); - it('propagates error when deleting setting fails', async () => { + it('answers a 500 when deleting setting fails', async () => { settingsRepository.delete.mockRejectedValue(new Error('boom')); - const req = { params: { key: 'theme_settings' } }; - const res = mockResponse(); - - await expect(routeHandlers['delete:/:key'](req, res)).rejects.toThrow('boom'); + const res = await api.delete(`${BASE}/theme_settings`).expect(500); + expect(res.body.error.message).toBe('boom'); }); }); }); - -function mockResponse() { - return createMockResponse(); -} diff --git a/apps/node-backend/tests/routes/splits.test.js b/apps/node-backend/tests/routes/splits.test.js index 2ebe9720..f0e46dd6 100644 --- a/apps/node-backend/tests/routes/splits.test.js +++ b/apps/node-backend/tests/routes/splits.test.js @@ -1,13 +1,16 @@ +/** + * Split route tests. + * + * Runs against the REAL router mounted on a throwaway Express app (see + * tests/helpers/routeApp.js). Mount is /api/splits (main.js:332, no + * per-mount `before` middleware). validateIdParam + * (routes/splits.js:168,178,193,247,263,271,285,299) now runs for real on + * every id-bearing route — every test here already used a valid numeric id, + * so nothing was fake-passing under the old bypass. + */ import { beforeEach, describe, expect, it, vi } from 'vitest'; import { mockLogger } from '../helpers/mockLogger.js'; -import { createMockRouter, createMockResponse } from '../helpers/routeHarness.js'; - -const { router: mockRouter, handlers: routeHandlers } = createMockRouter(); - -vi.mock('express', () => ({ - default: { Router: () => mockRouter }, - Router: () => mockRouter, -})); +import { routeAgent, okEnvelope, errEnvelope } from '../helpers/routeApp.js'; vi.mock('../../src/repositories/splitRepository.js', () => ({ default: { @@ -39,7 +42,11 @@ vi.mock('../../src/config/logger.js', () => ({ import splitRepository from '../../src/repositories/splitRepository.js'; import { ValidationError, NotFoundError } from '../../src/middleware/errorHandler.js'; -await import('../../src/routes/splits.js'); + +const { default: splitsRouter } = await import('../../src/routes/splits.js'); + +const BASE = '/api/splits'; +const api = routeAgent(splitsRouter, { mountPath: BASE }); describe('Splits Routes', () => { beforeEach(() => vi.clearAllMocks()); @@ -48,14 +55,9 @@ describe('Splits Routes', () => { it('returns owed summary items', async () => { splitRepository.getOwedSummary.mockResolvedValue([{ recipient_id: 2, amount: 12.5 }]); - const req = { params: {}, query: {}, get: () => null }; - const res = mockResponse(); - await routeHandlers['get:/owed'](req, res); + const res = await api.get(`${BASE}/owed`).expect(200); - expect(res.json).toHaveBeenCalledWith({ - ok: true, - data: { items: [{ recipient_id: 2, amount: 12.5 }], total: 1 }, - }); + expect(res.body).toEqual(okEnvelope({ items: [{ recipient_id: 2, amount: 12.5 }], total: 1 })); }); // The summary is derived in JS, so the route slices the computed array — @@ -65,22 +67,16 @@ describe('Splits Routes', () => { { recipient_id: 1 }, { recipient_id: 2 }, { recipient_id: 3 }, ]); - const req = { params: {}, query: { limit: '1', offset: '1' }, get: () => null }; - const res = mockResponse(); - await routeHandlers['get:/owed'](req, res); + const res = await api.get(`${BASE}/owed`).query({ limit: '1', offset: '1' }).expect(200); - expect(res.json).toHaveBeenCalledWith({ - ok: true, - data: { items: [{ recipient_id: 2 }], total: 3, limit: 1, offset: 1 }, - }); + expect(res.body).toEqual(okEnvelope({ items: [{ recipient_id: 2 }], total: 3, limit: 1, offset: 1 })); }); it('propagates error when owed summary fails', async () => { splitRepository.getOwedSummary.mockRejectedValue(new Error('boom')); - const req = { params: {}, query: {}, get: () => null }; - const res = mockResponse(); - await expect(routeHandlers['get:/owed'](req, res)).rejects.toThrow('boom'); + const res = await api.get(`${BASE}/owed`).expect(500); + expect(res.body).toEqual(errEnvelope({ code: 'INTERNAL_SERVER_ERROR', message: 'boom' })); }); }); @@ -88,38 +84,27 @@ describe('Splits Routes', () => { it('returns owed items by recipient', async () => { splitRepository.getOwedByRecipient.mockResolvedValue([{ id: 1, split_id: 4 }]); - const req = { params: { id: '7' }, get: () => null }; - const res = mockResponse(); - await routeHandlers['get:/owed/:id'](req, res); + const res = await api.get(`${BASE}/owed/7`).expect(200); expect(splitRepository.getOwedByRecipient).toHaveBeenCalledWith(7, {}); - expect(res.json).toHaveBeenCalledWith({ - ok: true, - data: { items: [{ id: 1, split_id: 4 }], total: 1 }, - }); + expect(res.body).toEqual(okEnvelope({ items: [{ id: 1, split_id: 4 }], total: 1 })); }); it('pages owed detail when limit/offset are supplied', async () => { splitRepository.getOwedByRecipient.mockResolvedValue([{ id: 2, split_id: 5 }]); splitRepository.countOwedByRecipient.mockResolvedValue(31); - const req = { params: { id: '7' }, query: { limit: '1', offset: '10' }, get: () => null }; - const res = mockResponse(); - await routeHandlers['get:/owed/:id'](req, res); + const res = await api.get(`${BASE}/owed/7`).query({ limit: '1', offset: '10' }).expect(200); expect(splitRepository.getOwedByRecipient).toHaveBeenCalledWith(7, { limit: 1, offset: 10 }); - expect(res.json).toHaveBeenCalledWith({ - ok: true, - data: { items: [{ id: 2, split_id: 5 }], total: 31, limit: 1, offset: 10 }, - }); + expect(res.body).toEqual(okEnvelope({ items: [{ id: 2, split_id: 5 }], total: 31, limit: 1, offset: 10 })); }); it('propagates error when owed by recipient fails', async () => { splitRepository.getOwedByRecipient.mockRejectedValue(new Error('boom')); - const req = { params: { id: '7' }, get: () => null }; - const res = mockResponse(); - await expect(routeHandlers['get:/owed/:id'](req, res)).rejects.toThrow('boom'); + const res = await api.get(`${BASE}/owed/7`).expect(500); + expect(res.body).toEqual(errEnvelope({ code: 'INTERNAL_SERVER_ERROR', message: 'boom' })); }); }); @@ -127,20 +112,16 @@ describe('Splits Routes', () => { it('throws ValidationError when split exceeds transaction total', async () => { splitRepository.createSplitAtomic.mockRejectedValue(new ValidationError('Split would exceed transaction total')); - const req = { body: { transaction_id: 1, recipient_id: 2, amount: 15 }, get: () => null }; - const res = mockResponse(); - await expect(routeHandlers['post:/'](req, res)).rejects.toBeInstanceOf(ValidationError); + const res = await api.post(`${BASE}/`).send({ transaction_id: 1, recipient_id: 2, amount: 15 }).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); }); it('creates split when amount fits remaining total', async () => { splitRepository.createSplitAtomic.mockResolvedValue({ id: 7, transaction_id: 1, recipient_id: 2, amount: 20 }); splitRepository.writeAudit.mockResolvedValue(); - const req = { body: { transaction_id: 1, recipient_id: 2, amount: 20 }, get: () => null }; - const res = mockResponse(); - await routeHandlers['post:/'](req, res); + await api.post(`${BASE}/`).send({ transaction_id: 1, recipient_id: 2, amount: 20 }).expect(201); - expect(res.status).toHaveBeenCalledWith(201); expect(splitRepository.createSplitAtomic).toHaveBeenCalledWith(expect.objectContaining({ amount: 20 })); }); @@ -151,15 +132,15 @@ describe('Splits Routes', () => { { transaction_id: 'abc', recipient_id: 2, amount: 5 }, { transaction_id: 1, recipient_id: 'abc', amount: 5 }, ]) { - await expect(routeHandlers['post:/']({ body, get: () => null }, mockResponse())) - .rejects.toBeInstanceOf(ValidationError); + const res = await api.post(`${BASE}/`).send(body).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); } expect(splitRepository.createSplitAtomic).not.toHaveBeenCalled(); }); it('throws ValidationError for a non-finite amount', async () => { - const req = { body: { transaction_id: 1, recipient_id: 2, amount: 'abc' }, get: () => null }; - await expect(routeHandlers['post:/'](req, mockResponse())).rejects.toBeInstanceOf(ValidationError); + const res = await api.post(`${BASE}/`).send({ transaction_id: 1, recipient_id: 2, amount: 'abc' }).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); expect(splitRepository.createSplitAtomic).not.toHaveBeenCalled(); }); @@ -167,8 +148,7 @@ describe('Splits Routes', () => { splitRepository.createSplitAtomic.mockResolvedValue({ id: 7, transaction_id: 1, recipient_id: 2, amount: 20 }); splitRepository.writeAudit.mockResolvedValue(); - const req = { body: { transaction_id: 1, recipient_id: 2, amount: '20' }, get: () => null }; - await routeHandlers['post:/'](req, mockResponse()); + await api.post(`${BASE}/`).send({ transaction_id: 1, recipient_id: 2, amount: '20' }).expect(201); expect(splitRepository.createSplitAtomic).toHaveBeenCalledWith(expect.objectContaining({ amount: '20' })); expect(splitRepository.writeAudit).toHaveBeenCalledWith( @@ -177,8 +157,8 @@ describe('Splits Routes', () => { }); it('throws ValidationError when a falsy recipient_id hits the required check', async () => { - const req = { body: { transaction_id: 1, recipient_id: 0, amount: 5 }, get: () => null }; - await expect(routeHandlers['post:/'](req, mockResponse())).rejects.toBeInstanceOf(ValidationError); + const res = await api.post(`${BASE}/`).send({ transaction_id: 1, recipient_id: 0, amount: 5 }).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); }); }); @@ -186,62 +166,46 @@ describe('Splits Routes', () => { it('throws NotFoundError when transaction does not exist', async () => { splitRepository.createSplitsBatchAtomic.mockRejectedValue(new NotFoundError('Transaction not found')); - const req = { - body: { - transaction_id: 1, - splits: [{ recipient_id: 2, amount: 10 }], - }, - get: () => null, - }; - const res = mockResponse(); - await expect(routeHandlers['post:/batch'](req, res)).rejects.toBeInstanceOf(NotFoundError); + const res = await api.post(`${BASE}/batch`).send({ + transaction_id: 1, + splits: [{ recipient_id: 2, amount: 10 }], + }).expect(404); + expect(res.body).toEqual(errEnvelope({ code: 'NOT_FOUND' })); }); it('throws ValidationError when cumulative amount exceeds transaction total', async () => { splitRepository.createSplitsBatchAtomic.mockRejectedValue(new ValidationError('Split would exceed transaction total')); - const req = { - body: { - transaction_id: 1, - splits: [ - { recipient_id: 2, amount: 20 }, - { recipient_id: 3, amount: 15 }, - ], - }, - get: () => null, - }; - const res = mockResponse(); - await expect(routeHandlers['post:/batch'](req, res)).rejects.toBeInstanceOf(ValidationError); + const res = await api.post(`${BASE}/batch`).send({ + transaction_id: 1, + splits: [ + { recipient_id: 2, amount: 20 }, + { recipient_id: 3, amount: 15 }, + ], + }).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); }); it('throws ValidationError for non-positive split amounts in batch', async () => { splitRepository.createSplitsBatchAtomic.mockRejectedValue(new ValidationError('Split amount must be a positive number')); - const req = { - body: { - transaction_id: 1, - splits: [{ recipient_id: 2, amount: 0 }], - }, - get: () => null, - }; - const res = mockResponse(); - await expect(routeHandlers['post:/batch'](req, res)).rejects.toBeInstanceOf(ValidationError); + const res = await api.post(`${BASE}/batch`).send({ + transaction_id: 1, + splits: [{ recipient_id: 2, amount: 0 }], + }).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); }); it('rejects a non-empty batch whose rows are all invalid with 400, not a 201 empty envelope', async () => { - const req = { - body: { - transaction_id: 1, - // Both rows fail row validation (missing recipient_id / amount). - splits: [ - { amount: 10 }, - { recipient_id: 3 }, - ], - }, - get: () => null, - }; - const res = mockResponse(); - await expect(routeHandlers['post:/batch'](req, res)).rejects.toBeInstanceOf(ValidationError); + const res = await api.post(`${BASE}/batch`).send({ + transaction_id: 1, + // Both rows fail row validation (missing recipient_id / amount). + splits: [ + { amount: 10 }, + { recipient_id: 3 }, + ], + }).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); expect(splitRepository.createSplitsBatchAtomic).not.toHaveBeenCalled(); }); @@ -249,18 +213,13 @@ describe('Splits Routes', () => { splitRepository.createSplitsBatchAtomic.mockResolvedValue([{ id: 1 }]); splitRepository.writeAudit.mockResolvedValue(); - const req = { - body: { - transaction_id: 1, - splits: [ - { recipient_id: 2, amount: 20, note: 'x' }, - { recipient_id: 4, amount: 5, note: 'y' }, - ], - }, - get: () => null, - }; - const res = mockResponse(); - await routeHandlers['post:/batch'](req, res); + const res = await api.post(`${BASE}/batch`).send({ + transaction_id: 1, + splits: [ + { recipient_id: 2, amount: 20, note: 'x' }, + { recipient_id: 4, amount: 5, note: 'y' }, + ], + }).expect(201); expect(splitRepository.createSplitsBatchAtomic).toHaveBeenCalledWith({ transaction_id: 1, @@ -269,60 +228,49 @@ describe('Splits Routes', () => { { recipient_id: 4, amount: 5, note: 'y' }, ], }); - expect(res.status).toHaveBeenCalledWith(201); - expect(res.json).toHaveBeenCalledWith({ - ok: true, - data: { items: [{ id: 1 }], total: 1 }, - }); + expect(res.body).toEqual(okEnvelope({ items: [{ id: 1 }], total: 1 })); }); // All-or-nothing (bulk-operation semantics): a batch mixing valid and // malformed rows used to commit the valid subset silently. It must now be // rejected wholesale, with the 400 naming each offending index. it('rejects the whole batch when one row of several is malformed, writing nothing', async () => { - const req = { - body: { - transaction_id: 1, - splits: [ - { recipient_id: 2, amount: 20, note: 'x' }, - { recipient_id: null, amount: 20, note: 'dropped before this fix' }, - { recipient_id: 3, amount: 5 }, - ], - }, - get: () => null, - }; - const res = mockResponse(); - await expect(routeHandlers['post:/batch'](req, res)).rejects.toThrow(/splits\[1\]/); - await expect(routeHandlers['post:/batch'](req, res)).rejects.toBeInstanceOf(ValidationError); + const res = await api.post(`${BASE}/batch`).send({ + transaction_id: 1, + splits: [ + { recipient_id: 2, amount: 20, note: 'x' }, + { recipient_id: null, amount: 20, note: 'dropped before this fix' }, + { recipient_id: 3, amount: 5 }, + ], + }).expect(400); + + expect(res.body.error.message).toMatch(/splits\[1\]/); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); expect(splitRepository.createSplitsBatchAtomic).not.toHaveBeenCalled(); expect(splitRepository.writeAudit).not.toHaveBeenCalled(); }); it('names every offending index when several rows are malformed', async () => { - const req = { - body: { - transaction_id: 1, - splits: [ - { recipient_id: 2, amount: 20 }, - { recipient_id: 'abc', amount: 5 }, - { recipient_id: 3, amount: 'not-a-number' }, - ], - }, - get: () => null, - }; - await expect(routeHandlers['post:/batch'](req, mockResponse())) - .rejects.toThrow(/splits\[1\].*recipient_id.*splits\[2\].*amount/s); + const res = await api.post(`${BASE}/batch`).send({ + transaction_id: 1, + splits: [ + { recipient_id: 2, amount: 20 }, + { recipient_id: 'abc', amount: 5 }, + { recipient_id: 3, amount: 'not-a-number' }, + ], + }).expect(400); + + expect(res.body.error.message).toMatch(/splits\[1\].*recipient_id.*splits\[2\].*amount/s); expect(splitRepository.createSplitsBatchAtomic).not.toHaveBeenCalled(); }); // Pins for the zod swap (ZOD-08): normalization keeps parseInt-style id // coercion ('12abc' → 12) and finite (even non-positive) amounts. it('throws ValidationError for a non-integer batch transaction_id', async () => { - const req = { - body: { transaction_id: 'abc', splits: [{ recipient_id: 2, amount: 10 }] }, - get: () => null, - }; - await expect(routeHandlers['post:/batch'](req, mockResponse())).rejects.toBeInstanceOf(ValidationError); + const res = await api.post(`${BASE}/batch`).send({ + transaction_id: 'abc', splits: [{ recipient_id: 2, amount: 10 }], + }).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); expect(splitRepository.createSplitsBatchAtomic).not.toHaveBeenCalled(); }); @@ -330,17 +278,13 @@ describe('Splits Routes', () => { splitRepository.createSplitsBatchAtomic.mockResolvedValue([{ id: 1 }]); splitRepository.writeAudit.mockResolvedValue(); - const req = { - body: { - transaction_id: 1, - splits: [ - { recipient_id: '12abc', amount: '5', note: 'n' }, - { recipient_id: 3, amount: 0 }, // finite non-positive amounts survive normalization - ], - }, - get: () => null, - }; - await routeHandlers['post:/batch'](req, mockResponse()); + await api.post(`${BASE}/batch`).send({ + transaction_id: 1, + splits: [ + { recipient_id: '12abc', amount: '5', note: 'n' }, + { recipient_id: 3, amount: 0 }, // finite non-positive amounts survive normalization + ], + }).expect(201); expect(splitRepository.createSplitsBatchAtomic).toHaveBeenCalledWith({ transaction_id: 1, @@ -352,15 +296,11 @@ describe('Splits Routes', () => { }); it('rejects the batch on null and non-object rows instead of dropping them', async () => { - const req = { - body: { - transaction_id: 1, - splits: [null, 'junk', { recipient_id: 2, amount: 10 }], - }, - get: () => null, - }; - await expect(routeHandlers['post:/batch'](req, mockResponse())) - .rejects.toBeInstanceOf(ValidationError); + const res = await api.post(`${BASE}/batch`).send({ + transaction_id: 1, + splits: [null, 'junk', { recipient_id: 2, amount: 10 }], + }).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); expect(splitRepository.createSplitsBatchAtomic).not.toHaveBeenCalled(); }); }); @@ -381,23 +321,19 @@ describe('Splits Routes', () => { }, ]); - const req = { params: { id: '7' }, get: () => null }; - const res = mockResponse(); - await routeHandlers['get:/owed/:id/export/csv'](req, res); + const res = await api.get(`${BASE}/owed/7/export/csv`).expect(200); expect(splitRepository.getOwedExportRowsByRecipient).toHaveBeenCalledWith(7); - expect(res.setHeader).toHaveBeenCalledWith('Content-Type', 'text/csv'); - const body = res.send.mock.calls[0][0]; - expect(body).toContain('Date,Bank Account,Recipient,Memo,Amount,Currency,Balance,Category,Comment'); - expect(body).toContain('2026-03-20,Main,Coffee Shop,Lunch,5,EUR,100,FOOD:LUNCH,shared'); + expect(res.headers['content-type']).toMatch(/text\/csv/); + expect(res.text).toContain('Date,Bank Account,Recipient,Memo,Amount,Currency,Balance,Category,Comment'); + expect(res.text).toContain('2026-03-20,Main,Coffee Shop,Lunch,5,EUR,100,FOOD:LUNCH,shared'); }); it('throws NotFoundError when recipient has no unsettled owed transactions', async () => { splitRepository.getOwedExportRowsByRecipient.mockResolvedValue([]); - const req = { params: { id: '7' }, get: () => null }; - const res = mockResponse(); - await expect(routeHandlers['get:/owed/:id/export/csv'](req, res)).rejects.toBeInstanceOf(NotFoundError); + const res = await api.get(`${BASE}/owed/7/export/csv`).expect(404); + expect(res.body).toEqual(errEnvelope({ code: 'NOT_FOUND' })); }); }); @@ -405,40 +341,29 @@ describe('Splits Routes', () => { it('returns splits for transaction', async () => { splitRepository.getSplitsByTransaction.mockResolvedValue([{ id: 8, transaction_id: 2 }]); - const req = { params: { id: '2' }, get: () => null }; - const res = mockResponse(); - await routeHandlers['get:/transaction/:id'](req, res); + const res = await api.get(`${BASE}/transaction/2`).expect(200); // No limit/offset on the request → the repository is left unbounded and // the response carries no limit/offset (the body IS the whole list). expect(splitRepository.getSplitsByTransaction).toHaveBeenCalledWith(2, {}); - expect(res.json).toHaveBeenCalledWith({ - ok: true, - data: { items: [{ id: 8, transaction_id: 2 }], total: 1 }, - }); + expect(res.body).toEqual(okEnvelope({ items: [{ id: 8, transaction_id: 2 }], total: 1 })); }); it('slices and reports the full total when limit/offset are supplied', async () => { splitRepository.getSplitsByTransaction.mockResolvedValue([{ id: 9, transaction_id: 2 }]); splitRepository.countSplitsByTransaction.mockResolvedValue(7); - const req = { params: { id: '2' }, query: { limit: '1', offset: '3' }, get: () => null }; - const res = mockResponse(); - await routeHandlers['get:/transaction/:id'](req, res); + const res = await api.get(`${BASE}/transaction/2`).query({ limit: '1', offset: '3' }).expect(200); expect(splitRepository.getSplitsByTransaction).toHaveBeenCalledWith(2, { limit: 1, offset: 3 }); - expect(res.json).toHaveBeenCalledWith({ - ok: true, - data: { items: [{ id: 9, transaction_id: 2 }], total: 7, limit: 1, offset: 3 }, - }); + expect(res.body).toEqual(okEnvelope({ items: [{ id: 9, transaction_id: 2 }], total: 7, limit: 1, offset: 3 })); }); it('propagates error when transaction splits lookup fails', async () => { splitRepository.getSplitsByTransaction.mockRejectedValue(new Error('boom')); - const req = { params: { id: '2' }, get: () => null }; - const res = mockResponse(); - await expect(routeHandlers['get:/transaction/:id'](req, res)).rejects.toThrow('boom'); + const res = await api.get(`${BASE}/transaction/2`).expect(500); + expect(res.body).toEqual(errEnvelope({ code: 'INTERNAL_SERVER_ERROR', message: 'boom' })); }); }); @@ -446,9 +371,8 @@ describe('Splits Routes', () => { it('throws NotFoundError when split does not exist', async () => { splitRepository.addPayment.mockRejectedValue(new NotFoundError('Split not found')); - const req = { params: { id: '5' }, body: { amount: 5 }, get: () => null }; - const res = mockResponse(); - await expect(routeHandlers['post:/:id/pay'](req, res)).rejects.toBeInstanceOf(NotFoundError); + const res = await api.post(`${BASE}/5/pay`).send({ amount: 5 }).expect(404); + expect(res.body).toEqual(errEnvelope({ code: 'NOT_FOUND' })); expect(splitRepository.addPayment).toHaveBeenCalled(); }); @@ -456,9 +380,8 @@ describe('Splits Routes', () => { splitRepository.getSplitById.mockResolvedValue({ id: 5, amount: 100 }); splitRepository.getAlreadyPaid.mockResolvedValue(0); - const req = { params: { id: '5' }, body: { amount: 0 }, get: () => null }; - const res = mockResponse(); - await expect(routeHandlers['post:/:id/pay'](req, res)).rejects.toBeInstanceOf(ValidationError); + const res = await api.post(`${BASE}/5/pay`).send({ amount: 0 }).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); expect(splitRepository.addPayment).not.toHaveBeenCalled(); }); @@ -467,9 +390,7 @@ describe('Splits Routes', () => { splitRepository.getAlreadyPaid.mockResolvedValue(0); splitRepository.addPayment.mockResolvedValue({ id: 5, split_id: 7, amount: 12 }); - const req = { params: { id: '7' }, body: { amount: 12, note: 'partial', paid_at: '2026-03-20' }, get: () => null }; - const res = mockResponse(); - await routeHandlers['post:/:id/pay'](req, res); + const res = await api.post(`${BASE}/7/pay`).send({ amount: 12, note: 'partial', paid_at: '2026-03-20' }).expect(201); expect(splitRepository.addPayment).toHaveBeenCalledWith({ split_id: 7, @@ -478,15 +399,14 @@ describe('Splits Routes', () => { paid_at: '2026-03-20', actor: null, }); - expect(res.status).toHaveBeenCalledWith(201); - expect(res.json).toHaveBeenCalledWith({ ok: true, data: { id: 5, split_id: 7, amount: 12 } }); + expect(res.body).toEqual(okEnvelope({ id: 5, split_id: 7, amount: 12 })); }); // Pins for the zod swap (ZOD-08): positive-finite check, raw forwarding. it('throws ValidationError for non-numeric or negative payment amounts', async () => { for (const amount of ['abc', -5, undefined]) { - const req = { params: { id: '5' }, body: { amount }, get: () => null }; - await expect(routeHandlers['post:/:id/pay'](req, mockResponse())).rejects.toBeInstanceOf(ValidationError); + const res = await api.post(`${BASE}/5/pay`).send({ amount }).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); } expect(splitRepository.addPayment).not.toHaveBeenCalled(); }); @@ -494,8 +414,7 @@ describe('Splits Routes', () => { it('forwards a numeric-string payment amount raw to the repo', async () => { splitRepository.addPayment.mockResolvedValue({ id: 5, split_id: 7, amount: 12 }); - const req = { params: { id: '7' }, body: { amount: '12' }, get: () => null }; - await routeHandlers['post:/:id/pay'](req, mockResponse()); + await api.post(`${BASE}/7/pay`).send({ amount: '12' }).expect(201); expect(splitRepository.addPayment).toHaveBeenCalledWith( expect.objectContaining({ amount: '12' }), @@ -507,9 +426,8 @@ describe('Splits Routes', () => { splitRepository.getAlreadyPaid.mockResolvedValue(0); splitRepository.addPayment.mockRejectedValue(new Error('boom')); - const req = { params: { id: '7' }, body: { amount: 12 }, get: () => null }; - const res = mockResponse(); - await expect(routeHandlers['post:/:id/pay'](req, res)).rejects.toThrow('boom'); + const res = await api.post(`${BASE}/7/pay`).send({ amount: 12 }).expect(500); + expect(res.body).toEqual(errEnvelope({ code: 'INTERNAL_SERVER_ERROR', message: 'boom' })); }); }); @@ -517,38 +435,27 @@ describe('Splits Routes', () => { it('returns split payments', async () => { splitRepository.getPayments.mockResolvedValue([{ id: 3, split_id: 7, amount: 6 }]); - const req = { params: { id: '7' }, get: () => null }; - const res = mockResponse(); - await routeHandlers['get:/:id/payments'](req, res); + const res = await api.get(`${BASE}/7/payments`).expect(200); expect(splitRepository.getPayments).toHaveBeenCalledWith(7, {}); - expect(res.json).toHaveBeenCalledWith({ - ok: true, - data: { items: [{ id: 3, split_id: 7, amount: 6 }], total: 1 }, - }); + expect(res.body).toEqual(okEnvelope({ items: [{ id: 3, split_id: 7, amount: 6 }], total: 1 })); }); it('pages payments when limit/offset are supplied', async () => { splitRepository.getPayments.mockResolvedValue([{ id: 4, split_id: 7, amount: 2 }]); splitRepository.countPayments.mockResolvedValue(12); - const req = { params: { id: '7' }, query: { limit: '1', offset: '1' }, get: () => null }; - const res = mockResponse(); - await routeHandlers['get:/:id/payments'](req, res); + const res = await api.get(`${BASE}/7/payments`).query({ limit: '1', offset: '1' }).expect(200); expect(splitRepository.getPayments).toHaveBeenCalledWith(7, { limit: 1, offset: 1 }); - expect(res.json).toHaveBeenCalledWith({ - ok: true, - data: { items: [{ id: 4, split_id: 7, amount: 2 }], total: 12, limit: 1, offset: 1 }, - }); + expect(res.body).toEqual(okEnvelope({ items: [{ id: 4, split_id: 7, amount: 2 }], total: 12, limit: 1, offset: 1 })); }); it('propagates error when payments lookup fails', async () => { splitRepository.getPayments.mockRejectedValue(new Error('boom')); - const req = { params: { id: '7' }, get: () => null }; - const res = mockResponse(); - await expect(routeHandlers['get:/:id/payments'](req, res)).rejects.toThrow('boom'); + const res = await api.get(`${BASE}/7/payments`).expect(500); + expect(res.body).toEqual(errEnvelope({ code: 'INTERNAL_SERVER_ERROR', message: 'boom' })); }); }); @@ -556,28 +463,24 @@ describe('Splits Routes', () => { it('throws NotFoundError when split is not found', async () => { splitRepository.settleSplit.mockResolvedValue(null); - const req = { params: { id: '9' }, get: () => null }; - const res = mockResponse(); - await expect(routeHandlers['post:/:id/settle'](req, res)).rejects.toBeInstanceOf(NotFoundError); + const res = await api.post(`${BASE}/9/settle`).expect(404); + expect(res.body).toEqual(errEnvelope({ code: 'NOT_FOUND' })); }); it('returns settled split when found', async () => { splitRepository.settleSplit.mockResolvedValue({ id: 9, settled: true }); splitRepository.writeAudit.mockResolvedValue(); - const req = { params: { id: '9' }, get: () => null }; - const res = mockResponse(); - await routeHandlers['post:/:id/settle'](req, res); + const res = await api.post(`${BASE}/9/settle`).expect(200); - expect(res.json).toHaveBeenCalledWith({ ok: true, data: { id: 9, settled: true } }); + expect(res.body).toEqual(okEnvelope({ id: 9, settled: true })); }); it('propagates error when settle fails', async () => { splitRepository.settleSplit.mockRejectedValue(new Error('boom')); - const req = { params: { id: '9' }, get: () => null }; - const res = mockResponse(); - await expect(routeHandlers['post:/:id/settle'](req, res)).rejects.toThrow('boom'); + const res = await api.post(`${BASE}/9/settle`).expect(500); + expect(res.body).toEqual(errEnvelope({ code: 'INTERNAL_SERVER_ERROR', message: 'boom' })); }); }); @@ -586,20 +489,17 @@ describe('Splits Routes', () => { splitRepository.settleAllByRecipient.mockResolvedValue({ settled_count: 2 }); splitRepository.writeAudit.mockResolvedValue(); - const req = { params: { id: '12' }, get: () => null }; - const res = mockResponse(); - await routeHandlers['post:/owed/:id/settle-all'](req, res); + const res = await api.post(`${BASE}/owed/12/settle-all`).expect(200); expect(splitRepository.settleAllByRecipient).toHaveBeenCalledWith(12); - expect(res.json).toHaveBeenCalledWith({ ok: true, data: { settled_count: 2 } }); + expect(res.body).toEqual(okEnvelope({ settled_count: 2 })); }); it('propagates error when settle-all fails', async () => { splitRepository.settleAllByRecipient.mockRejectedValue(new Error('boom')); - const req = { params: { id: '12' }, get: () => null }; - const res = mockResponse(); - await expect(routeHandlers['post:/owed/:id/settle-all'](req, res)).rejects.toThrow('boom'); + const res = await api.post(`${BASE}/owed/12/settle-all`).expect(500); + expect(res.body).toEqual(errEnvelope({ code: 'INTERNAL_SERVER_ERROR', message: 'boom' })); }); }); @@ -607,9 +507,8 @@ describe('Splits Routes', () => { it('throws NotFoundError when split does not exist', async () => { splitRepository.getSplitById.mockResolvedValue(null); - const req = { params: { id: '1' }, get: () => null }; - const res = mockResponse(); - await expect(routeHandlers['delete:/:id'](req, res)).rejects.toBeInstanceOf(NotFoundError); + const res = await api.delete(`${BASE}/1`).expect(404); + expect(res.body).toEqual(errEnvelope({ code: 'NOT_FOUND' })); expect(splitRepository.deleteSplit).not.toHaveBeenCalled(); }); @@ -617,9 +516,8 @@ describe('Splits Routes', () => { splitRepository.getSplitById.mockResolvedValue({ id: 1, transaction_id: 2, recipient_id: 3, amount: 10 }); splitRepository.deleteSplit.mockResolvedValue(false); - const req = { params: { id: '1' }, get: () => null }; - const res = mockResponse(); - await expect(routeHandlers['delete:/:id'](req, res)).rejects.toBeInstanceOf(NotFoundError); + const res = await api.delete(`${BASE}/1`).expect(404); + expect(res.body).toEqual(errEnvelope({ code: 'NOT_FOUND' })); }); it('returns 204 with no body when split is deleted', async () => { @@ -627,26 +525,16 @@ describe('Splits Routes', () => { splitRepository.deleteSplit.mockResolvedValue(true); splitRepository.writeAudit.mockResolvedValue(); - const req = { params: { id: '1' }, get: () => null }; - const res = mockResponse(); - await routeHandlers['delete:/:id'](req, res); - - expect(res.status).toHaveBeenCalledWith(204); - expect(res.send).toHaveBeenCalledWith(); - expect(res.json).not.toHaveBeenCalled(); + const res = await api.delete(`${BASE}/1`).expect(204); + expect(res.body).toEqual({}); }); it('propagates error when deleting split fails', async () => { splitRepository.getSplitById.mockResolvedValue({ id: 1, transaction_id: 2, recipient_id: 3, amount: 10 }); splitRepository.deleteSplit.mockRejectedValue(new Error('boom')); - const req = { params: { id: '1' }, get: () => null }; - const res = mockResponse(); - await expect(routeHandlers['delete:/:id'](req, res)).rejects.toThrow('boom'); + const res = await api.delete(`${BASE}/1`).expect(500); + expect(res.body).toEqual(errEnvelope({ code: 'INTERNAL_SERVER_ERROR', message: 'boom' })); }); }); }); - -function mockResponse() { - return createMockResponse({ setHeader: vi.fn() }); -} diff --git a/apps/node-backend/tests/routes/tags.test.js b/apps/node-backend/tests/routes/tags.test.js index 19dac246..b5dc6481 100644 --- a/apps/node-backend/tests/routes/tags.test.js +++ b/apps/node-backend/tests/routes/tags.test.js @@ -1,12 +1,11 @@ +/** + * Tag route tests. + * + * Runs against the REAL router mounted on a throwaway Express app (see + * tests/helpers/routeApp.js) — validateIdParam is no longer stubbed. + */ import { describe, it, expect, vi, beforeEach } from 'vitest'; -import { createMockRouter, createMockResponse } from '../helpers/routeHarness.js'; - -const { router: mockRouter, handlers: routeHandlers } = createMockRouter(); - -vi.mock('express', () => ({ - default: { Router: () => mockRouter }, - Router: () => mockRouter, -})); +import { routeAgent, errEnvelope } from '../helpers/routeApp.js'; vi.mock('../../src/repositories/tagRepository.js', () => ({ default: { @@ -20,13 +19,12 @@ vi.mock('../../src/repositories/tagRepository.js', () => ({ }, })); -vi.mock('../../src/middleware/validation.js', () => ({ - validateIdParam: vi.fn((_req, _res, next) => next()), -})); +import tagRepository from '../../src/repositories/tagRepository.js'; -await import('../../src/routes/tags.js'); +const { default: tagsRouter } = await import('../../src/routes/tags.js'); -import tagRepository from '../../src/repositories/tagRepository.js'; +const api = routeAgent(tagsRouter, { mountPath: '/api/tags' }); +const BASE = '/api/tags'; describe('GET /api/tags', () => { beforeEach(() => vi.clearAllMocks()); @@ -34,29 +32,23 @@ describe('GET /api/tags', () => { it('returns active tags by default', async () => { tagRepository.getAll.mockResolvedValue([{ id: 1, slug: 'rome-2020', is_active: true }]); tagRepository.getCount.mockResolvedValue(1); - const req = { query: {} }; - const res = mockResponse(); - await routeHandlers['get:/'](req, res); + const res = await api.get(BASE).expect(200); expect(tagRepository.getAll).toHaveBeenCalledWith(expect.objectContaining({ active: true })); - expect(res.json.mock.calls[0][0].data.total).toBe(1); - expect(res.json.mock.calls[0][0].data.items[0].slug).toBe('rome-2020'); + expect(res.body.data.total).toBe(1); + expect(res.body.data.items[0].slug).toBe('rome-2020'); }); it('passes active=false when ?active=false', async () => { tagRepository.getAll.mockResolvedValue([]); tagRepository.getCount.mockResolvedValue(0); - const req = { query: { active: 'false' } }; - const res = mockResponse(); - await routeHandlers['get:/'](req, res); + await api.get(`${BASE}?active=false`).expect(200); expect(tagRepository.getAll).toHaveBeenCalledWith(expect.objectContaining({ active: false })); }); it('passes active=null when ?active=all', async () => { tagRepository.getAll.mockResolvedValue([]); tagRepository.getCount.mockResolvedValue(0); - const req = { query: { active: 'all' } }; - const res = mockResponse(); - await routeHandlers['get:/'](req, res); + await api.get(`${BASE}?active=all`).expect(200); expect(tagRepository.getAll).toHaveBeenCalledWith(expect.objectContaining({ active: null })); }); }); @@ -70,13 +62,10 @@ describe('POST /api/tags', () => { tag: { id: 1, slug: 'rome-2020', color: null, is_active: true }, reactivated: false, }); - const req = { body: { slug: 'Rome 2020' } }; - const res = mockResponse(); - await routeHandlers['post:/'](req, res); - expect(res.status).toHaveBeenCalledWith(201); + const res = await api.post(BASE).send({ slug: 'Rome 2020' }).expect(201); expect(tagRepository.findOrCreateBySlug).toHaveBeenCalledWith('rome-2020', null); - expect(res.json.mock.calls[0][0].data.slug).toBe('rome-2020'); - expect(res.json.mock.calls[0][0].data.reactivated).toBe(false); + expect(res.body.data.slug).toBe('rome-2020'); + expect(res.body.data.reactivated).toBe(false); }); it('reactivates inactive tag, returns 201 with junction count', async () => { @@ -86,13 +75,9 @@ describe('POST /api/tags', () => { tag: { id: 5, slug: 'old', is_active: true }, reactivated: true, }); - const req = { body: { slug: 'old' } }; - const res = mockResponse(); - await routeHandlers['post:/'](req, res); - expect(res.status).toHaveBeenCalledWith(201); - const data = res.json.mock.calls[0][0].data; - expect(data.reactivated).toBe(true); - expect(data.reactivated_junction_count).toBe(3); + const res = await api.post(BASE).send({ slug: 'old' }).expect(201); + expect(res.body.data.reactivated).toBe(true); + expect(res.body.data.reactivated_junction_count).toBe(3); }); it('returns 200 when slug matches an active tag (conflict update path)', async () => { @@ -101,32 +86,23 @@ describe('POST /api/tags', () => { tag: { id: 6, slug: 'active', is_active: true }, reactivated: true, }); - const req = { body: { slug: 'active' } }; - const res = mockResponse(); - await routeHandlers['post:/'](req, res); - expect(res.status).toHaveBeenCalledWith(200); + await api.post(BASE).send({ slug: 'active' }).expect(200); }); - it('returns 400 when slug is missing', async () => { - const req = { body: {} }; - const res = mockResponse(); - await callHandler(routeHandlers['post:/'], req, res); - expect(res.status).toHaveBeenCalledWith(400); + it('returns a 400 VALIDATION_ERROR envelope when slug is missing', async () => { + const res = await api.post(BASE).send({}).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); }); - it('returns 400 when slug normalizes to empty string', async () => { - const req = { body: { slug: '!!!' } }; - const res = mockResponse(); - await callHandler(routeHandlers['post:/'], req, res); - expect(res.status).toHaveBeenCalledWith(400); + it('returns a 400 VALIDATION_ERROR envelope when slug normalizes to empty string', async () => { + const res = await api.post(BASE).send({ slug: '!!!' }).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); }); - it('returns 400 when color is not a string', async () => { + it('returns a 400 VALIDATION_ERROR envelope when color is not a string', async () => { tagRepository.getBySlug.mockResolvedValue(null); - const req = { body: { slug: 'valid', color: 123 } }; - const res = mockResponse(); - await callHandler(routeHandlers['post:/'], req, res); - expect(res.status).toHaveBeenCalledWith(400); + const res = await api.post(BASE).send({ slug: 'valid', color: 123 }).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); }); it('accepts null color', async () => { @@ -135,9 +111,7 @@ describe('POST /api/tags', () => { tag: { id: 1, slug: 'x', color: null, is_active: true }, reactivated: false, }); - const req = { body: { slug: 'x', color: null } }; - const res = mockResponse(); - await routeHandlers['post:/'](req, res); + await api.post(BASE).send({ slug: 'x', color: null }).expect(201); expect(tagRepository.findOrCreateBySlug).toHaveBeenCalledWith('x', null); }); }); @@ -147,32 +121,32 @@ describe('PATCH /api/tags/:id', () => { it('updates color and returns the tag', async () => { tagRepository.update.mockResolvedValue({ id: 1, slug: 'rome', color: '#f00', is_active: true }); - const req = { params: { id: '1' }, body: { color: '#f00' } }; - const res = mockResponse(); - await routeHandlers['patch:/:id'](req, res); - expect(res.json.mock.calls[0][0].data.color).toBe('#f00'); + const res = await api.patch(`${BASE}/1`).send({ color: '#f00' }).expect(200); + expect(res.body.data.color).toBe('#f00'); }); - it('returns 404 when tag not found', async () => { + it('returns a 404 NOT_FOUND envelope when tag not found', async () => { tagRepository.update.mockResolvedValue(null); - const req = { params: { id: '999' }, body: { color: '#f00' } }; - const res = mockResponse(); - await callHandler(routeHandlers['patch:/:id'], req, res); - expect(res.status).toHaveBeenCalledWith(404); + const res = await api.patch(`${BASE}/999`).send({ color: '#f00' }).expect(404); + expect(res.body).toEqual(errEnvelope({ code: 'NOT_FOUND' })); + }); + + it('returns a 400 VALIDATION_ERROR envelope when color is not a string', async () => { + const res = await api.patch(`${BASE}/1`).send({ color: 42 }).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); }); - it('returns 400 when color is not a string', async () => { - const req = { params: { id: '1' }, body: { color: 42 } }; - const res = mockResponse(); - await callHandler(routeHandlers['patch:/:id'], req, res); - expect(res.status).toHaveBeenCalledWith(400); + it('returns a 400 VALIDATION_ERROR envelope when is_active is not a boolean', async () => { + const res = await api.patch(`${BASE}/1`).send({ is_active: 'yes' }).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); }); - it('returns 400 when is_active is not a boolean', async () => { - const req = { params: { id: '1' }, body: { is_active: 'yes' } }; - const res = mockResponse(); - await callHandler(routeHandlers['patch:/:id'], req, res); - expect(res.status).toHaveBeenCalledWith(400); + it('rejects a non-integer :id via the real validateIdParam guard', async () => { + // Previously `vi.mock('.../middleware/validation.js')` replaced + // validateIdParam with a pass-through, so this guard was never tested. + const res = await api.patch(`${BASE}/abc`).send({ color: '#f00' }).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); + expect(tagRepository.update).not.toHaveBeenCalled(); }); }); @@ -183,33 +157,13 @@ describe('DELETE /api/tags/:id', () => { // docs/reference/code-patterns.md, "DELETE responses"). it('soft-deletes and returns 200 with the deactivated tag', async () => { tagRepository.softDelete.mockResolvedValue({ id: 1, slug: 'rome', is_active: false }); - const req = { params: { id: '1' } }; - const res = mockResponse(); - await routeHandlers['delete:/:id'](req, res); - const data = res.json.mock.calls[0][0].data; - expect(data).toMatchObject({ id: 1, slug: 'rome', is_active: false, links: [] }); - expect(res.status).not.toHaveBeenCalledWith(204); + const res = await api.delete(`${BASE}/1`).expect(200); + expect(res.body.data).toMatchObject({ id: 1, slug: 'rome', is_active: false, links: [] }); }); - it('returns 404 when tag not found', async () => { + it('returns a 404 NOT_FOUND envelope when tag not found', async () => { tagRepository.softDelete.mockResolvedValue(null); - const req = { params: { id: '999' } }; - const res = mockResponse(); - await callHandler(routeHandlers['delete:/:id'], req, res); - expect(res.status).toHaveBeenCalledWith(404); + const res = await api.delete(`${BASE}/999`).expect(404); + expect(res.body).toEqual(errEnvelope({ code: 'NOT_FOUND' })); }); }); - -function mockResponse() { - return createMockResponse({ headersSent: false }); -} - -async function callHandler(handler, req, res) { - try { - await handler(req, res); - } catch (err) { - const status = err.status ?? 500; - const code = err.code ?? 'INTERNAL_SERVER_ERROR'; - res.status(status).json({ ok: false, error: { code, message: err.message } }); - } -} diff --git a/apps/node-backend/tests/routes/transactions.test.js b/apps/node-backend/tests/routes/transactions.test.js index e627c451..de622638 100644 --- a/apps/node-backend/tests/routes/transactions.test.js +++ b/apps/node-backend/tests/routes/transactions.test.js @@ -1,19 +1,18 @@ /** * Transaction route tests. * Mirrors: apps/backend/tests/test_transactions.py + * + * Runs against the REAL router mounted on a throwaway Express app (see + * tests/helpers/routeApp.js) — so the per-route middleware chain + * (validateIdParam, the export rate limiters), Express query/body parsing, the + * ADR-026 envelope middleware and the centralized error handler are all on the + * tested path. Repositories/services are still mocked. */ import { describe, it, expect, vi, beforeEach } from 'vitest'; import { mockConnection } from '../helpers/repoMocks.js'; import { mockTransactionRepository, mockDeduplication, mockMaterializedViews, mockCurrencyConversion, mockAttachmentRecordService, mockAttachmentService } from '../helpers/transactionsRouteMocks.js'; import { mockLogger } from '../helpers/mockLogger.js'; -import { createMockRouter, createMockResponse } from '../helpers/routeHarness.js'; - -const { router: mockRouter, handlers: routeHandlers } = createMockRouter(); - -vi.mock('express', () => ({ - default: { Router: () => mockRouter }, - Router: () => mockRouter, -})); +import { routeAgent, okEnvelope, errEnvelope } from '../helpers/routeApp.js'; vi.mock('../../src/repositories/transactionRepository.js', () => mockTransactionRepository()); @@ -35,7 +34,7 @@ vi.mock('../../src/services/attachmentService.js', () => mockAttachmentService() vi.mock('../../src/services/transferReconciliationService.js', () => ({ scheduleReconcile: vi.fn(), - getTransferSuggestions: vi.fn(), + getTransferSuggestions: vi.fn(async () => []), markTransfer: vi.fn(), unmarkTransfer: vi.fn(), })); @@ -47,22 +46,38 @@ import { isManualDuplicate } from '../../src/services/deduplication.js'; import { convertRowsToEur } from '../../src/services/currency/currencyConversionService.js'; import { attachmentRepository } from '../../src/services/attachmentRecordService.js'; import { removeAttachmentFile } from '../../src/services/attachmentService.js'; -await import('../../src/routes/transactions.js'); + +const { default: transactionsRouter } = await import('../../src/routes/transactions.js'); + +const api = routeAgent(transactionsRouter, { mountPath: '/api/transactions' }); +// Same router behind an error handler in production mode (main.js:401 passes +// `settings.isProduction`), so the 5xx message-sanitization branch +// (errorHandler.js:234-235) is actually exercised rather than assumed. +const apiProd = routeAgent(transactionsRouter, { + mountPath: '/api/transactions', + isProduction: () => true, +}); + +const PROD_5XX_MESSAGE = 'An internal server error occurred. Please try again later.'; describe('Transaction Routes', () => { - beforeEach(() => vi.clearAllMocks()); + beforeEach(() => { + vi.clearAllMocks(); + // Re-arm the factory defaults that clearAllMocks wipes. + isManualDuplicate.mockResolvedValue({ isDuplicate: false }); + convertRowsToEur.mockImplementation(async (rows) => rows); + attachmentRepository.listPathsByTransactionIds.mockResolvedValue([]); + }); describe('GET /', () => { it('should return empty list', async () => { transactionRepository.getAllWithCount.mockResolvedValue({ rows: [], total: 0 }); - const req = { query: {} }; - const res = mockResponse(); - await routeHandlers['get:/'](req, res); + const res = await api.get('/api/transactions/').expect(200); - const result = res.json.mock.calls[0][0]; - expect(result.data.items).toEqual([]); - expect(result.data.total).toBe(0); + expect(res.body).toEqual(okEnvelope({ + items: [], total: 0, limit: expect.any(Number), offset: 0, links: [], + })); }); it('should return transactions with data', async () => { @@ -71,31 +86,24 @@ describe('Transaction Routes', () => { total: 1, }); - const req = { query: {} }; - const res = mockResponse(); - await routeHandlers['get:/'](req, res); + const res = await api.get('/api/transactions/').expect(200); - expect(res.json.mock.calls[0][0].data.total).toBe(1); + expect(res.body.data.total).toBe(1); }); it('should respect pagination', async () => { transactionRepository.getAllWithCount.mockResolvedValue({ rows: [], total: 10 }); - const req = { query: { limit: '2', offset: '3' } }; - const res = mockResponse(); - await routeHandlers['get:/'](req, res); + const res = await api.get('/api/transactions/?limit=2&offset=3').expect(200); - const result = res.json.mock.calls[0][0]; - expect(result.data.limit).toBe(2); - expect(result.data.offset).toBe(3); + expect(res.body.data.limit).toBe(2); + expect(res.body.data.offset).toBe(3); }); it('should handle uncategorised filter', async () => { transactionRepository.getUncategorisedWithCount.mockResolvedValue({ rows: [], total: 0 }); - const req = { query: { uncategorised: 'true' } }; - const res = mockResponse(); - await routeHandlers['get:/'](req, res); + await api.get('/api/transactions/?uncategorised=true').expect(200); expect(transactionRepository.getUncategorisedWithCount).toHaveBeenCalled(); }); @@ -106,12 +114,10 @@ describe('Transaction Routes', () => { total: 1, }); - const req = { query: { transaction_id: '42' } }; - const res = mockResponse(); - await routeHandlers['get:/'](req, res); + const res = await api.get('/api/transactions/?transaction_id=42').expect(200); expect(transactionRepository.getAllWithCount).toHaveBeenCalledWith(expect.objectContaining({ transactionId: 42 })); - expect(res.json.mock.calls[0][0].data.items).toHaveLength(1); + expect(res.body.data.items).toHaveLength(1); }); it('should normalize rows when normalize_to_eur is true', async () => { @@ -123,15 +129,15 @@ describe('Transaction Routes', () => { { id: 1, date: '2026-01-15', amount: '10', currency: 'USD', amount_eur: 9 }, ]); - const req = { query: { normalize_to_eur: 'true', target_currency: 'GBP' } }; - const res = mockResponse(); - await routeHandlers['get:/'](req, res); + const res = await api + .get('/api/transactions/?normalize_to_eur=true&target_currency=GBP') + .expect(200); expect(convertRowsToEur).toHaveBeenCalledWith( expect.arrayContaining([expect.objectContaining({ id: 1 })]), 'GBP' ); - expect(res.json).toHaveBeenCalled(); + expect(res.body.data.items[0].amount_eur).toBe(9); }); it('should thread include_balance to the repository and expose running_balance on rows (WP-B4)', async () => { @@ -143,14 +149,14 @@ describe('Transaction Routes', () => { total: 1, }); - const req = { query: { include_balance: 'true', account_id: '3' } }; - const res = mockResponse(); - await routeHandlers['get:/'](req, res); + const res = await api + .get('/api/transactions/?include_balance=true&account_id=3') + .expect(200); expect(transactionRepository.getAllWithCount).toHaveBeenCalledWith( expect.objectContaining({ includeBalance: true, accountId: 3 }), ); - expect(res.json.mock.calls[0][0].data.items[0].running_balance).toBe(974.5); + expect(res.body.data.items[0].running_balance).toBe(974.5); }); it('should omit the running_balance key entirely when include_balance is not set', async () => { @@ -159,14 +165,22 @@ describe('Transaction Routes', () => { total: 1, }); - const req = { query: {} }; - const res = mockResponse(); - await routeHandlers['get:/'](req, res); + const res = await api.get('/api/transactions/').expect(200); expect(transactionRepository.getAllWithCount).toHaveBeenCalledWith( expect.objectContaining({ includeBalance: false }), ); - expect('running_balance' in res.json.mock.calls[0][0].data.items[0]).toBe(false); + expect('running_balance' in res.body.data.items[0]).toBe(false); + }); + + it('rejects a malformed account_id through the real validation guard (400 envelope)', async () => { + const res = await api.get('/api/transactions/?account_id=abc').expect(400); + + expect(res.body).toEqual(errEnvelope({ + code: 'VALIDATION_ERROR', + message: 'account_id must be a positive integer', + })); + expect(transactionRepository.getAllWithCount).not.toHaveBeenCalled(); }); }); @@ -176,21 +190,31 @@ describe('Transaction Routes', () => { id: 1, date: '2026-01-15', amount: '50.00', bank_account: 'Chase', }); - const req = { params: { id: '1' } }; - const res = mockResponse(); - await routeHandlers['get:/:id'](req, res); + const res = await api.get('/api/transactions/1').expect(200); - expect(res.json).toHaveBeenCalled(); + expect(res.body.ok).toBe(true); + expect(res.body.data.id).toBe(1); }); it('should return 404 for non-existent', async () => { transactionRepository.getById.mockResolvedValue(null); - const req = { params: { id: '99999' } }; - const res = mockResponse(); - await callHandler(routeHandlers['get:/:id'], req, res); + const res = await api.get('/api/transactions/99999').expect(404); - expect(res.status).toHaveBeenCalledWith(404); + expect(res.body).toEqual(errEnvelope({ + code: 'NOT_FOUND', + message: 'Transaction with ID 99999 not found', + })); + }); + + it('rejects a non-integer :id via validateIdParam before the handler runs', async () => { + // validateIdParam is registered BEFORE the handler on this route + // (routes/transactions.js:523). The old mock-router harness kept only the + // last handler, so this guard was never on the tested path. + const res = await api.get('/api/transactions/abc').expect(400); + + expect(res.body.error.code).toBe('VALIDATION_ERROR'); + expect(transactionRepository.getById).not.toHaveBeenCalled(); }); }); @@ -212,13 +236,9 @@ describe('Transaction Routes', () => { ], }); - const req = { query: {} }; - const res = mockResponse(); - await routeHandlers['get:/export/csv'](req, res); + const res = await api.get('/api/transactions/export/csv').expect(200); - expect(res.write).toHaveBeenCalled(); - expect(res.end).toHaveBeenCalledTimes(1); - const csv = res.write.mock.calls.map(([chunk]) => chunk).join(''); + const csv = res.text; // Text columns are still guarded against spreadsheet formula injection. expect(csv).toContain(`'=HYPERLINK(""http://evil"")`); expect(csv).toContain("'+cmd"); @@ -233,14 +253,13 @@ describe('Transaction Routes', () => { it('should sanitize server error detail when export fails', async () => { dbQuery.mockRejectedValue(new Error('sensitive db failure')); - const req = { query: {} }; - const res = mockResponse(); - await callHandler(routeHandlers['get:/export/csv'], req, res); + const res = await apiProd.get('/api/transactions/export/csv').expect(500); - expect(res.status).toHaveBeenCalledWith(500); - expect(res.json).toHaveBeenCalledWith( - expect.objectContaining({ ok: false, error: expect.objectContaining({ message: expect.any(String) }) }) - ); + expect(res.body).toEqual(errEnvelope({ + code: 'INTERNAL_SERVER_ERROR', + message: PROD_5XX_MESSAGE, + })); + expect(res.text).not.toContain('sensitive db failure'); }); it('should apply transaction_type=expense filter to export query', async () => { @@ -248,9 +267,7 @@ describe('Transaction Routes', () => { .mockResolvedValueOnce({ rows: [{}] }) .mockResolvedValueOnce({ rows: [] }); - const req = { query: { transaction_type: 'expense' } }; - const res = mockResponse(); - await routeHandlers['get:/export/csv'](req, res); + await api.get('/api/transactions/export/csv?transaction_type=expense').expect(200); const probeSql = dbQuery.mock.calls[0][0]; expect(probeSql).toContain('t.amount < 0'); @@ -261,9 +278,7 @@ describe('Transaction Routes', () => { .mockResolvedValueOnce({ rows: [{}] }) .mockResolvedValueOnce({ rows: [] }); - const req = { query: { transaction_type: 'income' } }; - const res = mockResponse(); - await routeHandlers['get:/export/csv'](req, res); + await api.get('/api/transactions/export/csv?transaction_type=income').expect(200); const probeSql = dbQuery.mock.calls[0][0]; expect(probeSql).toContain('t.amount > 0'); @@ -274,9 +289,7 @@ describe('Transaction Routes', () => { .mockResolvedValueOnce({ rows: [{}] }) .mockResolvedValueOnce({ rows: [] }); - const req = { query: { recipient_id: '42' } }; - const res = mockResponse(); - await routeHandlers['get:/export/csv'](req, res); + await api.get('/api/transactions/export/csv?recipient_id=42').expect(200); const probeParams = dbQuery.mock.calls[0][1]; expect(probeParams).toContain(42); @@ -287,9 +300,7 @@ describe('Transaction Routes', () => { .mockResolvedValueOnce({ rows: [{}] }) .mockResolvedValueOnce({ rows: [] }); - const req = { query: { search: 'netflix' } }; - const res = mockResponse(); - await routeHandlers['get:/export/csv'](req, res); + await api.get('/api/transactions/export/csv?search=netflix').expect(200); const [probeSql, probeParams] = dbQuery.mock.calls[0]; expect(probeSql).toMatch(/t\.memo ILIKE/); @@ -301,9 +312,7 @@ describe('Transaction Routes', () => { .mockResolvedValueOnce({ rows: [{}] }) .mockResolvedValueOnce({ rows: [] }); - const req = { query: { transaction_id: '7' } }; - const res = mockResponse(); - await routeHandlers['get:/export/csv'](req, res); + await api.get('/api/transactions/export/csv?transaction_id=7').expect(200); const [probeSql, probeParams] = dbQuery.mock.calls[0]; expect(probeSql).toMatch(/t\.id = \$/); @@ -315,14 +324,23 @@ describe('Transaction Routes', () => { .mockResolvedValueOnce({ rows: [{}] }) .mockResolvedValueOnce({ rows: [] }); - const req = { query: { search: 'foo' } }; - const res = mockResponse(); - await routeHandlers['get:/export/csv'](req, res); + await api.get('/api/transactions/export/csv?search=foo').expect(200); const probeSql = dbQuery.mock.calls[0][0]; expect(probeSql).toContain('LEFT JOIN recipients r'); expect(probeSql).toContain('LEFT JOIN categories c'); }); + + it('sets the streamed CSV download headers', async () => { + dbQuery + .mockResolvedValueOnce({ rows: [{}] }) + .mockResolvedValueOnce({ rows: [] }); + + const res = await api.get('/api/transactions/export/csv').expect(200); + + expect(res.headers['content-type']).toMatch(/text\/csv/); + expect(res.headers['content-disposition']).toMatch(/filename=transactions_export.*\.csv/); + }); }); describe('GET /export/json', () => { @@ -345,16 +363,10 @@ describe('Transaction Routes', () => { .mockResolvedValueOnce({ rows: [{}] }) // probe .mockResolvedValueOnce({ rows: [sampleRow] }); // chunk (1 row < 1000 → break) - const req = { query: {} }; - const res = mockResponse(); - await routeHandlers['get:/export/json'](req, res); + const res = await api.get('/api/transactions/export/json').expect(200); - expect(res.setHeader).toHaveBeenCalledWith('Content-Type', 'application/x-ndjson'); - expect(res.setHeader).toHaveBeenCalledWith( - 'Content-Disposition', - expect.stringMatching(/filename=transactions_export.*\.ndjson/), - ); - expect(res.end).toHaveBeenCalledTimes(1); + expect(res.headers['content-type']).toMatch(/application\/x-ndjson/); + expect(res.headers['content-disposition']).toMatch(/filename=transactions_export.*\.ndjson/); }); it('should emit one JSON object per transaction line', async () => { @@ -363,12 +375,9 @@ describe('Transaction Routes', () => { .mockResolvedValueOnce({ rows: [{}] }) .mockResolvedValueOnce({ rows: [sampleRow, { ...sampleRow, id: 2, amount: '-5.00' }] }); - const req = { query: {} }; - const res = mockResponse(); - await routeHandlers['get:/export/json'](req, res); + const res = await api.get('/api/transactions/export/json').expect(200); - const written = res.write.mock.calls.map(([chunk]) => chunk).join(''); - const lines = written.trim().split('\n').filter(Boolean); + const lines = res.text.trim().split('\n').filter(Boolean); expect(lines).toHaveLength(2); const parsed = lines.map((l) => JSON.parse(l)); expect(parsed[0]).toMatchObject({ @@ -381,21 +390,19 @@ describe('Transaction Routes', () => { it('should return 404 when no transactions match filters', async () => { dbQuery.mockResolvedValueOnce({ rows: [] }); // probe - const req = { query: { start_date: '2099-01-01' } }; - const res = mockResponse(); - await callHandler(routeHandlers['get:/export/json'], req, res); + const res = await api + .get('/api/transactions/export/json?start_date=2099-01-01') + .expect(404); - expect(res.status).toHaveBeenCalledWith(404); + expect(res.body.error.code).toBe('NOT_FOUND'); }); it('should return 500 on unexpected error before headers sent', async () => { dbQuery.mockRejectedValueOnce(new Error('db error')); - const req = { query: {} }; - const res = mockResponse(); - await callHandler(routeHandlers['get:/export/json'], req, res); + const res = await api.get('/api/transactions/export/json').expect(500); - expect(res.status).toHaveBeenCalledWith(500); + expect(res.body.ok).toBe(false); }); it('should include all expected fields in output', async () => { @@ -404,12 +411,9 @@ describe('Transaction Routes', () => { .mockResolvedValueOnce({ rows: [{}] }) .mockResolvedValueOnce({ rows: [sampleRow] }); - const req = { query: {} }; - const res = mockResponse(); - await routeHandlers['get:/export/json'](req, res); + const res = await api.get('/api/transactions/export/json').expect(200); - const written = res.write.mock.calls.map(([chunk]) => chunk).join(''); - const obj = JSON.parse(written.trim().split('\n')[0]); + const obj = JSON.parse(res.text.trim().split('\n')[0]); expect(Object.keys(obj).sort()).toEqual( ['amount', 'balance', 'bank_account', 'category', 'comment', 'currency', 'date', 'id', 'memo', 'recipient', 'tags'].sort() ); @@ -422,78 +426,129 @@ describe('Transaction Routes', () => { id: 1, date: '2026-01-15', amount: '-50.00', bank_account: 'Chase', recipient_id: 1, }); - const req = { - body: { + const res = await api + .post('/api/transactions/') + .send({ transaction_date: '2026-01-15', bank_account: 'Chase', recipient_id: 1, amount: -50.00, memo: 'Test', - }, - }; - const res = mockResponse(); - await routeHandlers['post:/'](req, res); + }) + .expect(201); - expect(res.status).toHaveBeenCalledWith(201); + expect(res.body.ok).toBe(true); + expect(res.body.data.id).toBe(1); }); it('should return 400 for missing fields', async () => { - const req = { body: { bank_account: 'Chase', amount: -50.00 } }; - const res = mockResponse(); - await callHandler(routeHandlers['post:/'], req, res); + const res = await api + .post('/api/transactions/') + .send({ bank_account: 'Chase', amount: -50.00 }) + .expect(400); - expect(res.status).toHaveBeenCalledWith(400); + expect(res.body.error.code).toBe('VALIDATION_ERROR'); }); it('should return 400 for a zero amount', async () => { - const req = { - body: { + await api + .post('/api/transactions/') + .send({ transaction_date: '2026-01-15', bank_account: 'Chase', recipient_id: 1, amount: 0, - }, - }; - const res = mockResponse(); - await callHandler(routeHandlers['post:/'], req, res); + }) + .expect(400); - expect(res.status).toHaveBeenCalledWith(400); expect(transactionRepository.create).not.toHaveBeenCalled(); }); it('should return 400 for a non-numeric amount', async () => { - const req = { - body: { + await api + .post('/api/transactions/') + .send({ transaction_date: '2026-01-15', bank_account: 'Chase', recipient_id: 1, amount: 'abc', - }, - }; - const res = mockResponse(); - await callHandler(routeHandlers['post:/'], req, res); + }) + .expect(400); - expect(res.status).toHaveBeenCalledWith(400); expect(transactionRepository.create).not.toHaveBeenCalled(); }); it('should return 409 when manual duplicate is detected', async () => { isManualDuplicate.mockResolvedValue({ isDuplicate: true, existingTransactionId: 99 }); - const req = { - body: { + const res = await api + .post('/api/transactions/') + .send({ transaction_date: '2026-01-15', bank_account: 'Chase', recipient_id: 1, amount: -50, - }, - }; - const res = mockResponse(); - await callHandler(routeHandlers['post:/'], req, res); - - expect(res.status).toHaveBeenCalledWith(409); - expect(res.json).toHaveBeenCalledWith( - expect.objectContaining({ - ok: false, - error: expect.objectContaining({ - message: 'Duplicate transaction detected', - details: { existing_transaction_id: 99 }, - }), }) - ); + .expect(409); + + expect(res.body).toEqual(errEnvelope({ + code: 'CONFLICT', + message: 'Duplicate transaction detected', + details: { existing_transaction_id: 99 }, + })); + expect(transactionRepository.create).not.toHaveBeenCalled(); + }); + + it('a malformed JSON body yields a 400 with the parser reason, in both modes', async () => { + // body-parser raises a SyntaxError carrying `status = 400` and + // `type = 'entity.parse.failed'`. It is not an AppError, so it used to + // collapse to a 500 INTERNAL_SERVER_ERROR — a client typo reported as a + // server fault. errorHandler.js now forwards the 4xx and echoes the + // trusted body-parser message (THE RULE, errorHandler.js:91-110). + const res = await api + .post('/api/transactions/') + .set('Content-Type', 'application/json') + .send('{"amount": '); + + expect(res.status).toBe(400); + expect(res.body.error.code).toBe('VALIDATION_ERROR'); + expect(res.body.error.message).toMatch(/JSON/i); + + // Production must not sanitize it away: the 4xx never reaches the 5xx branch. + const prod = await apiProd + .post('/api/transactions/') + .set('Content-Type', 'application/json') + .send('{"amount": '); + expect(prod.status).toBe(400); + expect(prod.body.error.message).toBe(res.body.error.message); + expect(prod.body.error.message).not.toBe(PROD_5XX_MESSAGE); + + expect(transactionRepository.create).not.toHaveBeenCalled(); + }); + + it('an over-limit body yields a 413 whose reason survives production', async () => { + // body-parser's PayloadTooLargeError carries `status = 413` and + // `type = 'entity.too.large'`. It used to become a 500, and the 5xx + // sanitizer then replaced "request entity too large" with the generic + // message — a client posting an oversized bulk/import payload could not + // tell why it failed, and the typo counted against server-error monitoring. + const oversize = { memo: 'x'.repeat(1024 * 1024 + 100) }; + + const dev = await api.post('/api/transactions/').send(oversize); + expect(dev.status).toBe(413); + expect(dev.body.error.code).toBe('VALIDATION_ERROR'); + expect(dev.body.error.message).toBe('request entity too large'); + + const prod = await apiProd.post('/api/transactions/').send(oversize); + expect(prod.status).toBe(413); + expect(prod.body.error.message).toBe('request entity too large'); + expect(prod.body.error.message).not.toBe(PROD_5XX_MESSAGE); + }); + + it('the CSRF guard blocks a cross-site POST before the router runs', async () => { + const res = await api + .post('/api/transactions/') + .set('Sec-Fetch-Site', 'cross-site') + .send({ transaction_date: '2026-01-15', bank_account: 'Chase', recipient_id: 1, amount: -1 }) + .expect(403); + + expect(res.body).toEqual(errEnvelope({ + code: 'FORBIDDEN', + message: 'Cross-site request blocked', + })); expect(transactionRepository.create).not.toHaveBeenCalled(); }); }); @@ -504,59 +559,50 @@ describe('Transaction Routes', () => { id: 1, date: '2026-01-15', amount: '-75.00', bank_account: 'Chase', }); - const req = { params: { id: '1' }, body: { amount: -75.00 } }; - const res = mockResponse(); - await routeHandlers['patch:/:id'](req, res); + const res = await api + .patch('/api/transactions/1') + .send({ amount: -75.00 }) + .expect(200); - expect(res.json).toHaveBeenCalled(); + expect(res.body.ok).toBe(true); + expect(res.body.data.amount).toBe(-75); }); it('should return 404 for non-existent', async () => { transactionRepository.update.mockResolvedValue(null); - const req = { params: { id: '99999' }, body: { amount: -75.00 } }; - const res = mockResponse(); - await callHandler(routeHandlers['patch:/:id'], req, res); - - expect(res.status).toHaveBeenCalledWith(404); + await api.patch('/api/transactions/99999').send({ amount: -75.00 }).expect(404); }); it('should sanitize server error detail when patch fails', async () => { transactionRepository.update.mockRejectedValue(new Error('constraint: internal detail')); - const req = { params: { id: '1' }, body: { amount: -75.00 } }; - const res = mockResponse(); - await callHandler(routeHandlers['patch:/:id'], req, res); + const res = await apiProd.patch('/api/transactions/1').send({ amount: -75.00 }).expect(500); - expect(res.status).toHaveBeenCalledWith(500); - expect(res.json).toHaveBeenCalledWith( - expect.objectContaining({ ok: false, error: expect.objectContaining({ message: expect.any(String) }) }) - ); + expect(res.body).toEqual(errEnvelope({ + code: 'INTERNAL_SERVER_ERROR', + message: PROD_5XX_MESSAGE, + })); + expect(res.text).not.toContain('internal detail'); }); it('should return 400 when recipient_name cannot be resolved', async () => { dbQuery.mockResolvedValueOnce({ rows: [] }); - const req = { - params: { id: '1' }, - body: { recipient_name: 'Missing Name' }, - }; - const res = mockResponse(); - await callHandler(routeHandlers['patch:/:id'], req, res); + await api + .patch('/api/transactions/1') + .send({ recipient_name: 'Missing Name' }) + .expect(400); - expect(res.status).toHaveBeenCalledWith(400); expect(transactionRepository.update).not.toHaveBeenCalled(); }); it('should return 400 for invalid category_name format', async () => { - const req = { - params: { id: '1' }, - body: { category_name: 'INVALID' }, - }; - const res = mockResponse(); - await callHandler(routeHandlers['patch:/:id'], req, res); - - expect(res.status).toHaveBeenCalledWith(400); + await api + .patch('/api/transactions/1') + .send({ category_name: 'INVALID' }) + .expect(400); + expect(transactionRepository.update).not.toHaveBeenCalled(); }); @@ -564,17 +610,20 @@ describe('Transaction Routes', () => { dbQuery.mockResolvedValueOnce({ rows: [{ id: 11 }] }); dbQuery.mockResolvedValueOnce({ rows: [] }); - const req = { - params: { id: '1' }, - body: { + await api + .patch('/api/transactions/1') + .send({ recipient_name: 'Known Recipient', category_name: 'FOOD:UNKNOWN', - }, - }; - const res = mockResponse(); - await callHandler(routeHandlers['patch:/:id'], req, res); + }) + .expect(400); + + expect(transactionRepository.update).not.toHaveBeenCalled(); + }); + + it('rejects a non-integer :id via validateIdParam before the rate limiter and handler', async () => { + await api.patch('/api/transactions/abc').send({ amount: -75 }).expect(400); - expect(res.status).toHaveBeenCalledWith(400); expect(transactionRepository.update).not.toHaveBeenCalled(); }); }); @@ -583,23 +632,15 @@ describe('Transaction Routes', () => { it('should delete and return 204 with no body', async () => { transactionRepository.hardDelete.mockResolvedValue(true); - const req = { params: { id: '1' } }; - const res = mockResponse(); - await routeHandlers['delete:/:id'](req, res); + const res = await api.delete('/api/transactions/1').expect(204); - expect(res.status).toHaveBeenCalledWith(204); - expect(res.send).toHaveBeenCalledWith(); - expect(res.json).not.toHaveBeenCalled(); + expect(res.text).toBe(''); }); it('should return 404 for non-existent', async () => { transactionRepository.hardDelete.mockResolvedValue(false); - const req = { params: { id: '99999' } }; - const res = mockResponse(); - await callHandler(routeHandlers['delete:/:id'], req, res); - - expect(res.status).toHaveBeenCalledWith(404); + await api.delete('/api/transactions/99999').expect(404); }); it('removes attachment files from disk after the delete', async () => { @@ -611,9 +652,7 @@ describe('Transaction Routes', () => { 'attachments/1/receipt-b.pdf', ]); - const req = { params: { id: '1' } }; - const res = mockResponse(); - await routeHandlers['delete:/:id'](req, res); + await api.delete('/api/transactions/1').expect(204); expect(attachmentRepository.listPathsByTransactionIds).toHaveBeenCalledWith([1]); expect(removeAttachmentFile).toHaveBeenCalledWith('attachments/1/receipt-a.png'); @@ -624,9 +663,7 @@ describe('Transaction Routes', () => { transactionRepository.hardDelete.mockResolvedValue(false); attachmentRepository.listPathsByTransactionIds.mockResolvedValue(['attachments/9/x.png']); - const req = { params: { id: '99999' } }; - const res = mockResponse(); - await callHandler(routeHandlers['delete:/:id'], req, res); + await api.delete('/api/transactions/99999').expect(404); expect(removeAttachmentFile).not.toHaveBeenCalled(); }); @@ -634,38 +671,19 @@ describe('Transaction Routes', () => { describe('DELETE /transfers/:id', () => { it('clears the transfer mark and returns 204 with no body', async () => { - const req = { params: { id: '10' } }; - const res = mockResponse(); - await routeHandlers['delete:/transfers/:id'](req, res); + const res = await api.delete('/api/transactions/transfers/10').expect(204); expect(unmarkTransfer).toHaveBeenCalledWith(10); expect(scheduleReconcile).toHaveBeenCalled(); - expect(res.status).toHaveBeenCalledWith(204); - expect(res.send).toHaveBeenCalledWith(); - expect(res.json).not.toHaveBeenCalled(); + expect(res.text).toBe(''); }); }); -}); -function mockResponse() { - return createMockResponse({ setHeader: vi.fn(), write: vi.fn(), end: vi.fn(), headersSent: false }); -} + describe('unmatched paths', () => { + it('falls through to the 404 error envelope', async () => { + const res = await api.get('/api/transactions/1/nope/nope').expect(404); -/** - * Simulates Express error-handler middleware for routes that throw typed errors. - * Routes use `throw new NotFoundError / ValidationError / ConflictError` which - * propagates to the centralized error handler in production. In unit tests we - * catch the error here and replicate the handler's response shape. - */ -async function callHandler(handler, req, res) { - try { - await handler(req, res); - } catch (err) { - const status = err.status ?? 500; - const code = err.code ?? 'INTERNAL_SERVER_ERROR'; - const message = err.message ?? 'Internal server error'; - const error = { code, message }; - if (err.details !== undefined) error.details = err.details; - res.status(status).json({ ok: false, error }); - } -} + expect(res.body.error.code).toBe('NOT_FOUND'); + }); + }); +}); diff --git a/apps/node-backend/tests/routes/transactionsBulkDelete.test.js b/apps/node-backend/tests/routes/transactionsBulkDelete.test.js index 2ebd2929..fd85366d 100644 --- a/apps/node-backend/tests/routes/transactionsBulkDelete.test.js +++ b/apps/node-backend/tests/routes/transactionsBulkDelete.test.js @@ -1,18 +1,15 @@ /** * POST /bulk-delete — id-mode, filter-mode, validation, atomicity. + * + * Driven over HTTP against the real router (tests/helpers/routeApp.js), so the + * route's rate limiter, JSON body parsing and the centralized error handler are + * all on the tested path. */ import { describe, it, expect, vi, beforeEach } from 'vitest'; import { mockPooledTxConnection } from '../helpers/repoMocks.js'; import { mockTransactionRepository, mockDeduplication, mockTransferReconciliation, mockCurrencyConversion, mockAttachmentRecordService, mockAttachmentService } from '../helpers/transactionsRouteMocks.js'; import { mockLogger } from '../helpers/mockLogger.js'; -import { createMockRouter, createMockResponse } from '../helpers/routeHarness.js'; - -const { router: mockRouter, handlers: routeHandlers } = createMockRouter(); - -vi.mock('express', () => ({ - default: { Router: () => mockRouter }, - Router: () => mockRouter, -})); +import { routeAgent } from '../helpers/routeApp.js'; vi.mock('../../src/repositories/transactionRepository.js', () => mockTransactionRepository()); @@ -32,47 +29,27 @@ vi.mock('../../src/services/attachmentRecordService.js', () => mockAttachmentRec vi.mock('../../src/services/attachmentService.js', () => mockAttachmentService()); -await import('../../src/routes/transactions.js'); +const { default: transactionsRouter } = await import('../../src/routes/transactions.js'); import { getClient, query as dbQuery } from '../../src/database/connection.js'; import { scheduleReconcile } from '../../src/services/transferReconciliationService.js'; -import { attachmentRepository } from '../../src/services/attachmentRecordService.js'; -import { removeAttachmentFile } from '../../src/services/attachmentService.js'; - -const handler = routeHandlers['post:/bulk-delete']; -function mockResponse() { - return createMockResponse({ headersSent: false }); -} - -async function callHandler(req, res) { - try { - await handler(req, res); - } catch (err) { - const status = err.status ?? 500; - res.status(status).json({ ok: false, error: { code: err.code ?? 'INTERNAL_SERVER_ERROR', message: err.message } }); - } -} +const api = routeAgent(transactionsRouter, { mountPath: '/api/transactions' }); +const bulkDelete = (body) => api.post('/api/transactions/bulk-delete').send(body); describe('POST /bulk-delete — input validation', () => { beforeEach(() => vi.clearAllMocks()); it('returns 400 when neither ids nor filter is given', async () => { - const res = mockResponse(); - await callHandler({ body: {} }, res); - expect(res.status).toHaveBeenCalledWith(400); + await bulkDelete({}).expect(400); }); it('returns 400 when both ids and filter are given', async () => { - const res = mockResponse(); - await callHandler({ body: { ids: [1], filter: { search: 'x' } } }, res); - expect(res.status).toHaveBeenCalledWith(400); + await bulkDelete({ ids: [1], filter: { search: 'x' } }).expect(400); }); it('returns 400 when ids exceeds 500 entries', async () => { - const res = mockResponse(); - await callHandler({ body: { ids: Array.from({ length: 501 }, (_, i) => i + 1) } }, res); - expect(res.status).toHaveBeenCalledWith(400); + await bulkDelete({ ids: Array.from({ length: 501 }, (_, i) => i + 1) }).expect(400); }); }); @@ -87,11 +64,9 @@ describe('POST /bulk-delete — id-mode success', () => { const release = vi.fn(); getClient.mockResolvedValue({ query: clientQuery, release }); - const res = mockResponse(); - await callHandler({ body: { ids: [1, 2, 3] } }, res); + const res = await bulkDelete({ ids: [1, 2, 3] }).expect(200); - const data = res.json.mock.calls[0][0].data; - expect(data.deleted).toBe(3); + expect(res.body.data.deleted).toBe(3); expect(scheduleReconcile).toHaveBeenCalledTimes(1); const deleteCall = clientQuery.mock.calls.find(([sql]) => sql.includes('DELETE FROM transactions')); @@ -107,10 +82,9 @@ describe('POST /bulk-delete — id-mode success', () => { const release = vi.fn(); getClient.mockResolvedValue({ query: clientQuery, release }); - const res = mockResponse(); - await callHandler({ body: { ids: [9999] } }, res); + const res = await bulkDelete({ ids: [9999] }).expect(200); - expect(res.json.mock.calls[0][0].data.deleted).toBe(0); + expect(res.body.data.deleted).toBe(0); expect(scheduleReconcile).not.toHaveBeenCalled(); }); }); @@ -120,9 +94,9 @@ describe('POST /bulk-delete — filter-mode', () => { it('rejects filter requests that exceed the cap', async () => { dbQuery.mockResolvedValueOnce({ rows: [{ n: 6000 }] }); - const res = mockResponse(); - await callHandler({ body: { filter: { search: 'big' } } }, res); - expect(res.status).toHaveBeenCalledWith(400); + + await bulkDelete({ filter: { search: 'big' } }).expect(400); + expect(getClient).not.toHaveBeenCalled(); }); @@ -138,10 +112,9 @@ describe('POST /bulk-delete — filter-mode', () => { const release = vi.fn(); getClient.mockResolvedValue({ query: clientQuery, release }); - const res = mockResponse(); - await callHandler({ body: { filter: { search: 'cafe' } } }, res); + const res = await bulkDelete({ filter: { search: 'cafe' } }).expect(200); - expect(res.json.mock.calls[0][0].data.deleted).toBe(2); + expect(res.body.data.deleted).toBe(2); const deleteCall = clientQuery.mock.calls.find(([sql]) => sql.includes('DELETE')); expect(deleteCall[1]).toEqual([[11, 22]]); }); @@ -158,11 +131,10 @@ describe('POST /bulk-delete — atomicity', () => { const release = vi.fn(); getClient.mockResolvedValue({ query: clientQuery, release }); - const res = mockResponse(); - await callHandler({ body: { ids: [1] } }, res); + const res = await bulkDelete({ ids: [1] }).expect(500); expect(scheduleReconcile).not.toHaveBeenCalled(); - expect(res.status).toHaveBeenCalledWith(500); + expect(res.body.error.code).toBe('INTERNAL_SERVER_ERROR'); const rollback = clientQuery.mock.calls.find(([sql]) => sql === 'ROLLBACK'); expect(rollback).toBeDefined(); expect(release).toHaveBeenCalledTimes(1); diff --git a/apps/node-backend/tests/routes/transactionsBulkExport.test.js b/apps/node-backend/tests/routes/transactionsBulkExport.test.js index f4524769..91ec29b5 100644 --- a/apps/node-backend/tests/routes/transactionsBulkExport.test.js +++ b/apps/node-backend/tests/routes/transactionsBulkExport.test.js @@ -1,18 +1,15 @@ /** * POST /bulk-export — id-mode + filter-mode streaming, format gate. + * + * Driven over HTTP against the real router (tests/helpers/routeApp.js), so the + * streamed body and the download headers are read off a real response instead + * of `res.write`/`res.setHeader` spies. */ import { describe, it, expect, vi, beforeEach } from 'vitest'; import { mockConnection } from '../helpers/repoMocks.js'; import { mockTransactionRepository, mockDeduplication, mockMaterializedViews, mockCurrencyConversion } from '../helpers/transactionsRouteMocks.js'; import { mockLogger } from '../helpers/mockLogger.js'; -import { createMockRouter, createMockResponse } from '../helpers/routeHarness.js'; - -const { router: mockRouter, handlers: routeHandlers } = createMockRouter(); - -vi.mock('express', () => ({ - default: { Router: () => mockRouter }, - Router: () => mockRouter, -})); +import { routeAgent } from '../helpers/routeApp.js'; vi.mock('../../src/repositories/transactionRepository.js', () => mockTransactionRepository()); @@ -28,24 +25,12 @@ vi.mock('../../src/services/currency/currencyConversionService.js', () => mockCu vi.mock('../../src/database/connection.js', () => mockConnection({ getClient: vi.fn() })); -await import('../../src/routes/transactions.js'); +const { default: transactionsRouter } = await import('../../src/routes/transactions.js'); import { query as dbQuery } from '../../src/database/connection.js'; -const handler = routeHandlers['post:/bulk-export']; - -function mockResponse() { - return createMockResponse({ setHeader: vi.fn(), write: vi.fn(), end: vi.fn(), headersSent: false }); -} - -async function callHandler(req, res) { - try { - await handler(req, res); - } catch (err) { - const status = err.status ?? 500; - res.status(status).json({ ok: false, error: { code: err.code ?? 'INTERNAL_SERVER_ERROR', message: err.message } }); - } -} +const api = routeAgent(transactionsRouter, { mountPath: '/api/transactions' }); +const bulkExport = (body) => api.post('/api/transactions/bulk-export').send(body); const SAMPLE_ROW = { id: 1, @@ -65,15 +50,11 @@ describe('POST /bulk-export — validation', () => { beforeEach(() => vi.clearAllMocks()); it('rejects unknown format', async () => { - const res = mockResponse(); - await callHandler({ body: { ids: [1], format: 'xml' } }, res); - expect(res.status).toHaveBeenCalledWith(400); + await bulkExport({ ids: [1], format: 'xml' }).expect(400); }); it('rejects when neither ids nor filter is given', async () => { - const res = mockResponse(); - await callHandler({ body: { format: 'csv' } }, res); - expect(res.status).toHaveBeenCalledWith(400); + await bulkExport({ format: 'csv' }).expect(400); }); }); @@ -85,18 +66,14 @@ describe('POST /bulk-export — CSV success', () => { .mockResolvedValueOnce({ rows: [{ '?column?': 1 }] }) // probe .mockResolvedValueOnce({ rows: [SAMPLE_ROW] }); // chunk - const res = mockResponse(); - await callHandler({ body: { ids: [1], format: 'csv' } }, res); + const res = await bulkExport({ ids: [1], format: 'csv' }).expect(200); - const headers = res.setHeader.mock.calls; - expect(headers.find(([k]) => k === 'Content-Type')[1]).toBe('text/csv'); - expect(headers.find(([k]) => k === 'Content-Disposition')[1]).toMatch(/transactions_export_/); + expect(res.headers['content-type']).toBe('text/csv'); + expect(res.headers['content-disposition']).toMatch(/transactions_export_/); - const writes = res.write.mock.calls.map(([s]) => s); - expect(writes[0]).toMatch(/^Date,Bank Account,Recipient/); - expect(writes.join('')).toContain('Trader Joe'); - expect(writes.join('')).toContain('weekly'); - expect(res.end).toHaveBeenCalledTimes(1); + expect(res.text).toMatch(/^Date,Bank Account,Recipient/); + expect(res.text).toContain('Trader Joe'); + expect(res.text).toContain('weekly'); }); }); @@ -108,14 +85,11 @@ describe('POST /bulk-export — NDJSON success', () => { .mockResolvedValueOnce({ rows: [{ '?column?': 1 }] }) // probe .mockResolvedValueOnce({ rows: [SAMPLE_ROW] }); // chunk - const res = mockResponse(); - await callHandler({ body: { ids: [1], format: 'json' } }, res); + const res = await bulkExport({ ids: [1], format: 'json' }).expect(200); - const ctype = res.setHeader.mock.calls.find(([k]) => k === 'Content-Type')[1]; - expect(ctype).toBe('application/x-ndjson'); + expect(res.headers['content-type']).toBe('application/x-ndjson'); - const body = res.write.mock.calls.map(([s]) => s).join(''); - const firstLine = body.split('\n').filter(Boolean)[0]; + const firstLine = res.text.split('\n').filter(Boolean)[0]; const parsed = JSON.parse(firstLine); expect(parsed.id).toBe(1); expect(parsed.recipient).toBe('Trader Joe'); @@ -129,8 +103,6 @@ describe('POST /bulk-export — filter-mode resolves through bulk selection', () it('rejects filter requests over the cap', async () => { dbQuery.mockResolvedValueOnce({ rows: [{ n: 7000 }] }); - const res = mockResponse(); - await callHandler({ body: { filter: { search: 'big' }, format: 'csv' } }, res); - expect(res.status).toHaveBeenCalledWith(400); + await bulkExport({ filter: { search: 'big' }, format: 'csv' }).expect(400); }); }); diff --git a/apps/node-backend/tests/routes/transactionsBulkTag.test.js b/apps/node-backend/tests/routes/transactionsBulkTag.test.js index ab3affb9..b9b8c248 100644 --- a/apps/node-backend/tests/routes/transactionsBulkTag.test.js +++ b/apps/node-backend/tests/routes/transactionsBulkTag.test.js @@ -1,19 +1,16 @@ /** * Bulk-tag route tests — isolated file so we can add withTransaction to the - * connection mock without touching the 584-line transactions.test.js. + * connection mock without touching the large transactions.test.js. + * + * Driven over HTTP against the real router (tests/helpers/routeApp.js): the + * per-route rate limiter declared on POST /bulk-tag (routes/transactions.js:433) + * and the centralized error handler are both on the tested path. */ import { describe, it, expect, vi, beforeEach } from 'vitest'; import { mockPooledTxConnection } from '../helpers/repoMocks.js'; import { mockTransactionRepository, mockDeduplication, mockTransferReconciliation, mockCurrencyConversion } from '../helpers/transactionsRouteMocks.js'; import { mockLogger } from '../helpers/mockLogger.js'; -import { createMockRouter, createMockResponse } from '../helpers/routeHarness.js'; - -const { router: mockRouter, handlers: routeHandlers } = createMockRouter(); - -vi.mock('express', () => ({ - default: { Router: () => mockRouter }, - Router: () => mockRouter, -})); +import { routeAgent } from '../helpers/routeApp.js'; vi.mock('../../src/repositories/transactionRepository.js', () => mockTransactionRepository()); @@ -29,47 +26,41 @@ vi.mock('../../src/services/currency/currencyConversionService.js', () => mockCu vi.mock('../../src/database/connection.js', () => mockPooledTxConnection()); -await import('../../src/routes/transactions.js'); +const { default: transactionsRouter } = await import('../../src/routes/transactions.js'); import { getClient, query as dbQuery } from '../../src/database/connection.js'; import { scheduleReconcile } from '../../src/services/transferReconciliationService.js'; +const api = routeAgent(transactionsRouter, { mountPath: '/api/transactions' }); +const bulkTag = (body) => api.post('/api/transactions/bulk-tag').send(body); + describe('POST /bulk-tag — input validation', () => { beforeEach(() => vi.clearAllMocks()); it('returns 400 when transaction_ids is missing', async () => { - const req = { body: { add_slugs: ['rome-2020'] } }; - const res = mockResponse(); - await callHandler(routeHandlers['post:/bulk-tag'], req, res); - expect(res.status).toHaveBeenCalledWith(400); + await bulkTag({ add_slugs: ['rome-2020'] }).expect(400); }); it('returns 400 when transaction_ids is empty array', async () => { - const req = { body: { transaction_ids: [], add_slugs: ['rome-2020'] } }; - const res = mockResponse(); - await callHandler(routeHandlers['post:/bulk-tag'], req, res); - expect(res.status).toHaveBeenCalledWith(400); + await bulkTag({ transaction_ids: [], add_slugs: ['rome-2020'] }).expect(400); }); it('returns 400 when transaction_ids exceeds 500 entries', async () => { - const req = { body: { transaction_ids: Array.from({ length: 501 }, (_, i) => i + 1), add_slugs: ['rome'] } }; - const res = mockResponse(); - await callHandler(routeHandlers['post:/bulk-tag'], req, res); - expect(res.status).toHaveBeenCalledWith(400); + await bulkTag({ + transaction_ids: Array.from({ length: 501 }, (_, i) => i + 1), + add_slugs: ['rome'], + }).expect(400); }); it('returns 400 when add_slugs exceeds 50 entries', async () => { - const req = { body: { transaction_ids: [1], add_slugs: Array.from({ length: 51 }, (_, i) => `tag-${i}`) } }; - const res = mockResponse(); - await callHandler(routeHandlers['post:/bulk-tag'], req, res); - expect(res.status).toHaveBeenCalledWith(400); + await bulkTag({ + transaction_ids: [1], + add_slugs: Array.from({ length: 51 }, (_, i) => `tag-${i}`), + }).expect(400); }); it('returns 400 when both add_slugs and remove_slugs are empty', async () => { - const req = { body: { transaction_ids: [1], add_slugs: [], remove_slugs: [] } }; - const res = mockResponse(); - await callHandler(routeHandlers['post:/bulk-tag'], req, res); - expect(res.status).toHaveBeenCalledWith(400); + await bulkTag({ transaction_ids: [1], add_slugs: [], remove_slugs: [] }).expect(400); }); }); @@ -78,20 +69,18 @@ describe('POST /bulk-tag — unknown slug rejection', () => { it('returns 400 listing unknown add slug before writing anything', async () => { dbQuery.mockResolvedValueOnce({ rows: [] }); // no active tag found - const req = { body: { transaction_ids: [1], add_slugs: ['ghost-tag'] } }; - const res = mockResponse(); - await callHandler(routeHandlers['post:/bulk-tag'], req, res); - expect(res.status).toHaveBeenCalledWith(400); - expect(res.json.mock.calls[0][0].error.message).toContain('ghost-tag'); + + const res = await bulkTag({ transaction_ids: [1], add_slugs: ['ghost-tag'] }).expect(400); + + expect(res.body.error.message).toContain('ghost-tag'); expect(getClient).not.toHaveBeenCalled(); }); it('returns 400 listing unknown remove slug before writing anything', async () => { dbQuery.mockResolvedValueOnce({ rows: [] }); // no tag found - const req = { body: { transaction_ids: [1], remove_slugs: ['ghost-tag'] } }; - const res = mockResponse(); - await callHandler(routeHandlers['post:/bulk-tag'], req, res); - expect(res.status).toHaveBeenCalledWith(400); + + await bulkTag({ transaction_ids: [1], remove_slugs: ['ghost-tag'] }).expect(400); + expect(getClient).not.toHaveBeenCalled(); }); }); @@ -108,14 +97,11 @@ describe('POST /bulk-tag — success paths', () => { const release = vi.fn(); getClient.mockResolvedValue({ query: clientQuery, release }); - const req = { body: { transaction_ids: [1, 2], add_slugs: ['rome-2020'] } }; - const res = mockResponse(); - await routeHandlers['post:/bulk-tag'](req, res); + const res = await bulkTag({ transaction_ids: [1, 2], add_slugs: ['rome-2020'] }).expect(200); - const data = res.json.mock.calls[0][0].data; - expect(data.added).toBe(2); - expect(data.removed).toBe(0); - expect(data.transactions_affected).toBe(2); + expect(res.body.data.added).toBe(2); + expect(res.body.data.removed).toBe(0); + expect(res.body.data.transactions_affected).toBe(2); expect(scheduleReconcile).toHaveBeenCalledTimes(1); }); @@ -128,13 +114,10 @@ describe('POST /bulk-tag — success paths', () => { const release = vi.fn(); getClient.mockResolvedValue({ query: clientQuery, release }); - const req = { body: { transaction_ids: [1], remove_slugs: ['rome-2020'] } }; - const res = mockResponse(); - await routeHandlers['post:/bulk-tag'](req, res); + const res = await bulkTag({ transaction_ids: [1], remove_slugs: ['rome-2020'] }).expect(200); - const data = res.json.mock.calls[0][0].data; - expect(data.removed).toBe(1); - expect(data.added).toBe(0); + expect(res.body.data.removed).toBe(1); + expect(res.body.data.added).toBe(0); }); it('adds and removes in a single transaction', async () => { @@ -149,13 +132,12 @@ describe('POST /bulk-tag — success paths', () => { const release = vi.fn(); getClient.mockResolvedValue({ query: clientQuery, release }); - const req = { body: { transaction_ids: [1], add_slugs: ['rome-2020'], remove_slugs: ['work-trip'] } }; - const res = mockResponse(); - await routeHandlers['post:/bulk-tag'](req, res); + const res = await bulkTag({ + transaction_ids: [1], add_slugs: ['rome-2020'], remove_slugs: ['work-trip'], + }).expect(200); - const data = res.json.mock.calls[0][0].data; - expect(data.added).toBe(1); - expect(data.removed).toBe(1); + expect(res.body.data.added).toBe(1); + expect(res.body.data.removed).toBe(1); }); }); @@ -171,28 +153,12 @@ describe('POST /bulk-tag — atomicity', () => { const release = vi.fn(); getClient.mockResolvedValue({ query: clientQuery, release }); - const req = { body: { transaction_ids: [1], add_slugs: ['rome-2020'] } }; - const res = mockResponse(); - await callHandler(routeHandlers['post:/bulk-tag'], req, res); + const res = await bulkTag({ transaction_ids: [1], add_slugs: ['rome-2020'] }).expect(500); expect(scheduleReconcile).not.toHaveBeenCalled(); - expect(res.status).toHaveBeenCalledWith(500); + expect(res.body.error.code).toBe('INTERNAL_SERVER_ERROR'); const rollbackCall = clientQuery.mock.calls.find(([sql]) => sql === 'ROLLBACK'); expect(rollbackCall).toBeDefined(); expect(release).toHaveBeenCalledTimes(1); }); }); - -function mockResponse() { - return createMockResponse({ headersSent: false }); -} - -async function callHandler(handler, req, res) { - try { - await handler(req, res); - } catch (err) { - const status = err.status ?? 500; - const code = err.code ?? 'INTERNAL_SERVER_ERROR'; - res.status(status).json({ ok: false, error: { code, message: err.message } }); - } -} diff --git a/apps/node-backend/tests/routes/transactionsBulkUpdate.test.js b/apps/node-backend/tests/routes/transactionsBulkUpdate.test.js index df6ba3dd..2bb24667 100644 --- a/apps/node-backend/tests/routes/transactionsBulkUpdate.test.js +++ b/apps/node-backend/tests/routes/transactionsBulkUpdate.test.js @@ -1,18 +1,15 @@ /** * POST /bulk-update — field validation, FK pre-checks, single transaction. + * + * Driven over HTTP against the real router (tests/helpers/routeApp.js), so the + * route's rate limiter, JSON body parsing and the centralized error handler are + * all on the tested path. */ import { describe, it, expect, vi, beforeEach } from 'vitest'; import { mockPooledTxConnection } from '../helpers/repoMocks.js'; import { mockTransactionRepository, mockDeduplication, mockTransferReconciliation, mockCurrencyConversion } from '../helpers/transactionsRouteMocks.js'; import { mockLogger } from '../helpers/mockLogger.js'; -import { createMockRouter, createMockResponse } from '../helpers/routeHarness.js'; - -const { router: mockRouter, handlers: routeHandlers } = createMockRouter(); - -vi.mock('express', () => ({ - default: { Router: () => mockRouter }, - Router: () => mockRouter, -})); +import { routeAgent } from '../helpers/routeApp.js'; vi.mock('../../src/repositories/transactionRepository.js', () => mockTransactionRepository()); @@ -28,46 +25,28 @@ vi.mock('../../src/services/currency/currencyConversionService.js', () => mockCu vi.mock('../../src/database/connection.js', () => mockPooledTxConnection()); -await import('../../src/routes/transactions.js'); +const { default: transactionsRouter } = await import('../../src/routes/transactions.js'); import { getClient, query as dbQuery } from '../../src/database/connection.js'; import { scheduleReconcile } from '../../src/services/transferReconciliationService.js'; -const handler = routeHandlers['post:/bulk-update']; - -function mockResponse() { - return createMockResponse({ headersSent: false }); -} - -async function callHandler(req, res) { - try { - await handler(req, res); - } catch (err) { - const status = err.status ?? 500; - res.status(status).json({ ok: false, error: { code: err.code ?? 'INTERNAL_SERVER_ERROR', message: err.message } }); - } -} +const api = routeAgent(transactionsRouter, { mountPath: '/api/transactions' }); +const bulkUpdate = (body) => api.post('/api/transactions/bulk-update').send(body); describe('POST /bulk-update — field validation', () => { beforeEach(() => vi.clearAllMocks()); it('rejects when fields object is missing', async () => { - const res = mockResponse(); - await callHandler({ body: { ids: [1] } }, res); - expect(res.status).toHaveBeenCalledWith(400); + await bulkUpdate({ ids: [1] }).expect(400); }); it('rejects when no recognized field is set', async () => { - const res = mockResponse(); - await callHandler({ body: { ids: [1], fields: { foo: 1 } } }, res); - expect(res.status).toHaveBeenCalledWith(400); - expect(res.json.mock.calls[0][0].error.message).toMatch(/at least one/i); + const res = await bulkUpdate({ ids: [1], fields: { foo: 1 } }).expect(400); + expect(res.body.error.message).toMatch(/at least one/i); }); it('rejects non-integer category_id', async () => { - const res = mockResponse(); - await callHandler({ body: { ids: [1], fields: { category_id: 'abc' } } }, res); - expect(res.status).toHaveBeenCalledWith(400); + await bulkUpdate({ ids: [1], fields: { category_id: 'abc' } }).expect(400); }); it('accepts category_id = null (uncategorize)', async () => { @@ -78,25 +57,20 @@ describe('POST /bulk-update — field validation', () => { const release = vi.fn(); getClient.mockResolvedValue({ query: clientQuery, release }); - const res = mockResponse(); - await callHandler({ body: { ids: [1], fields: { category_id: null } } }, res); + const res = await bulkUpdate({ ids: [1], fields: { category_id: null } }).expect(200); - expect(res.json.mock.calls[0][0].data.updated).toBe(1); + expect(res.body.data.updated).toBe(1); const updateCall = clientQuery.mock.calls.find(([sql]) => sql.includes('UPDATE transactions')); expect(updateCall[0]).toMatch(/category_id = \$2/); expect(updateCall[1]).toEqual([[1], null]); }); it('rejects null recipient_id (column is NOT NULL)', async () => { - const res = mockResponse(); - await callHandler({ body: { ids: [1], fields: { recipient_id: null } } }, res); - expect(res.status).toHaveBeenCalledWith(400); + await bulkUpdate({ ids: [1], fields: { recipient_id: null } }).expect(400); }); it('rejects non-boolean is_active', async () => { - const res = mockResponse(); - await callHandler({ body: { ids: [1], fields: { is_active: 'true' } } }, res); - expect(res.status).toHaveBeenCalledWith(400); + await bulkUpdate({ ids: [1], fields: { is_active: 'true' } }).expect(400); }); }); @@ -105,17 +79,17 @@ describe('POST /bulk-update — FK pre-checks', () => { it('returns 400 when category does not exist', async () => { dbQuery.mockResolvedValueOnce({ rows: [] }); // category lookup - const res = mockResponse(); - await callHandler({ body: { ids: [1], fields: { category_id: 999 } } }, res); - expect(res.status).toHaveBeenCalledWith(400); + + await bulkUpdate({ ids: [1], fields: { category_id: 999 } }).expect(400); + expect(getClient).not.toHaveBeenCalled(); }); it('returns 400 when recipient does not exist', async () => { dbQuery.mockResolvedValueOnce({ rows: [] }); // recipient lookup - const res = mockResponse(); - await callHandler({ body: { ids: [1], fields: { recipient_id: 999 } } }, res); - expect(res.status).toHaveBeenCalledWith(400); + + await bulkUpdate({ ids: [1], fields: { recipient_id: 999 } }).expect(400); + expect(getClient).not.toHaveBeenCalled(); }); }); @@ -132,10 +106,9 @@ describe('POST /bulk-update — success paths', () => { const release = vi.fn(); getClient.mockResolvedValue({ query: clientQuery, release }); - const res = mockResponse(); - await callHandler({ body: { ids: [1, 2], fields: { category_id: 7 } } }, res); + const res = await bulkUpdate({ ids: [1, 2], fields: { category_id: 7 } }).expect(200); - expect(res.json.mock.calls[0][0].data.updated).toBe(2); + expect(res.body.data.updated).toBe(2); expect(scheduleReconcile).toHaveBeenCalledTimes(1); }); @@ -150,15 +123,12 @@ describe('POST /bulk-update — success paths', () => { const release = vi.fn(); getClient.mockResolvedValue({ query: clientQuery, release }); - const res = mockResponse(); - await callHandler({ - body: { - ids: [5], - fields: { category_id: 7, recipient_id: 99, is_active: false }, - }, - }, res); + const res = await bulkUpdate({ + ids: [5], + fields: { category_id: 7, recipient_id: 99, is_active: false }, + }).expect(200); - expect(res.json.mock.calls[0][0].data.updated).toBe(1); + expect(res.body.data.updated).toBe(1); const updateCall = clientQuery.mock.calls.find(([sql]) => sql.includes('UPDATE transactions')); expect(updateCall[0]).toMatch(/category_id = \$2.*recipient_id = \$3.*is_active = \$4.*updated_at = NOW\(\)/s); expect(updateCall[1]).toEqual([[5], 7, 99, false]); @@ -172,10 +142,9 @@ describe('POST /bulk-update — success paths', () => { const release = vi.fn(); getClient.mockResolvedValue({ query: clientQuery, release }); - const res = mockResponse(); - await callHandler({ body: { ids: [12345], fields: { is_active: true } } }, res); + const res = await bulkUpdate({ ids: [12345], fields: { is_active: true } }).expect(200); - expect(res.json.mock.calls[0][0].data.updated).toBe(0); + expect(res.body.data.updated).toBe(0); expect(scheduleReconcile).not.toHaveBeenCalled(); }); }); @@ -191,11 +160,9 @@ describe('POST /bulk-update — atomicity', () => { const release = vi.fn(); getClient.mockResolvedValue({ query: clientQuery, release }); - const res = mockResponse(); - await callHandler({ body: { ids: [1], fields: { is_active: false } } }, res); + await bulkUpdate({ ids: [1], fields: { is_active: false } }).expect(500); expect(scheduleReconcile).not.toHaveBeenCalled(); - expect(res.status).toHaveBeenCalledWith(500); expect(release).toHaveBeenCalledTimes(1); }); }); diff --git a/apps/node-backend/tests/routes/transactionsValidationPins.test.js b/apps/node-backend/tests/routes/transactionsValidationPins.test.js index 948a2218..781fbf4c 100644 --- a/apps/node-backend/tests/routes/transactionsValidationPins.test.js +++ b/apps/node-backend/tests/routes/transactionsValidationPins.test.js @@ -1,12 +1,15 @@ /** * Validation-behavior pins for the transactions route bodies (ZOD-06). * - * These tests pin the exact accept/reject/coercion behavior of the hand-rolled - * validation in POST /, PATCH /:id, POST /bulk-tag, and POST /bulk-update - * BEFORE the zod migration, so the swap cannot silently change the wire - * contract: rejected inputs stay 400/VALIDATION_ERROR, accepted inputs reach - * the repository byte-identically (raw vs coerced), and clear-vs-absent - * semantics survive. + * These tests pin the exact accept/reject/coercion behavior of the body + * validation in POST /, PATCH /:id, POST /bulk-tag, and POST /bulk-update, so + * the wire contract cannot silently change: rejected inputs stay + * 400/VALIDATION_ERROR, accepted inputs reach the repository byte-identically + * (raw vs coerced), and clear-vs-absent semantics survive. + * + * Driven over HTTP against the real router (tests/helpers/routeApp.js), so the + * status/envelope assertions are the ones the error handler actually emits + * rather than a hand-replayed approximation. */ import { describe, it, expect, vi, beforeEach } from 'vitest'; import { mockPooledTxConnection } from '../helpers/repoMocks.js'; @@ -17,14 +20,7 @@ import { mockCurrencyConversion, } from '../helpers/transactionsRouteMocks.js'; import { mockLogger } from '../helpers/mockLogger.js'; -import { createMockRouter, createMockResponse } from '../helpers/routeHarness.js'; - -const { router: mockRouter, handlers: routeHandlers } = createMockRouter(); - -vi.mock('express', () => ({ - default: { Router: () => mockRouter }, - Router: () => mockRouter, -})); +import { routeAgent } from '../helpers/routeApp.js'; vi.mock('../../src/repositories/transactionRepository.js', () => mockTransactionRepository()); @@ -45,25 +41,11 @@ vi.mock('../../src/services/plannedMatchService.js', () => ({ })); import transactionRepository from '../../src/repositories/transactionRepository.js'; -import { recordManualRawTransaction } from '../../src/services/deduplication.js'; -import { ValidationError } from '../../src/middleware/errorHandler.js'; -await import('../../src/routes/transactions.js'); +import { recordManualRawTransaction, isManualDuplicate } from '../../src/services/deduplication.js'; -function mockResponse() { - return createMockResponse({ headersSent: false }); -} +const { default: transactionsRouter } = await import('../../src/routes/transactions.js'); -// Runs a handler through the same status/envelope shaping as errorHandler.js -// so pins can assert the wire format (400 + VALIDATION_ERROR). -async function callHandler(handler, req, res) { - try { - await handler(req, res); - } catch (err) { - const status = err.status ?? 500; - const code = err.code ?? 'INTERNAL_SERVER_ERROR'; - res.status(status).json({ ok: false, error: { code, message: err.message } }); - } -} +const api = routeAgent(transactionsRouter, { mountPath: '/api/transactions' }); const validPostBody = () => ({ transaction_date: '2026-01-15', @@ -72,58 +54,65 @@ const validPostBody = () => ({ amount: -50, }); -const runPost = (body) => routeHandlers['post:/']( - { body }, - createMockResponse({ headersSent: false }), -); - -const runPatch = (body) => routeHandlers['patch:/:id']( - { params: { id: '1' }, body }, - createMockResponse({ headersSent: false }), -); +const post = (body) => api.post('/api/transactions/').send(body); +const patch = (body) => api.patch('/api/transactions/1').send(body); +const bulkTag = (body) => api.post('/api/transactions/bulk-tag').send(body); +const bulkUpdate = (body) => api.post('/api/transactions/bulk-update').send(body); + +/** Assert a 400 VALIDATION_ERROR envelope and return the response. */ +async function expectValidationError(pending) { + const res = await pending; + expect(res.status).toBe(400); + expect(res.body.ok).toBe(false); + expect(res.body.error.code).toBe('VALIDATION_ERROR'); + return res; +} beforeEach(() => { vi.clearAllMocks(); + isManualDuplicate.mockResolvedValue({ isDuplicate: false }); transactionRepository.create.mockResolvedValue({ id: 1, amount: '-50', date: '2026-01-15' }); transactionRepository.update.mockResolvedValue({ id: 1, amount: '10', date: '2026-07-01' }); }); describe('POST / — validation pins', () => { it('emits the 400 VALIDATION_ERROR envelope for a missing-fields body', async () => { - const res = mockResponse(); - await callHandler(routeHandlers['post:/'], { body: {} }, res); - expect(res.status).toHaveBeenCalledWith(400); - const body = res.json.mock.calls[0][0]; - expect(body.ok).toBe(false); - expect(body.error.code).toBe('VALIDATION_ERROR'); - expect(body.error.message).toContain('Missing required fields'); + const res = await expectValidationError(post({})); + expect(res.body.error.message).toContain('Missing required fields'); + // The failure envelope carries the request id injected by requestId middleware. + expect(res.body.meta.requestId).toEqual(expect.any(String)); + expect(res.headers['x-request-id']).toBe(res.body.meta.requestId); }); it('rejects when any required field is absent', async () => { for (const missing of ['transaction_date', 'bank_account', 'recipient_id', 'amount']) { const body = validPostBody(); delete body[missing]; - await expect(runPost(body)).rejects.toBeInstanceOf(ValidationError); + await expectValidationError(post(body)); } expect(transactionRepository.create).not.toHaveBeenCalled(); }); it('falls back from transaction_date to date', async () => { - await runPost({ ...validPostBody(), transaction_date: undefined, date: '2026-02-03' }); + const body = validPostBody(); + delete body.transaction_date; + await post({ ...body, date: '2026-02-03' }).expect(201); expect(transactionRepository.create).toHaveBeenCalledWith( expect.objectContaining({ transaction_date: '2026-02-03' }), ); }); it('rejects zero, non-numeric, and out-of-range amounts', async () => { - for (const amount of [0, '0', 'abc', Infinity, -Infinity, 1e12 + 1, -1e13]) { - await expect(runPost({ ...validPostBody(), amount })).rejects.toBeInstanceOf(ValidationError); + // Infinity/-Infinity are not representable in JSON, so they arrive as the + // string forms a client would actually send. + for (const amount of [0, '0', 'abc', 'Infinity', '-Infinity', 1e12 + 1, -1e13]) { + await expectValidationError(post({ ...validPostBody(), amount })); } expect(transactionRepository.create).not.toHaveBeenCalled(); }); it('passes an accepted numeric-string amount through RAW (no coercion)', async () => { - await runPost({ ...validPostBody(), amount: '-12.5' }); + await post({ ...validPostBody(), amount: '-12.5' }).expect(201); expect(transactionRepository.create).toHaveBeenCalledWith( expect.objectContaining({ amount: '-12.5' }), ); @@ -131,53 +120,53 @@ describe('POST / — validation pins', () => { it('rejects non-positive-integer recipient ids', async () => { for (const recipient_id of [1.5, 'abc', -1, '2.5']) { - await expect(runPost({ ...validPostBody(), recipient_id })).rejects.toBeInstanceOf(ValidationError); + await expectValidationError(post({ ...validPostBody(), recipient_id })); } }); it('passes an accepted numeric-string recipient_id through RAW', async () => { - await runPost({ ...validPostBody(), recipient_id: '5' }); + await post({ ...validPostBody(), recipient_id: '5' }).expect(201); expect(transactionRepository.create).toHaveBeenCalledWith( expect.objectContaining({ recipient_id: '5' }), ); }); it('rejects non-array tags, defaults absent tags to null, passes arrays through', async () => { - await expect(runPost({ ...validPostBody(), tags: 'x' })).rejects.toBeInstanceOf(ValidationError); - await expect(runPost({ ...validPostBody(), tags: null })).rejects.toBeInstanceOf(ValidationError); + await expectValidationError(post({ ...validPostBody(), tags: 'x' })); + await expectValidationError(post({ ...validPostBody(), tags: null })); - await runPost(validPostBody()); + await post(validPostBody()).expect(201); expect(transactionRepository.create).toHaveBeenCalledWith( expect.objectContaining({ tags: null }), ); - await runPost({ ...validPostBody(), tags: ['a', 'b'] }); + await post({ ...validPostBody(), tags: ['a', 'b'] }).expect(201); expect(transactionRepository.create).toHaveBeenLastCalledWith( expect.objectContaining({ tags: ['a', 'b'] }), ); }); it('normalises currency to uppercase ISO and rejects free text', async () => { - await expect(runPost({ ...validPostBody(), currency: 'euro' })).rejects.toBeInstanceOf(ValidationError); + await expectValidationError(post({ ...validPostBody(), currency: 'euro' })); - await runPost({ ...validPostBody(), currency: 'usd' }); + await post({ ...validPostBody(), currency: 'usd' }).expect(201); expect(transactionRepository.create).toHaveBeenCalledWith( expect.objectContaining({ currency: 'USD' }), ); - await runPost(validPostBody()); + await post(validPostBody()).expect(201); expect(transactionRepository.create).toHaveBeenLastCalledWith( expect.objectContaining({ currency: undefined }), ); }); it('rejects a bank_account longer than 100 characters', async () => { - await expect(runPost({ ...validPostBody(), bank_account: 'a'.repeat(101) })) - .rejects.toThrow(/bank_account/); + const res = await expectValidationError(post({ ...validPostBody(), bank_account: 'a'.repeat(101) })); + expect(res.body.error.message).toMatch(/bank_account/); }); it('still records the raw-transaction mirror with the accepted values', async () => { - await runPost({ ...validPostBody(), memo: 'm', category_id: 4 }); + await post({ ...validPostBody(), memo: 'm', category_id: 4 }).expect(201); expect(recordManualRawTransaction).toHaveBeenCalledWith( expect.objectContaining({ date: '2026-01-15', recipientId: 1, categoryId: 4, memo: 'm' }), ); @@ -186,11 +175,11 @@ describe('POST / — validation pins', () => { describe('PATCH /:id — validation pins', () => { it('rejects a cleared or free-text currency but uppercases a valid one', async () => { - await expect(runPatch({ currency: '' })).rejects.toBeInstanceOf(ValidationError); - await expect(runPatch({ currency: null })).rejects.toBeInstanceOf(ValidationError); - await expect(runPatch({ currency: 'euro' })).rejects.toBeInstanceOf(ValidationError); + await expectValidationError(patch({ currency: '' })); + await expectValidationError(patch({ currency: null })); + await expectValidationError(patch({ currency: 'euro' })); - await runPatch({ currency: 'usd' }); + await patch({ currency: 'usd' }).expect(200); expect(transactionRepository.update).toHaveBeenCalledWith( 1, expect.objectContaining({ currency: 'USD' }), @@ -198,7 +187,7 @@ describe('PATCH /:id — validation pins', () => { }); it('allows a zero amount on PATCH (unlike POST) and coerces it to a number', async () => { - await runPatch({ amount: '0' }); + await patch({ amount: '0' }).expect(200); expect(transactionRepository.update).toHaveBeenCalledWith( 1, expect.objectContaining({ amount: 0 }), @@ -206,14 +195,14 @@ describe('PATCH /:id — validation pins', () => { }); it('rejects an out-of-range amount', async () => { - await expect(runPatch({ amount: 1e13 })).rejects.toBeInstanceOf(ValidationError); - await expect(runPatch({ amount: Infinity })).rejects.toBeInstanceOf(ValidationError); + await expectValidationError(patch({ amount: 1e13 })); + await expectValidationError(patch({ amount: 'Infinity' })); }); it('caps bank_account at 100 chars but lets null through untouched', async () => { - await expect(runPatch({ bank_account: 'a'.repeat(101) })).rejects.toBeInstanceOf(ValidationError); + await expectValidationError(patch({ bank_account: 'a'.repeat(101) })); - await runPatch({ bank_account: null }); + await patch({ bank_account: null }).expect(200); expect(transactionRepository.update).toHaveBeenCalledWith( 1, expect.objectContaining({ bank_account: null }), @@ -221,9 +210,9 @@ describe('PATCH /:id — validation pins', () => { }); it('rejects non-array tags and passes arrays through', async () => { - await expect(runPatch({ tags: 'x' })).rejects.toBeInstanceOf(ValidationError); + await expectValidationError(patch({ tags: 'x' })); - await runPatch({ tags: ['a'] }); + await patch({ tags: ['a'] }).expect(200); expect(transactionRepository.update).toHaveBeenCalledWith( 1, expect.objectContaining({ tags: ['a'] }), @@ -231,7 +220,7 @@ describe('PATCH /:id — validation pins', () => { }); it('keeps the boundary loose: unvalidated fields pass through untouched', async () => { - await runPatch({ memo: 'hi', is_active: 'yes', comment: 7 }); + await patch({ memo: 'hi', is_active: 'yes', comment: 7 }).expect(200); expect(transactionRepository.update).toHaveBeenCalledWith( 1, expect.objectContaining({ memo: 'hi', is_active: 'yes', comment: 7 }), @@ -239,7 +228,7 @@ describe('PATCH /:id — validation pins', () => { }); it('absent fields stay absent in the patch handed to the repository', async () => { - await runPatch({ memo: 'only-this' }); + await patch({ memo: 'only-this' }).expect(200); const patchArg = transactionRepository.update.mock.calls[0][1]; expect('amount' in patchArg).toBe(false); expect('transaction_date' in patchArg).toBe(false); @@ -249,7 +238,7 @@ describe('PATCH /:id — validation pins', () => { }); it('coerces numeric-string FK ids to numbers', async () => { - await runPatch({ recipient_id: '7', category_id: '3' }); + await patch({ recipient_id: '7', category_id: '3' }).expect(200); expect(transactionRepository.update).toHaveBeenCalledWith( 1, expect.objectContaining({ recipient_id: 7, category_id: 3 }), @@ -257,7 +246,7 @@ describe('PATCH /:id — validation pins', () => { }); it('strips read-only keys (id, created_at, links) before the repository', async () => { - await runPatch({ id: 99, created_at: 'x', links: ['a'], memo: 'ok' }); + await patch({ id: 99, created_at: 'x', links: ['a'], memo: 'ok' }).expect(200); const patchArg = transactionRepository.update.mock.calls[0][1]; expect('id' in patchArg).toBe(false); expect('created_at' in patchArg).toBe(false); @@ -273,41 +262,35 @@ describe('POST /bulk-tag — validation pins', () => { { transaction_ids: [1], remove_slugs: { a: 1 } }, { transaction_ids: [1], add_slugs: null }, ]) { - const res = mockResponse(); - await callHandler(routeHandlers['post:/bulk-tag'], { body }, res); - expect(res.status).toHaveBeenCalledWith(400); + await expectValidationError(bulkTag(body)); } }); it('rejects remove_slugs above 50 entries', async () => { - const res = mockResponse(); - const body = { transaction_ids: [1], remove_slugs: Array.from({ length: 51 }, (_, i) => `t-${i}`) }; - await callHandler(routeHandlers['post:/bulk-tag'], { body }, res); - expect(res.status).toHaveBeenCalledWith(400); + await expectValidationError(bulkTag({ + transaction_ids: [1], + remove_slugs: Array.from({ length: 51 }, (_, i) => `t-${i}`), + })); }); it('checks the both-empty rule before filtering invalid ids', async () => { - const res = mockResponse(); - const body = { transaction_ids: ['abc'], add_slugs: [], remove_slugs: [] }; - await callHandler(routeHandlers['post:/bulk-tag'], { body }, res); - expect(res.status).toHaveBeenCalledWith(400); - expect(res.json.mock.calls[0][0].error.message).toMatch(/at least one/i); + const res = await expectValidationError(bulkTag({ + transaction_ids: ['abc'], add_slugs: [], remove_slugs: [], + })); + expect(res.body.error.message).toMatch(/at least one/i); }); it('rejects when transaction_ids contains no int4-valid ids', async () => { - const res = mockResponse(); - const body = { transaction_ids: ['abc', 0, -2, 2 ** 31], add_slugs: ['x'] }; - await callHandler(routeHandlers['post:/bulk-tag'], { body }, res); - expect(res.status).toHaveBeenCalledWith(400); - expect(res.json.mock.calls[0][0].error.message).toContain('no valid IDs'); + const res = await expectValidationError(bulkTag({ + transaction_ids: ['abc', 0, -2, 2 ** 31], add_slugs: ['x'], + })); + expect(res.body.error.message).toContain('no valid IDs'); }); }); describe('POST /bulk-update — validation pins', () => { it('rejects an array fields value', async () => { - const res = mockResponse(); - await callHandler(routeHandlers['post:/bulk-update'], { body: { ids: [1], fields: [] } }, res); - expect(res.status).toHaveBeenCalledWith(400); + await expectValidationError(bulkUpdate({ ids: [1], fields: [] })); }); it('keeps FK ids strict: numeric strings and floats are rejected', async () => { @@ -318,10 +301,7 @@ describe('POST /bulk-update — validation pins', () => { { recipient_id: 0 }, { recipient_id: true }, ]) { - const res = mockResponse(); - await callHandler(routeHandlers['post:/bulk-update'], { body: { ids: [1], fields } }, res); - expect(res.status).toHaveBeenCalledWith(400); - expect(res.json.mock.calls[0][0].error.code).toBe('VALIDATION_ERROR'); + await expectValidationError(bulkUpdate({ ids: [1], fields })); } }); }); diff --git a/apps/node-backend/tests/routes/watchlist.test.js b/apps/node-backend/tests/routes/watchlist.test.js index 2d9b8adc..4314886d 100644 --- a/apps/node-backend/tests/routes/watchlist.test.js +++ b/apps/node-backend/tests/routes/watchlist.test.js @@ -1,14 +1,15 @@ +/** + * Watchlist route tests. + * + * Runs against the REAL router mounted on a throwaway Express app (see + * tests/helpers/routeApp.js) — validateIdParam is no longer stubbed. + */ import { beforeEach, describe, expect, it, vi } from 'vitest'; import { mockLogger } from '../helpers/mockLogger.js'; -import { createMockRouter, createMockResponse } from '../helpers/routeHarness.js'; - -const { router: mockRouter, handlers: routeHandlers } = createMockRouter(); - -vi.mock('express', () => ({ - default: { Router: () => mockRouter }, - Router: () => mockRouter, -})); +import { routeAgent, errEnvelope } from '../helpers/routeApp.js'; +// The route imports its repository through services/watchlistService.js, which +// re-exports this named binding — mocking the repository here intercepts it. vi.mock('../../src/repositories/watchlistRepository.js', () => ({ watchlistRepository: { getAllWithCount: vi.fn(), @@ -24,9 +25,11 @@ vi.mock('../../src/config/logger.js', () => ({ })); import { watchlistRepository } from '../../src/repositories/watchlistRepository.js'; -import { ValidationError, NotFoundError } from '../../src/middleware/errorHandler.js'; -await import('../../src/routes/watchlist.js'); +const { default: watchlistRouter } = await import('../../src/routes/watchlist.js'); + +const api = routeAgent(watchlistRouter, { mountPath: '/api/watchlist' }); +const BASE = '/api/watchlist'; describe('Watchlist Routes', () => { beforeEach(() => { @@ -37,73 +40,69 @@ describe('Watchlist Routes', () => { it('clamps pagination and forwards asset class filter', async () => { watchlistRepository.getAllWithCount.mockResolvedValue({ rows: [{ id: 1 }], total: 1 }); - const req = { query: { limit: '10000', offset: '-15', asset_class: 'stocks' } }; - const res = mockResponse(); - - await routeHandlers['get:/'](req, res); + const res = await api.get(`${BASE}?limit=10000&offset=-15&asset_class=stocks`).expect(200); expect(watchlistRepository.getAllWithCount).toHaveBeenCalledWith({ limit: 5000, offset: 0, assetClass: 'stocks', }); - expect(res.json).toHaveBeenCalledWith({ - ok: true, - data: { items: [{ id: 1 }], total: 1, limit: 5000, offset: 0 }, - }); + expect(res.body.data).toEqual({ items: [{ id: 1 }], total: 1, limit: 5000, offset: 0 }); }); - it('propagates error when repository throws', async () => { + it('answers a 500 when repository throws', async () => { watchlistRepository.getAllWithCount.mockRejectedValue(new Error('db exploded')); - const req = { query: {} }; - const res = mockResponse(); - - await expect(routeHandlers['get:/'](req, res)).rejects.toThrow('db exploded'); + const res = await api.get(BASE).expect(500); + expect(res.body.error.message).toBe('db exploded'); }); }); describe('GET /:id', () => { - it('throws NotFoundError for missing watchlist item', async () => { + it('returns a 404 NOT_FOUND envelope for missing watchlist item', async () => { watchlistRepository.getById.mockResolvedValue(null); - const req = { params: { id: '7' } }; - const res = mockResponse(); + const res = await api.get(`${BASE}/7`).expect(404); + expect(res.body).toEqual(errEnvelope({ code: 'NOT_FOUND' })); + }); - await expect(routeHandlers['get:/:id'](req, res)).rejects.toBeInstanceOf(NotFoundError); + it('rejects a non-integer :id via the real validateIdParam guard', async () => { + // Previously this suite ran through the mock-router harness, which + // silently dropped validateIdParam from the chain, so the guard was + // never actually tested. + const res = await api.get(`${BASE}/abc`).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); + expect(watchlistRepository.getById).not.toHaveBeenCalled(); }); }); describe('POST /', () => { - it('throws ValidationError when required fields are missing', async () => { - const req = { body: { name: 'ETF' } }; - const res = mockResponse(); - - await expect(routeHandlers['post:/'](req, res)).rejects.toBeInstanceOf(ValidationError); + it('returns a 400 VALIDATION_ERROR envelope when required fields are missing', async () => { + const res = await api.post(BASE).send({ name: 'ETF' }).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); expect(watchlistRepository.create).not.toHaveBeenCalled(); }); it('creates watchlist item', async () => { watchlistRepository.create.mockResolvedValue({ id: 9, name: 'ETF Idea' }); - const req = { - body: { - name: 'ETF Idea', - symbol: 'VUSA.AS', - asset_class: 'etf', - target_price: 100, - currency: 'EUR', - notes: 'watch this', - price_provider_id: 'yahoo', - }, + const body = { + name: 'ETF Idea', + symbol: 'VUSA.AS', + asset_class: 'etf', + target_price: 100, + currency: 'EUR', + notes: 'watch this', + price_provider_id: 'yahoo', }; - const res = mockResponse(); - await routeHandlers['post:/'](req, res); + const res = await api.post(BASE).send(body).expect(201); - expect(watchlistRepository.create).toHaveBeenCalledWith(req.body); - expect(res.status).toHaveBeenCalledWith(201); - expect(res.json).toHaveBeenCalledWith({ ok: true, data: { id: 9, name: 'ETF Idea' } }); + expect(watchlistRepository.create).toHaveBeenCalledWith({ + name: 'ETF Idea', symbol: 'VUSA.AS', asset_class: 'etf', target_price: 100, + currency: 'EUR', notes: 'watch this', price_provider_id: 'yahoo', added_price: undefined, + }); + expect(res.body.data).toEqual({ id: 9, name: 'ETF Idea' }); }); }); @@ -112,80 +111,75 @@ describe('Watchlist Routes', () => { name: 'NVIDIA', symbol: 'NVDA', asset_class: 'stock', target_price: 100, currency: 'USD', }; - it('rejects non-numeric target_price with ValidationError (not a DB 500)', async () => { - const req = { body: { ...validBody, target_price: 'abc' } }; - await expect(routeHandlers['post:/'](req, mockResponse())).rejects.toBeInstanceOf(ValidationError); + it('rejects non-numeric target_price with a 400 VALIDATION_ERROR envelope (not a DB 500)', async () => { + const res = await api.post(BASE).send({ ...validBody, target_price: 'abc' }).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); expect(watchlistRepository.create).not.toHaveBeenCalled(); }); it('rejects negative target_price', async () => { - const req = { body: { ...validBody, target_price: -5 } }; - await expect(routeHandlers['post:/'](req, mockResponse())).rejects.toBeInstanceOf(ValidationError); + await api.post(BASE).send({ ...validBody, target_price: -5 }).expect(400); }); it('rejects a zero target_price (meaningless for the at-or-below alert)', async () => { - const req = { body: { ...validBody, target_price: 0 } }; - await expect(routeHandlers['post:/'](req, mockResponse())).rejects.toBeInstanceOf(ValidationError); + await api.post(BASE).send({ ...validBody, target_price: 0 }).expect(400); expect(watchlistRepository.create).not.toHaveBeenCalled(); }); it('rejects target_price beyond the NUMERIC(18,6) cap (was a DB overflow 500)', async () => { - const req = { body: { ...validBody, target_price: 1e15 } }; - await expect(routeHandlers['post:/'](req, mockResponse())).rejects.toBeInstanceOf(ValidationError); + await api.post(BASE).send({ ...validBody, target_price: 1e15 }).expect(400); expect(watchlistRepository.create).not.toHaveBeenCalled(); }); it('rejects Infinity target_price', async () => { - const req = { body: { ...validBody, target_price: Infinity } }; - await expect(routeHandlers['post:/'](req, mockResponse())).rejects.toBeInstanceOf(ValidationError); + // Infinity is not valid JSON — JSON.stringify(Infinity) === 'null', so + // send the raw body with a JSON-invalid literal to keep the original + // intent (a non-finite value reaching the route) instead of silently + // becoming `target_price: null`. + await api.post(BASE) + .set('Content-Type', 'application/json') + .send(JSON.stringify({ ...validBody, target_price: 'Infinity' })) + .expect(400); }); it('rejects added_price beyond the NUMERIC(18,6) cap', async () => { - const req = { body: { ...validBody, added_price: 1e15 } }; - await expect(routeHandlers['post:/'](req, mockResponse())).rejects.toBeInstanceOf(ValidationError); + await api.post(BASE).send({ ...validBody, added_price: 1e15 }).expect(400); }); it('rejects an over-length name (VARCHAR(200)) before the DB 22001', async () => { - const req = { body: { ...validBody, name: 'x'.repeat(201) } }; - await expect(routeHandlers['post:/'](req, mockResponse())).rejects.toBeInstanceOf(ValidationError); + await api.post(BASE).send({ ...validBody, name: 'x'.repeat(201) }).expect(400); expect(watchlistRepository.create).not.toHaveBeenCalled(); }); it('rejects an over-length symbol (VARCHAR(20))', async () => { - const req = { body: { ...validBody, symbol: 'A'.repeat(21) } }; - await expect(routeHandlers['post:/'](req, mockResponse())).rejects.toBeInstanceOf(ValidationError); + await api.post(BASE).send({ ...validBody, symbol: 'A'.repeat(21) }).expect(400); expect(watchlistRepository.create).not.toHaveBeenCalled(); }); it('rejects unknown asset_class', async () => { - const req = { body: { ...validBody, asset_class: 'beanie-babies' } }; - await expect(routeHandlers['post:/'](req, mockResponse())).rejects.toBeInstanceOf(ValidationError); + await api.post(BASE).send({ ...validBody, asset_class: 'beanie-babies' }).expect(400); }); it('rejects malformed currency', async () => { - const req = { body: { ...validBody, currency: 'EURO' } }; - await expect(routeHandlers['post:/'](req, mockResponse())).rejects.toBeInstanceOf(ValidationError); + await api.post(BASE).send({ ...validBody, currency: 'EURO' }).expect(400); }); it('normalises a lower-case currency to uppercase before the repository', async () => { watchlistRepository.create.mockResolvedValue({ id: 1 }); - const req = { body: { ...validBody, currency: 'usd' } }; - await routeHandlers['post:/'](req, mockResponse()); + await api.post(BASE).send({ ...validBody, currency: 'usd' }).expect(201); expect(watchlistRepository.create).toHaveBeenCalledWith( expect.objectContaining({ currency: 'USD' }), ); }); it('rejects a whitespace-only name (truthy, so the POST presence check let it through)', async () => { - const req = { body: { ...validBody, name: ' ' } }; - await expect(routeHandlers['post:/'](req, mockResponse())).rejects.toBeInstanceOf(ValidationError); + await api.post(BASE).send({ ...validBody, name: ' ' }).expect(400); expect(watchlistRepository.create).not.toHaveBeenCalled(); }); it('coerces numeric-string target_price before reaching the repository', async () => { watchlistRepository.create.mockResolvedValue({ id: 1 }); - const req = { body: { ...validBody, target_price: '123.45' } }; - await routeHandlers['post:/'](req, mockResponse()); + await api.post(BASE).send({ ...validBody, target_price: '123.45' }).expect(201); expect(watchlistRepository.create).toHaveBeenCalledWith( expect.objectContaining({ target_price: 123.45 }), ); @@ -195,8 +189,7 @@ describe('Watchlist Routes', () => { watchlistRepository.create.mockResolvedValue({ id: 1 }); const name = 'n'.repeat(200); const symbol = 'S'.repeat(20); - const req = { body: { ...validBody, name, symbol } }; - await routeHandlers['post:/'](req, mockResponse()); + await api.post(BASE).send({ ...validBody, name, symbol }).expect(201); expect(watchlistRepository.create).toHaveBeenCalledWith( expect.objectContaining({ name, symbol }), ); @@ -206,39 +199,33 @@ describe('Watchlist Routes', () => { watchlistRepository.create.mockResolvedValue({ id: 1 }); const MAX_PRICE = 999_999_999_999; - await routeHandlers['post:/']({ body: { ...validBody, target_price: MAX_PRICE } }, mockResponse()); + await api.post(BASE).send({ ...validBody, target_price: MAX_PRICE }).expect(201); expect(watchlistRepository.create).toHaveBeenCalledWith( expect.objectContaining({ target_price: MAX_PRICE }), ); - await expect( - routeHandlers['post:/']({ body: { ...validBody, target_price: MAX_PRICE + 1 } }, mockResponse()), - ).rejects.toBeInstanceOf(ValidationError); + await api.post(BASE).send({ ...validBody, target_price: MAX_PRICE + 1 }).expect(400); }); it('rejects an over-length price_provider_id (VARCHAR(200))', async () => { - const req = { body: { ...validBody, price_provider_id: 'p'.repeat(201) } }; - await expect(routeHandlers['post:/'](req, mockResponse())).rejects.toBeInstanceOf(ValidationError); + await api.post(BASE).send({ ...validBody, price_provider_id: 'p'.repeat(201) }).expect(400); }); it('trims and uppercases a padded lower-case currency', async () => { watchlistRepository.create.mockResolvedValue({ id: 1 }); - const req = { body: { ...validBody, currency: ' usd ' } }; - await routeHandlers['post:/'](req, mockResponse()); + await api.post(BASE).send({ ...validBody, currency: ' usd ' }).expect(201); expect(watchlistRepository.create).toHaveBeenCalledWith( expect.objectContaining({ currency: 'USD' }), ); }); it('rejects an empty-string currency (explicit key must carry a real code)', async () => { - const req = { body: { ...validBody, currency: '' } }; - await expect(routeHandlers['post:/'](req, mockResponse())).rejects.toBeInstanceOf(ValidationError); + await api.post(BASE).send({ ...validBody, currency: '' }).expect(400); }); it('accepts the metals asset_class', async () => { watchlistRepository.create.mockResolvedValue({ id: 1 }); - const req = { body: { ...validBody, asset_class: 'metals' } }; - await routeHandlers['post:/'](req, mockResponse()); + await api.post(BASE).send({ ...validBody, asset_class: 'metals' }).expect(201); expect(watchlistRepository.create).toHaveBeenCalledWith( expect.objectContaining({ asset_class: 'metals' }), ); @@ -247,63 +234,56 @@ describe('Watchlist Routes', () => { it('accepts a zero added_price (only target_price has the >0 rule) and coerces strings', async () => { watchlistRepository.create.mockResolvedValue({ id: 1 }); - await routeHandlers['post:/']({ body: { ...validBody, added_price: 0 } }, mockResponse()); + await api.post(BASE).send({ ...validBody, added_price: 0 }).expect(201); expect(watchlistRepository.create).toHaveBeenCalledWith( expect.objectContaining({ added_price: 0 }), ); - await routeHandlers['post:/']({ body: { ...validBody, added_price: '12.5' } }, mockResponse()); + await api.post(BASE).send({ ...validBody, added_price: '12.5' }).expect(201); expect(watchlistRepository.create).toHaveBeenCalledWith( expect.objectContaining({ added_price: 12.5 }), ); }); it('rejects a non-numeric added_price', async () => { - const req = { body: { ...validBody, added_price: 'soon' } }; - await expect(routeHandlers['post:/'](req, mockResponse())).rejects.toBeInstanceOf(ValidationError); + await api.post(BASE).send({ ...validBody, added_price: 'soon' }).expect(400); }); }); describe('PATCH /:id field validation', () => { it('rejects non-numeric target_price before hitting the repository', async () => { - const req = { params: { id: '1' }, body: { target_price: 'abc' } }; - await expect(routeHandlers['patch:/:id'](req, mockResponse())).rejects.toBeInstanceOf(ValidationError); + const res = await api.patch(`${BASE}/1`).send({ target_price: 'abc' }).expect(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); expect(watchlistRepository.update).not.toHaveBeenCalled(); }); it('rejects unknown asset_class on partial update', async () => { - const req = { params: { id: '1' }, body: { asset_class: 'nft' } }; - await expect(routeHandlers['patch:/:id'](req, mockResponse())).rejects.toBeInstanceOf(ValidationError); + await api.patch(`${BASE}/1`).send({ asset_class: 'nft' }).expect(400); }); it('allows partial updates that omit typed fields', async () => { watchlistRepository.update.mockResolvedValue({ id: 1, notes: 'watch earnings' }); - const req = { params: { id: '1' }, body: { notes: 'watch earnings' } }; - await routeHandlers['patch:/:id'](req, mockResponse()); + await api.patch(`${BASE}/1`).send({ notes: 'watch earnings' }).expect(200); expect(watchlistRepository.update).toHaveBeenCalledWith(1, { notes: 'watch earnings' }); }); it('rejects an empty name on PATCH (400, not a persisted blank label)', async () => { - const req = { params: { id: '1' }, body: { name: '' } }; - await expect(routeHandlers['patch:/:id'](req, mockResponse())).rejects.toBeInstanceOf(ValidationError); + await api.patch(`${BASE}/1`).send({ name: '' }).expect(400); expect(watchlistRepository.update).not.toHaveBeenCalled(); }); it('rejects a whitespace-only name on PATCH', async () => { - const req = { params: { id: '1' }, body: { name: ' ' } }; - await expect(routeHandlers['patch:/:id'](req, mockResponse())).rejects.toBeInstanceOf(ValidationError); + await api.patch(`${BASE}/1`).send({ name: ' ' }).expect(400); expect(watchlistRepository.update).not.toHaveBeenCalled(); }); it('rejects a null name on PATCH', async () => { - const req = { params: { id: '1' }, body: { name: null } }; - await expect(routeHandlers['patch:/:id'](req, mockResponse())).rejects.toBeInstanceOf(ValidationError); + await api.patch(`${BASE}/1`).send({ name: null }).expect(400); }); it('coerces a numeric-string target_price on PATCH', async () => { watchlistRepository.update.mockResolvedValue({ id: 1 }); - const req = { params: { id: '1' }, body: { target_price: '50.5' } }; - await routeHandlers['patch:/:id'](req, mockResponse()); + await api.patch(`${BASE}/1`).send({ target_price: '50.5' }).expect(200); expect(watchlistRepository.update).toHaveBeenCalledWith( 1, expect.objectContaining({ target_price: 50.5 }), @@ -311,14 +291,12 @@ describe('Watchlist Routes', () => { }); it('rejects a zero target_price on PATCH', async () => { - const req = { params: { id: '1' }, body: { target_price: 0 } }; - await expect(routeHandlers['patch:/:id'](req, mockResponse())).rejects.toBeInstanceOf(ValidationError); + await api.patch(`${BASE}/1`).send({ target_price: 0 }).expect(400); }); it('uppercases currency on PATCH', async () => { watchlistRepository.update.mockResolvedValue({ id: 1 }); - const req = { params: { id: '1' }, body: { currency: 'gbp' } }; - await routeHandlers['patch:/:id'](req, mockResponse()); + await api.patch(`${BASE}/1`).send({ currency: 'gbp' }).expect(200); expect(watchlistRepository.update).toHaveBeenCalledWith( 1, expect.objectContaining({ currency: 'GBP' }), @@ -327,8 +305,7 @@ describe('Watchlist Routes', () => { it('passes a null currency through untouched on PATCH', async () => { watchlistRepository.update.mockResolvedValue({ id: 1 }); - const req = { params: { id: '1' }, body: { currency: null } }; - await routeHandlers['patch:/:id'](req, mockResponse()); + await api.patch(`${BASE}/1`).send({ currency: null }).expect(200); expect(watchlistRepository.update).toHaveBeenCalledWith( 1, expect.objectContaining({ currency: null }), @@ -336,16 +313,14 @@ describe('Watchlist Routes', () => { }); it('rejects an over-length symbol on PATCH', async () => { - const req = { params: { id: '1' }, body: { symbol: 'A'.repeat(21) } }; - await expect(routeHandlers['patch:/:id'](req, mockResponse())).rejects.toBeInstanceOf(ValidationError); + await api.patch(`${BASE}/1`).send({ symbol: 'A'.repeat(21) }).expect(400); }); it('passes unvalidated fields through to the repository untouched', async () => { // The repository allow-list (not the route) is what drops unknown keys — // the route must forward them so notes/other allow-listed columns update. watchlistRepository.update.mockResolvedValue({ id: 1 }); - const req = { params: { id: '1' }, body: { notes: 'hold', unknown_field: 'kept' } }; - await routeHandlers['patch:/:id'](req, mockResponse()); + await api.patch(`${BASE}/1`).send({ notes: 'hold', unknown_field: 'kept' }).expect(200); expect(watchlistRepository.update).toHaveBeenCalledWith( 1, { notes: 'hold', unknown_field: 'kept' }, @@ -354,8 +329,7 @@ describe('Watchlist Routes', () => { it('allows an explicit null added_price on PATCH (only non-null values are frozen)', async () => { watchlistRepository.update.mockResolvedValue({ id: 1 }); - const req = { params: { id: '1' }, body: { added_price: null } }; - await routeHandlers['patch:/:id'](req, mockResponse()); + await api.patch(`${BASE}/1`).send({ added_price: null }).expect(200); expect(watchlistRepository.update).toHaveBeenCalledWith( 1, expect.objectContaining({ added_price: null }), @@ -366,47 +340,33 @@ describe('Watchlist Routes', () => { // added_price is an add-time snapshot: the repository update allow-list omits // it, so before the fix a valid value validated fine yet never persisted (a // no-op). It must now surface a 400 rather than the silent no-op. - const req = { params: { id: '1' }, body: { added_price: 123.45 } }; - await expect(routeHandlers['patch:/:id'](req, mockResponse())).rejects.toBeInstanceOf(ValidationError); + await api.patch(`${BASE}/1`).send({ added_price: 123.45 }).expect(400); expect(watchlistRepository.update).not.toHaveBeenCalled(); }); }); describe('PATCH /:id', () => { - it('throws NotFoundError when updating a missing item', async () => { + it('returns a 404 NOT_FOUND envelope when updating a missing item', async () => { watchlistRepository.update.mockResolvedValue(null); - const req = { params: { id: '99' }, body: { notes: 'updated' } }; - const res = mockResponse(); - - await expect(routeHandlers['patch:/:id'](req, res)).rejects.toBeInstanceOf(NotFoundError); + const res = await api.patch(`${BASE}/99`).send({ notes: 'updated' }).expect(404); + expect(res.body).toEqual(errEnvelope({ code: 'NOT_FOUND' })); }); }); describe('DELETE /:id', () => { - it('throws NotFoundError when delete returns false', async () => { + it('returns a 404 NOT_FOUND envelope when delete returns false', async () => { watchlistRepository.delete.mockResolvedValue(false); - const req = { params: { id: '33' } }; - const res = mockResponse(); - - await expect(routeHandlers['delete:/:id'](req, res)).rejects.toBeInstanceOf(NotFoundError); + const res = await api.delete(`${BASE}/33`).expect(404); + expect(res.body).toEqual(errEnvelope({ code: 'NOT_FOUND' })); }); - it('returns 204 on successful delete', async () => { + it('returns 204 with no body on successful delete', async () => { watchlistRepository.delete.mockResolvedValue(true); - const req = { params: { id: '33' } }; - const res = mockResponse(); - - await routeHandlers['delete:/:id'](req, res); - - expect(res.status).toHaveBeenCalledWith(204); - expect(res.send).toHaveBeenCalled(); + const res = await api.delete(`${BASE}/33`).expect(204); + expect(res.text).toBe(''); }); }); }); - -function mockResponse() { - return createMockResponse(); -} diff --git a/apps/node-backend/tests/services/cashflowForecastMethods.test.js b/apps/node-backend/tests/services/cashflowForecastMethods.test.js index 46c337f8..0ccaa880 100644 --- a/apps/node-backend/tests/services/cashflowForecastMethods.test.js +++ b/apps/node-backend/tests/services/cashflowForecastMethods.test.js @@ -420,6 +420,8 @@ describe('orchestrator computeCashflowForecast', () => { const history = syntheticHistory({ days: 400 }); return { infoRepository: { + // ADR-083 cache-key input (forecast/index.js filterHash). + getIncludeTransfers: vi.fn().mockResolvedValue(false), getCashflowForecastData: vi.fn().mockResolvedValue({ history, currentActual: [], @@ -458,6 +460,8 @@ describe('orchestrator computeCashflowForecast', () => { const history = syntheticHistory({ days: 200 }); return { infoRepository: { + // ADR-083 cache-key input (forecast/index.js filterHash). + getIncludeTransfers: vi.fn().mockResolvedValue(false), getCashflowForecastData: vi.fn().mockResolvedValue({ history, currentActual: [], @@ -480,6 +484,8 @@ describe('orchestrator computeCashflowForecast', () => { const history = syntheticHistory({ days: 200 }); return { infoRepository: { + // ADR-083 cache-key input (forecast/index.js filterHash). + getIncludeTransfers: vi.fn().mockResolvedValue(false), getCashflowForecastData: vi.fn().mockResolvedValue({ history, currentActual: [], diff --git a/apps/node-backend/tests/services/cashflowForecastRolling.test.js b/apps/node-backend/tests/services/cashflowForecastRolling.test.js index b2c267c7..1ceca7ea 100644 --- a/apps/node-backend/tests/services/cashflowForecastRolling.test.js +++ b/apps/node-backend/tests/services/cashflowForecastRolling.test.js @@ -27,6 +27,8 @@ const isoOffsetFromToday = (offsetDays) => addDaysYmd(todayAppDateString(), offs vi.mock('../../src/repositories/infoRepository.js', () => ({ infoRepository: { + // ADR-083 cache-key input (forecast/index.js filterHash). + getIncludeTransfers: vi.fn(async () => false), getCashflowForecastDataRolling: vi.fn(async (historyMonths, daysBack, daysForward) => ({ history: buildHistory({ days: 400 }), currentActual: Array.from({ length: daysBack + 1 }, (_, i) => ({ diff --git a/apps/node-backend/tests/services/cashflowForecastRollingBacktest.test.js b/apps/node-backend/tests/services/cashflowForecastRollingBacktest.test.js index ec493ccd..f47d9b3f 100644 --- a/apps/node-backend/tests/services/cashflowForecastRollingBacktest.test.js +++ b/apps/node-backend/tests/services/cashflowForecastRollingBacktest.test.js @@ -43,6 +43,8 @@ const stubMethod = (id) => ({ vi.mock('../../src/repositories/infoRepository.js', () => ({ infoRepository: { + // ADR-083 cache-key input (forecast/index.js filterHash). + getIncludeTransfers: vi.fn(async () => false), getCashflowForecastDataRolling: vi.fn(async (historyMonths, daysBack, daysForward) => ({ history: buildHistory({ days: 400 }), currentActual: Array.from({ length: daysBack + 1 }, (_, i) => ({ @@ -271,6 +273,30 @@ describe('computeCashflowForecastRolling — MC cache', () => { await Promise.resolve(); expect(mcRollingCacheRepo.upsert).not.toHaveBeenCalled(); }); + + // ADR-083 `includeTransfers` changes what the forecast repositories return, + // so it must be part of the cache identity. Before it was hashed in, toggling + // the setting kept serving the pre-toggle forecast for the cache's 6h TTL. + it('includeTransfers is a cache-key input → toggling it misses the cache', async () => { + const { computeCashflowForecastRolling } = await import( + '../../src/services/calculations/forecast/index.js' + ); + const args = { daysBack: 10, daysForward: 10, mcPaths: 500, mcPercentiles: [25, 75], userId: 'u_tx' }; + + infoRepository.getIncludeTransfers.mockResolvedValue(false); + await computeCashflowForecastRolling(args); + const hashOff = mcRollingCacheRepo.get.mock.calls.at(-1)[0].filterHash; + + infoRepository.getIncludeTransfers.mockResolvedValue(true); + await computeCashflowForecastRolling(args); + const hashOn = mcRollingCacheRepo.get.mock.calls.at(-1)[0].filterHash; + + expect(hashOff).not.toBe(hashOn); + // Every other input is identical, so the difference is the toggle alone. + expect(hashOff.replace(/\|t0$/, '')).toBe(hashOn.replace(/\|t1$/, '')); + + infoRepository.getIncludeTransfers.mockResolvedValue(false); + }); }); // ─── rolling diagnostics integration tests ─────────────────────────────────── diff --git a/apps/node-backend/tests/services/sankey.test.js b/apps/node-backend/tests/services/sankey.test.js index 4c20f60a..69c33ef4 100644 --- a/apps/node-backend/tests/services/sankey.test.js +++ b/apps/node-backend/tests/services/sankey.test.js @@ -4,14 +4,21 @@ vi.mock('../../src/database/connection.js', () => ({ query: vi.fn() })); vi.mock('../../src/services/currency/currencyConversionService.js', () => ({ convertRowsToEur: vi.fn(), })); +vi.mock('../../src/repositories/infoRepositoryHelpers.js', async () => { + const actual = await vi.importActual('../../src/repositories/infoRepositoryHelpers.js'); + return { ...actual, getIncludeTransfers: vi.fn() }; +}); import { query } from '../../src/database/connection.js'; import { convertRowsToEur } from '../../src/services/currency/currencyConversionService.js'; +import { getIncludeTransfers } from '../../src/repositories/infoRepositoryHelpers.js'; import { computeSankeyFlow } from '../../src/services/calculations/aggregation/sankey.js'; beforeEach(() => { query.mockReset(); convertRowsToEur.mockReset(); + getIncludeTransfers.mockReset(); + getIncludeTransfers.mockResolvedValue(false); }); describe('computeSankeyFlow (SQL-grouped rows)', () => { @@ -48,4 +55,51 @@ describe('computeSankeyFlow (SQL-grouped rows)', () => { expect(query.mock.calls[0][0]).toContain('GROUP BY 1, 2, 3'); expect(query.mock.calls[0][0]).toContain('SUM(ABS(t.amount))'); }); + + // The emitted SQL must resolve the effective category over all THREE levels + // (own → recipient default → PRIMARY recipient's default), in the category + // JOIN *and* in the exclusion clause. With the former 2-level resolution a + // row recorded under an alias whose PRIMARY carries the default category + // landed in "Uncategorised" and survived an exclusion of that same category. + // + // The exclusion clauses are the canonical ones from + // lib/filterBuilder.buildExclusionClauses — the `-1` NULL sentinel and the + // alias-aware recipient form are the load-bearing halves (see the DB pins in + // tests/aliasCategoryResolution.db.test.js for the behaviour). Numbering + // starts at $3 because the year range owns $1/$2. + it('emits the canonical exclusion clauses, numbered after the date range', async () => { + query.mockResolvedValueOnce({ rows: [] }); + + await computeSankeyFlow({ year: 2025, excludedCategoryIds: [7], excludedRecipientIds: [9] }); + + const sql = query.mock.calls[0][0]; + expect(sql).toContain('LEFT JOIN recipients pr ON r.primary_recipient_id = pr.id'); + expect(sql).toContain( + 'LEFT JOIN categories c ON COALESCE(t.category_id, r.default_category_id, pr.default_category_id) = c.id', + ); + // NULL sentinel: without the trailing -1 a NULL effective category makes + // the predicate NULL, dropping every uncategorised row. + expect(sql).toContain( + 'COALESCE(t.category_id, r.default_category_id, pr.default_category_id, -1) NOT IN ($3)', + ); + // Alias-aware: excluding a PRIMARY must also exclude rows booked on its aliases. + expect(sql).toContain('COALESCE(r.primary_recipient_id, t.recipient_id, -1) NOT IN ($4)'); + expect(query.mock.calls[0][1]).toEqual(['2025-01-01', '2025-12-31', 7, 9]); + }); + + // ADR-083: transfers are excluded from income/spending by default, governed + // by the runtime `includeTransfers` setting — same conditional predicate as + // every sibling aggregation. Without it a savings transfer's two legs + // inflated BOTH the income and the spending side of the flow graph. + it('filters internal transfers by default and honours the includeTransfers setting', async () => { + query.mockResolvedValueOnce({ rows: [] }); + await computeSankeyFlow({ year: 2025 }); + expect(query.mock.calls[0][0]).toContain('AND t.is_transfer = false'); + + query.mockReset(); + getIncludeTransfers.mockResolvedValue(true); + query.mockResolvedValueOnce({ rows: [] }); + await computeSankeyFlow({ year: 2025 }); + expect(query.mock.calls[0][0]).not.toContain('is_transfer'); + }); }); diff --git a/apps/node-backend/tests/settingsStorage.test.js b/apps/node-backend/tests/settingsStorage.test.js index a7f2a323..4a0136e1 100644 --- a/apps/node-backend/tests/settingsStorage.test.js +++ b/apps/node-backend/tests/settingsStorage.test.js @@ -1,25 +1,21 @@ import { describe, it, expect, vi, beforeEach } from 'vitest'; import { mockLogger } from './helpers/mockLogger.js'; import { mockConnection } from './helpers/repoMocks.js'; -import { createMockRouter, createMockResponse as mockResponse } from './helpers/routeHarness.js'; +import { routeAgent, okEnvelope } from './helpers/routeApp.js'; // Mock the DB layer used by the settings repository vi.mock('../src/database/connection.js', () => mockConnection()); -const { router: mockRouter, handlers: routeHandlers } = createMockRouter(); - -vi.mock('express', () => ({ - default: { Router: () => mockRouter }, - Router: () => mockRouter, -})); - vi.mock('../src/config/logger.js', () => ({ logger: mockLogger(), })); import { query } from '../src/database/connection.js'; import settingsRepository from '../src/repositories/settingsRepository.js'; -await import('../src/routes/settings.js'); +const { default: settingsRouter } = await import('../src/routes/settings.js'); + +const api = routeAgent(settingsRouter, { mountPath: '/api/settings' }); +const BASE = '/api/settings'; describe('Settings storage and retrieval', () => { beforeEach(() => { @@ -73,12 +69,10 @@ describe('Settings storage and retrieval', () => { query.mockResolvedValue({}); const payload = { value: { excludedCategoryIds: [7], excludedRecipientIds: [8] } }; - const req = { params: { key: 'dashboard_settings' }, body: payload }; - const res = mockResponse(); - await routeHandlers['put:/:key'](req, res); + const res = await api.put(`${BASE}/dashboard_settings`).send(payload).expect(200); - expect(res.json).toHaveBeenCalledWith({ ok: true, data: { key: 'dashboard_settings', value: payload.value } }); + expect(res.body).toEqual(okEnvelope({ key: 'dashboard_settings', value: payload.value })); }); it('settingsRepository.getAll should parse JSON string values and preserve invalid JSON strings', async () => { @@ -172,58 +166,60 @@ describe('rebalance_plans setting (ADR-098 custom rebalancing plans)', () => { it('accepts a valid list of plans and upserts it', async () => { query.mockResolvedValue({}); - const req = { params: { key: 'rebalance_plans' }, body: { value: [validPlan] } }; - const res = mockResponse(); - await routeHandlers['put:/:key'](req, res); + const res = await api.put(`${BASE}/rebalance_plans`).send({ value: [validPlan] }).expect(200); - expect(res.json).toHaveBeenCalledWith({ ok: true, data: { key: 'rebalance_plans', value: [validPlan] } }); + expect(res.body).toEqual(okEnvelope({ key: 'rebalance_plans', value: [validPlan] })); }); it('accepts a plan without a cashCap', async () => { query.mockResolvedValue({}); const { cashCap: _cashCap, ...noCap } = validPlan; - const req = { params: { key: 'rebalance_plans' }, body: { value: [noCap] } }; - const res = mockResponse(); - await routeHandlers['put:/:key'](req, res); + const res = await api.put(`${BASE}/rebalance_plans`).send({ value: [noCap] }).expect(200); - expect(res.json).toHaveBeenCalledWith({ ok: true, data: { key: 'rebalance_plans', value: [noCap] } }); + expect(res.body).toEqual(okEnvelope({ key: 'rebalance_plans', value: [noCap] })); }); it('rejects a non-array value', async () => { - const req = { params: { key: 'rebalance_plans' }, body: { value: { not: 'an array' } } }; - await expect(routeHandlers['put:/:key'](req, mockResponse())).rejects.toThrow(/expected array/); + const res = await api.put(`${BASE}/rebalance_plans`).send({ value: { not: 'an array' } }).expect(400); + expect(res.body.error.message).toMatch(/expected array/); }); it('rejects a plan with a blank name', async () => { - const req = { params: { key: 'rebalance_plans' }, body: { value: [{ ...validPlan, name: ' ' }] } }; - await expect(routeHandlers['put:/:key'](req, mockResponse())).rejects.toThrow(/name must not be blank/); + const res = await api.put(`${BASE}/rebalance_plans`) + .send({ value: [{ ...validPlan, name: ' ' }] }) + .expect(400); + expect(res.body.error.message).toMatch(/name must not be blank/); }); it('rejects a plan with empty targetWeights', async () => { - const req = { params: { key: 'rebalance_plans' }, body: { value: [{ ...validPlan, targetWeights: {} }] } }; - await expect(routeHandlers['put:/:key'](req, mockResponse())).rejects.toThrow(/at least one sleeve/); + const res = await api.put(`${BASE}/rebalance_plans`) + .send({ value: [{ ...validPlan, targetWeights: {} }] }) + .expect(400); + expect(res.body.error.message).toMatch(/at least one sleeve/); }); it('rejects a negative target weight', async () => { - const req = { params: { key: 'rebalance_plans' }, body: { value: [{ ...validPlan, targetWeights: { stocks: -0.1 } }] } }; - await expect(routeHandlers['put:/:key'](req, mockResponse())).rejects.toThrow(/non-negative number/); + const res = await api.put(`${BASE}/rebalance_plans`) + .send({ value: [{ ...validPlan, targetWeights: { stocks: -0.1 } }] }) + .expect(400); + expect(res.body.error.message).toMatch(/non-negative number/); }); it('rejects a negative cashCap', async () => { - const req = { params: { key: 'rebalance_plans' }, body: { value: [{ ...validPlan, cashCap: -1 }] } }; - await expect(routeHandlers['put:/:key'](req, mockResponse())).rejects.toThrow(/cashCap must be a non-negative number/); + const res = await api.put(`${BASE}/rebalance_plans`) + .send({ value: [{ ...validPlan, cashCap: -1 }] }) + .expect(400); + expect(res.body.error.message).toMatch(/cashCap must be a non-negative number/); }); it('returns an empty list as the default when unset', async () => { query.mockResolvedValue({ rows: [] }); - const req = { params: { key: 'rebalance_plans' } }; - const res = mockResponse(); - await routeHandlers['get:/:key'](req, res); + const res = await api.get(`${BASE}/rebalance_plans`).expect(200); - expect(res.json).toHaveBeenCalledWith({ ok: true, data: { key: 'rebalance_plans', value: [] } }); + expect(res.body).toEqual(okEnvelope({ key: 'rebalance_plans', value: [] })); }); }); @@ -234,10 +230,7 @@ describe('belgian_tax_profile setting validation (TODO E6)', () => { const put = async (key, value) => { query.mockResolvedValue({}); - const req = { params: { key }, body: { value } }; - const res = mockResponse(); - await routeHandlers['put:/:key'](req, res); - return res; + return api.put(`${BASE}/${key}`).send({ value }); }; const validProfile = { @@ -252,70 +245,90 @@ describe('belgian_tax_profile setting validation (TODO E6)', () => { it('accepts a well-formed profile', async () => { const res = await put('belgian_tax_profile', validProfile); - expect(res.json).toHaveBeenCalledWith( - expect.objectContaining({ ok: true }), - ); + expect(res.status).toBe(200); + expect(res.body.ok).toBe(true); }); it('rejects a negative communal surcharge (would become a tax credit)', async () => { - await expect(put('belgian_tax_profile', { ...validProfile, communalSurchargePercent: -7 })) - .rejects.toMatchObject({ name: 'ValidationError' }); + const res = await put('belgian_tax_profile', { ...validProfile, communalSurchargePercent: -7 }); + expect(res.status).toBe(400); + expect(res.body.error.code).toBe('VALIDATION_ERROR'); }); it('rejects a fat-fingered 70% surcharge (10x the real 7.0)', async () => { - await expect(put('belgian_tax_profile', { ...validProfile, communalSurchargePercent: 70 })) - .rejects.toMatchObject({ name: 'ValidationError' }); + const res = await put('belgian_tax_profile', { ...validProfile, communalSurchargePercent: 70 }); + expect(res.status).toBe(400); + expect(res.body.error.code).toBe('VALIDATION_ERROR'); }); it('rejects negative and absurd money fields', async () => { - await expect(put('belgian_tax_profile', { ...validProfile, grossAnnualIncome: -50000 })) - .rejects.toMatchObject({ name: 'ValidationError' }); - await expect(put('belgian_tax_profile', { ...validProfile, medicalExpenses: 1e15 })) - .rejects.toMatchObject({ name: 'ValidationError' }); - await expect(put('belgian_tax_profile', { ...validProfile, unionDues: 'lots' })) - .rejects.toMatchObject({ name: 'ValidationError' }); + let res = await put('belgian_tax_profile', { ...validProfile, grossAnnualIncome: -50000 }); + expect(res.status).toBe(400); + expect(res.body.error.code).toBe('VALIDATION_ERROR'); + + res = await put('belgian_tax_profile', { ...validProfile, medicalExpenses: 1e15 }); + expect(res.status).toBe(400); + expect(res.body.error.code).toBe('VALIDATION_ERROR'); + + res = await put('belgian_tax_profile', { ...validProfile, unionDues: 'lots' }); + expect(res.status).toBe(400); + expect(res.body.error.code).toBe('VALIDATION_ERROR'); }); it('rejects negative childcareEligibleDays and non-integer counts', async () => { - await expect(put('belgian_tax_profile', { ...validProfile, childcareEligibleDays: -5 })) - .rejects.toMatchObject({ name: 'ValidationError' }); - await expect(put('belgian_tax_profile', { ...validProfile, dependentChildren: 1.5 })) - .rejects.toMatchObject({ name: 'ValidationError' }); + let res = await put('belgian_tax_profile', { ...validProfile, childcareEligibleDays: -5 }); + expect(res.status).toBe(400); + expect(res.body.error.code).toBe('VALIDATION_ERROR'); + + res = await put('belgian_tax_profile', { ...validProfile, dependentChildren: 1.5 }); + expect(res.status).toBe(400); + expect(res.body.error.code).toBe('VALIDATION_ERROR'); }); it('rejects bad additional residences and non-object profiles', async () => { - await expect(put('belgian_tax_profile', { + let res = await put('belgian_tax_profile', { ...validProfile, additionalResidences: [{ cadastralIncome: -1 }], - })).rejects.toMatchObject({ name: 'ValidationError' }); - await expect(put('belgian_tax_profile', [validProfile])) - .rejects.toMatchObject({ name: 'ValidationError' }); + }); + expect(res.status).toBe(400); + expect(res.body.error.code).toBe('VALIDATION_ERROR'); + + res = await put('belgian_tax_profile', [validProfile]); + expect(res.status).toBe(400); + expect(res.body.error.code).toBe('VALIDATION_ERROR'); }); it('validates each snapshot in the year-keyed snapshots map', async () => { - const res = await put('belgian_tax_profile_snapshots_v1', { 2025: validProfile }); - expect(res.json).toHaveBeenCalledWith(expect.objectContaining({ ok: true })); + let res = await put('belgian_tax_profile_snapshots_v1', { 2025: validProfile }); + expect(res.status).toBe(200); + expect(res.body.ok).toBe(true); - await expect(put('belgian_tax_profile_snapshots_v1', { + res = await put('belgian_tax_profile_snapshots_v1', { 2025: { ...validProfile, communalSurchargePercent: -3 }, - })).rejects.toMatchObject({ name: 'ValidationError' }); + }); + expect(res.status).toBe(400); + expect(res.body.error.code).toBe('VALIDATION_ERROR'); }); it('requires snapshot meta to be a year-keyed object map', async () => { - const res = await put('belgian_tax_profile_snapshot_meta_v1', { 2025: { history: [] } }); - expect(res.json).toHaveBeenCalledWith(expect.objectContaining({ ok: true })); + let res = await put('belgian_tax_profile_snapshot_meta_v1', { 2025: { history: [] } }); + expect(res.status).toBe(200); + expect(res.body.ok).toBe(true); + + res = await put('belgian_tax_profile_snapshot_meta_v1', { 2025: 'filed' }); + expect(res.status).toBe(400); + expect(res.body.error.code).toBe('VALIDATION_ERROR'); - await expect(put('belgian_tax_profile_snapshot_meta_v1', { 2025: 'filed' })) - .rejects.toMatchObject({ name: 'ValidationError' }); - await expect(put('belgian_tax_profile_snapshot_meta_v1', 'nope')) - .rejects.toMatchObject({ name: 'ValidationError' }); + res = await put('belgian_tax_profile_snapshot_meta_v1', 'nope'); + expect(res.status).toBe(400); + expect(res.body.error.code).toBe('VALIDATION_ERROR'); }); it('bulk PUT enforces the same profile rules', async () => { query.mockResolvedValue({}); - const req = { body: { belgian_tax_profile: { ...validProfile, communalSurchargePercent: -1 } } }; - const res = mockResponse(); - await expect(routeHandlers['put:/'](req, res)) - .rejects.toMatchObject({ name: 'ValidationError' }); + const res = await api.put(BASE) + .send({ belgian_tax_profile: { ...validProfile, communalSurchargePercent: -1 } }) + .expect(400); + expect(res.body.error.code).toBe('VALIDATION_ERROR'); }); }); diff --git a/apps/node-backend/tests/transactionPatchValidation.test.js b/apps/node-backend/tests/transactionPatchValidation.test.js index ee8625b8..73fd8e89 100644 --- a/apps/node-backend/tests/transactionPatchValidation.test.js +++ b/apps/node-backend/tests/transactionPatchValidation.test.js @@ -1,7 +1,7 @@ import { describe, it, expect, vi, beforeEach } from 'vitest'; import { mockLogger } from './helpers/mockLogger.js'; import { mockConnection } from './helpers/repoMocks.js'; -import { createMockRouter, createMockResponse } from './helpers/routeHarness.js'; +import { routeAgent, errEnvelope } from './helpers/routeApp.js'; import { mockDeduplication, mockCurrencyConversion, @@ -13,12 +13,6 @@ import { // as `SET "date" = ''` (22007 → 500), and non-numeric amount / non-integer // FK ids surfaced as DB cast errors instead of 400s. -const { router: mockRouter, handlers: routeHandlers } = createMockRouter(); - -vi.mock('express', () => ({ - default: { Router: () => mockRouter }, - Router: () => mockRouter, -})); vi.mock('../src/database/connection.js', () => mockConnection({ query: vi.fn().mockResolvedValue({ rows: [] }) })); vi.mock('../src/services/transactionService.js', () => ({ @@ -42,11 +36,12 @@ vi.mock('../src/services/transactionExport.js', () => ({ vi.mock('../src/services/bulkSelection.js', () => ({ resolveBulkSelection: vi.fn() })); import transactionRepository from '../src/services/transactionService.js'; -import { ValidationError } from '../src/middleware/errorHandler.js'; -await import('../src/routes/transactions.js'); -const patchReq = (body) => ({ params: { id: '1' }, body }); -const runPatch = (body) => routeHandlers['patch:/:id'](patchReq(body), createMockResponse()); +const { default: transactionsRouter } = await import('../src/routes/transactions.js'); + +const api = routeAgent(transactionsRouter, { mountPath: '/api/transactions' }); + +const runPatch = (body) => api.patch('/api/transactions/1').send(body); beforeEach(() => { vi.clearAllMocks(); @@ -55,17 +50,26 @@ beforeEach(() => { describe('PATCH /api/transactions/:id validation', () => { it('rejects a cleared date instead of forwarding SET "date" = \'\'', async () => { - await expect(runPatch({ date: '' })).rejects.toBeInstanceOf(ValidationError); - await expect(runPatch({ transaction_date: null })).rejects.toBeInstanceOf(ValidationError); + let res = await runPatch({ date: '' }); + expect(res.status).toBe(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); + + res = await runPatch({ transaction_date: null }); + expect(res.status).toBe(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); + expect(transactionRepository.update).not.toHaveBeenCalled(); }); it('rejects a malformed date', async () => { - await expect(runPatch({ date: 'banana' })).rejects.toBeInstanceOf(ValidationError); + const res = await runPatch({ date: 'banana' }); + expect(res.status).toBe(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); }); it('accepts a valid Y-M-D date and remaps it to transaction_date', async () => { - await runPatch({ date: '2026-07-01' }); + const res = await runPatch({ date: '2026-07-01' }); + expect(res.status).toBe(200); expect(transactionRepository.update).toHaveBeenCalledWith( 1, expect.objectContaining({ transaction_date: '2026-07-01' }), @@ -73,13 +77,16 @@ describe('PATCH /api/transactions/:id validation', () => { }); it('rejects non-numeric or cleared amounts', async () => { - await expect(runPatch({ amount: 'abc' })).rejects.toBeInstanceOf(ValidationError); - await expect(runPatch({ amount: null })).rejects.toBeInstanceOf(ValidationError); - await expect(runPatch({ amount: '' })).rejects.toBeInstanceOf(ValidationError); + for (const amount of ['abc', null, '']) { + const res = await runPatch({ amount }); + expect(res.status).toBe(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); + } }); it('coerces a numeric-string amount', async () => { - await runPatch({ amount: '-12.5' }); + const res = await runPatch({ amount: '-12.5' }); + expect(res.status).toBe(200); expect(transactionRepository.update).toHaveBeenCalledWith( 1, expect.objectContaining({ amount: -12.5 }), @@ -87,10 +94,16 @@ describe('PATCH /api/transactions/:id validation', () => { }); it('rejects non-integer FK ids but lets null clear them', async () => { - await expect(runPatch({ recipient_id: 'abc' })).rejects.toBeInstanceOf(ValidationError); - await expect(runPatch({ category_id: 1.5 })).rejects.toBeInstanceOf(ValidationError); + let res = await runPatch({ recipient_id: 'abc' }); + expect(res.status).toBe(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); + + res = await runPatch({ category_id: 1.5 }); + expect(res.status).toBe(400); + expect(res.body).toEqual(errEnvelope({ code: 'VALIDATION_ERROR' })); - await runPatch({ recipient_id: null, category_id: null }); + res = await runPatch({ recipient_id: null, category_id: null }); + expect(res.status).toBe(200); expect(transactionRepository.update).toHaveBeenCalledWith( 1, expect.objectContaining({ recipient_id: null, category_id: null }), diff --git a/apps/node-backend/tests/transactionRepository.db.test.js b/apps/node-backend/tests/transactionRepository.db.test.js index 197e9e67..9a349a49 100644 --- a/apps/node-backend/tests/transactionRepository.db.test.js +++ b/apps/node-backend/tests/transactionRepository.db.test.js @@ -606,6 +606,91 @@ describe.skipIf(!hasTestDatabase())('repositories/transactionRepository (real DB }); }); + // ─────────────────────────────────────────────────────────────────────────── + // Effective-category id vs displayed name — one category, one label + // ─────────────────────────────────────────────────────────────────────────── + // + // Formerly a real disagreement: EFFECTIVE_CATEGORY_ID_SQL resolves + // own → recipient default (rc) → primary default (pc), while the category-name + // CASE tested `pc` BEFORE `rc`. The two orders only diverge on one topology — + // an ALIAS recipient that has its OWN default category AND a primary carrying + // a DIFFERENT one — which the shared corpus does not contain (its alias has no + // default of its own). On that topology the transactions list reported the + // alias's category id next to the primary's category NAME, and the aggregation + // surfaces, which follow the id, disagreed with the list's label. + describe('alias with its own default under a differently-defaulted primary', () => { + let aliasId; + let aliasTxn; + + beforeEach(async () => { + const pool = getTestPool(); + // Sorts after every other fixture category, so the ORDER BY assertion + // below can tell the two resolutions apart. + const { rows: catRows } = await pool.query( + `INSERT INTO categories (general, detail) VALUES ('Zzz', 'Last') RETURNING id`, + ); + cat.Zzz = catRows[0].id; + // ALIAS of Electrabel (whose default is Bills) with its OWN default Zzz. + const { rows: recRows } = await pool.query( + `INSERT INTO recipients (name, normalized_name, default_category_id, primary_recipient_id) + VALUES ('Electrabel Retail', 'electrabel retail', $1, $2) RETURNING id`, + [cat.Zzz, rec.electrabel], + ); + aliasId = recRows[0].id; + aliasTxn = await insertTxn({ date: '2024-03-05', amount: '-11.11', recipientId: aliasId }); + }); + + it('getById / getAll / getAllWithCount label the row with the category the id names', async () => { + const expected = { effective_category_id: cat.Zzz, category_name: 'Zzz:Last' }; + + expect(await transactionRepository.getById(aliasTxn)).toMatchObject(expected); + + const all = await transactionRepository.getAll({}); + expect(all.find((r) => r.id === aliasTxn)).toMatchObject(expected); + + const paged = await transactionRepository.getAllWithCount({}); + expect(paged.rows.find((r) => r.id === aliasTxn)).toMatchObject(expected); + }); + + it('create / update return the same pairing as an immediate GET', async () => { + const created = await transactionRepository.create({ + transaction_date: '2024-03-06', + recipient_id: aliasId, + amount: '-22.22', + category_id: null, + comment: null, + }); + expect(created.effective_category_id).toBe(cat.Zzz); + expect(created.category_name).toBe('Zzz:Last'); + expect(await transactionRepository.getById(created.id)).toEqual(created); + + const updated = await transactionRepository.update(aliasTxn, { amount: '-33.33' }); + expect(updated.effective_category_id).toBe(cat.Zzz); + expect(updated.category_name).toBe('Zzz:Last'); + expect(await transactionRepository.getById(aliasTxn)).toEqual(updated); + + // The tags-path fetch (fields + tags in one PATCH) resolves identically. + const withTags = await transactionRepository.update(aliasTxn, { amount: '-34.00', tags: [] }); + expect(withTags.effective_category_id).toBe(cat.Zzz); + expect(withTags.category_name).toBe('Zzz:Last'); + }); + + it("an own category_id still wins over both defaults", async () => { + const row = await transactionRepository.update(aliasTxn, { category_id: cat.Food }); + expect(row.effective_category_id).toBe(cat.Food); + expect(row.category_name).toBe('Food:Groceries'); + }); + + it('sorts by the same resolved name it displays', async () => { + const rows = await transactionRepository.getAll({ sortBy: 'category', sortDir: 'asc' }); + const ids = rows.map((r) => r.id); + // 'Zzz:Last' sorts after t7's 'Bills:Utilities'. Under the old pc-first + // order the alias row was labelled 'Bills:Utilities' and — being the + // later date — sorted BEFORE t7 on the tiebreaker. + expect(ids.indexOf(aliasTxn)).toBeGreaterThan(ids.indexOf(T.t7)); + }); + }); + describe('hardDelete', () => { it('deletes exactly once (junction rows cascade)', async () => { expect(await transactionRepository.hardDelete(T.t1)).toBe(true); diff --git a/apps/node-backend/tests/validation.test.js b/apps/node-backend/tests/validation.test.js index 1e19c309..69d019a7 100644 --- a/apps/node-backend/tests/validation.test.js +++ b/apps/node-backend/tests/validation.test.js @@ -3,7 +3,6 @@ * Mirrors validation-related tests from Python test suite. */ import { describe, it, expect, vi } from 'vitest'; -import { createMockResponse as mockResponse } from './helpers/routeHarness.js'; import { validateId, sanitizeString, validateNumber, validateDateString, @@ -12,6 +11,15 @@ import { } from '../src/middleware/validation.js'; import { ValidationError } from '../src/middleware/errorHandler.js'; +// validateIdParam is unit-tested as a plain middleware function +// (req, res, next) — a minimal res stub is enough; there is no router/HTTP +// path to exercise. +function mockResponse() { + const res = { json: vi.fn(), status: vi.fn(), send: vi.fn() }; + res.status.mockReturnValue(res); + return res; +} + describe('Validation Middleware', () => { describe('validateId', () => { it('should accept valid positive integers', () => { diff --git a/bun.lock b/bun.lock index affc10c9..253e6a2e 100644 --- a/bun.lock +++ b/bun.lock @@ -128,6 +128,7 @@ "archiver": "^8.0.0", "eslint": "^10.7.0", "globals": "^17.7.0", + "supertest": "^7.2.2", "typescript": "^6.0.3", "vitest": "^4.1.10", "yauzl": "^3.4.0", @@ -408,6 +409,8 @@ "@oxc-project/types": ["@oxc-project/types@0.133.0", "", {}, "sha512-KzkdCd6Uxqnf6l3HOw1xfatAlUURA0g14cvBYFyJ5SaNOQbOUvBr9PKArcPcrNIeRsBdgcUzOGrhKveVpvOIGA=="], + "@paralleldrive/cuid2": ["@paralleldrive/cuid2@2.3.1", "", { "dependencies": { "@noble/hashes": "^1.1.5" } }, "sha512-XO7cAxhnTZl0Yggq6jOgjiOHhbgcO4NqFqwSmQpjK3b6TEE6Uj/jfSk6wzYyemh3+I0sHirKSetjQwn5cZktFw=="], + "@playwright/test": ["@playwright/test@1.61.1", "", { "dependencies": { "playwright": "1.61.1" }, "bin": { "playwright": "cli.js" } }, "sha512-8nKv6+0RJSL9FE4jYOEGXnPeM/Hg12qZpmqzZjRh3qM0Y7c3z1mrOTfFLids72RDQYVh9WpLEfR5WdpNX4fkig=="], "@polka/url": ["@polka/url@1.0.0-next.29", "", {}, "sha512-wwQAWhWSuHaag8c4q/KN/vCoeOJYshAIvMQwD4GpSb3OiZklFfvAgmj0VCBBImRpuF/aFgIRzllXlVX93Jevww=="], @@ -812,12 +815,16 @@ "aria-query": ["aria-query@5.3.2", "", {}, "sha512-COROpnaoap1E2F000S62r6A60uHZnmlvomhfyT2DlTcrY1OrBKn2UhH7qn5wTC9zMvD0AY7csdPSNwKP+7WiQw=="], + "asap": ["asap@2.0.6", "", {}, "sha512-BSHWgDSAiKs50o2Re8ppvp3seVHXSRM44cdSsT9FfNEUUZLOGWVCsiWaRPWM1Znn+mqZ1OfVZ3z3DWEzSp7hRA=="], + "assertion-error": ["assertion-error@2.0.1", "", {}, "sha512-Izi8RQcffqCeNVgFigKli1ssklIbpHnCYc6AknXGYoB6grJqyeby7jv12JUQgmTAnIDnbck1uxksT4dzN3PWBA=="], "ast-v8-to-istanbul": ["ast-v8-to-istanbul@1.0.0", "", { "dependencies": { "@jridgewell/trace-mapping": "^0.3.31", "estree-walker": "^3.0.3", "js-tokens": "^10.0.0" } }, "sha512-1fSfIwuDICFA4LKkCzRPO7F0hzFf0B7+Xqrl27ynQaa+Rh0e1Es0v6kWHPott3lU10AyAr7oKHa65OppjLn3Rg=="], "async": ["async@3.2.6", "", {}, "sha512-htCUDlxyyCLMgaM3xXg0C0LW2xqfuQ6p05pCEIsXuyQ+a1koYKTuBMzRNwmybfLgvJDMd0r1LTn4+E0Ti6C2AA=="], + "asynckit": ["asynckit@0.4.0", "", {}, "sha512-Oei9OH4tRh0YqU3GxhX79dM/mwVgvbZJaSNaRk+bshkj0S5cfHcgYakreBjrHwatXKbz+IoIdYLxrKim2MjW0Q=="], + "autoprefixer": ["autoprefixer@10.5.2", "", { "dependencies": { "browserslist": "^4.28.4", "caniuse-lite": "^1.0.30001799", "fraction.js": "^5.3.4", "picocolors": "^1.1.1", "postcss-value-parser": "^4.2.0" }, "peerDependencies": { "postcss": "^8.1.0" }, "bin": { "autoprefixer": "bin/autoprefixer" } }, "sha512-rD5t5DwOjJdmSORcTq64j8MawTC+tbQ+HHqjR4NDumamy/ambn1UJrlKL+KdwujWxMkFjPM3pPHOEA9tl4767Q=="], "axe-core": ["axe-core@4.12.1", "", {}, "sha512-s7iGf5GaVMxEG0ENN9x+xTr7GFZCb1ZP/1uATUpCEK2X78nDB3RwbtFCo9pGAf9ru+VwoQ464DkaLEeRM08wJA=="], @@ -896,8 +903,12 @@ "colorette": ["colorette@1.4.0", "", {}, "sha512-Y2oEozpomLn7Q3HFP7dpww7AtMJplbM9lGZP6RDfHqmbeRjiwRg4n6VM6j4KLmRke85uWEI7JqF17f3pqdRA0g=="], + "combined-stream": ["combined-stream@1.0.8", "", { "dependencies": { "delayed-stream": "~1.0.0" } }, "sha512-FQN4MRfuJeHf7cBbBMJFXhKSDq+2kAArBlmRBvcvFE5BB1HZKXtSFASDhdlz9zOYwxh8lDdnvmMOe/+5cdoEdg=="], + "commander": ["commander@14.0.3", "", {}, "sha512-H+y0Jo/T1RZ9qPP4Eh1pkcQcLRglraJaSLoyOtHxu6AapkjWVCy2Sit1QQ4x3Dng8qDlSsZEet7g5Pq06MvTgw=="], + "component-emitter": ["component-emitter@1.3.1", "", {}, "sha512-T0+barUSQRTUQASh8bx02dl+DhF54GtIDY13Y3m9oWTklKbb3Wv974meRpeZ3lp1JpLVECWWNHC4vaG2XHXouQ=="], + "compress-commons": ["compress-commons@7.0.1", "", { "dependencies": { "crc-32": "^1.2.0", "crc32-stream": "^7.0.1", "is-stream": "^4.0.0", "normalize-path": "^3.0.0", "readable-stream": "^4.0.0" } }, "sha512-g0S8KAD8qf4+V//pr3BfB1aBnARLXNz2Gx+jmHU0LEriUuoQUOPOulVquHKTJ8+EAIIO7fhseNDr9wK5Q9FKBQ=="], "concat-stream": ["concat-stream@2.0.0", "", { "dependencies": { "buffer-from": "^1.0.0", "inherits": "^2.0.3", "readable-stream": "^3.0.2", "typedarray": "^0.0.6" } }, "sha512-MWufYdFw53ccGjCA+Ol7XJYpAlW6/prSMzuPOTRnJGcGzuhLn4Scrz7qf6o8bROZ514ltazcIFJZevcfbo0x7A=="], @@ -920,6 +931,8 @@ "cookie-signature": ["cookie-signature@1.2.2", "", {}, "sha512-D76uU73ulSXrD1UXF4KE2TMxVVwhsnCgfAyTg9k8P6KGZjlXKrOLe4dJQKI3Bxi5wjesZoFXJWElNWBjPZMbhg=="], + "cookiejar": ["cookiejar@2.1.4", "", {}, "sha512-LDx6oHrK+PhzLKJU9j5S7/Y3jM/mUHvD/DeI1WQmJn652iPC5Y4TBzC9l+5OMOXlyTTA+SmVUPm0HQUwpD5Jqw=="], + "core-util-is": ["core-util-is@1.0.3", "", {}, "sha512-ZQBvi1DcpJ4GDqanjucZ2Hj3wEO5pZDS89BWbkcrvdxksJorwUDDZamX9ldFkp9aw2lmBDLgkObEA4DWNJ9FYQ=="], "cors": ["cors@2.8.6", "", { "dependencies": { "object-assign": "^4", "vary": "^1" } }, "sha512-tJtZBBHA6vjIAaF6EnIaq6laBBP9aq/Y3ouVJjEfoHbRBcHBAHYcMh/w8LDrk2PvIMMq8gmopa5D4V8RmbrxGw=="], @@ -984,6 +997,8 @@ "delaunator": ["delaunator@5.1.0", "", { "dependencies": { "robust-predicates": "^3.0.2" } }, "sha512-AGrQ4QSgssa1NGmWmLPqN5NY2KajF5MqxetNEO+o0n3ZwZZeTmt7bBnvzHWrmkZFxGgr4HdyFgelzgi06otLuQ=="], + "delayed-stream": ["delayed-stream@1.0.0", "", {}, "sha512-ZySD7Nf91aLB0RxL4KGrKHBXl7Eds1DAmEdcoVawXnLD7SDhpNgtuII2aAkg7a7QS41jxPSZ17p4VdGnMHk3MQ=="], + "depd": ["depd@2.0.0", "", {}, "sha512-g7nH6P6dyDioJogAAGprGpCtVImJhpPk/roCzdb3fIh61/s/nPsfR6onyMwkCAR/OlC3yBC0lESvUoQEAssIrw=="], "dequal": ["dequal@2.0.3", "", {}, "sha512-0je+qPKHEMohvfRTCEo3CrPG6cAzAYgmzKyxRiYSSDkS6eGJdyVJm7WaYA5ECaAD9wLB2T4EEeymA5aFVcYXCA=="], @@ -996,6 +1011,8 @@ "devtools-protocol": ["devtools-protocol@0.0.1638949", "", {}, "sha512-mXwg4Fqnv0WR4iuAT/gYUmctNkjILwXFHyZ+m7Ty1dfr0ezZt2U3gnrrJTfRobJTHoXf+IbuFvFITzLrLFjwJA=="], + "dezalgo": ["dezalgo@1.0.4", "", { "dependencies": { "asap": "^2.0.0", "wrappy": "1" } }, "sha512-rXSP0bf+5n0Qonsb+SVVfNfIsimO4HEtmnIpPHY8Q1UCzKlQrDMfdobr8nJOOsRgWCyMRqeSBQzmWUMq7zvVig=="], + "diff-match-patch": ["diff-match-patch@1.0.5", "", {}, "sha512-IayShXAgj/QMXgB0IWmKx+rOPuGMhqm5w6jvFxmVenXKIzRqTAAsbBPT3kWQeGANj3jGgvcvv4yK6SxqYmikgw=="], "dom-accessibility-api": ["dom-accessibility-api@0.6.3", "", {}, "sha512-7ZgogeTnjuHbo+ct10G9Ffp0mif17idi0IyWNVA/wcwcm7NPOD/WEHVP3n7n3MhXqxoIYm8d6MuZohYWIZ4T3w=="], @@ -1026,6 +1043,8 @@ "es-object-atoms": ["es-object-atoms@1.1.1", "", { "dependencies": { "es-errors": "^1.3.0" } }, "sha512-FGgH2h8zKNim9ljj7dankFPcICIK9Cp5bm+c2gQSYePhpaG5+esrLODihIorn+Pe6FGJzWhXQotPv73jTaldXA=="], + "es-set-tostringtag": ["es-set-tostringtag@2.1.0", "", { "dependencies": { "es-errors": "^1.3.0", "get-intrinsic": "^1.2.6", "has-tostringtag": "^1.0.2", "hasown": "^2.0.2" } }, "sha512-j6vWzfrGVfyXxge+O0x5sh6cvxAog0a/4Rdd2K36zCMV5eJ+/+tOAngRO8cODMNWbVRdVlmGZQL2YS3yR8bIUA=="], + "es-toolkit": ["es-toolkit@1.46.1", "", {}, "sha512-5eNtXOs3tbfxXOj04tjjseeWkRWaoCjdEI+96DgwzZoe6c9juL49pXlzAFTI72aWC9Y8p7168g6XIKjh7k6pyQ=="], "escalade": ["escalade@3.2.0", "", {}, "sha512-WUj2qlxaQtO4g6Pq5c29GTcWGDyd8itL8zTlipgECz3JesAiiOKotd8JU6otB3PACgG6xkJUyVhboMS+bje/jA=="], @@ -1088,6 +1107,8 @@ "fast-levenshtein": ["fast-levenshtein@2.0.6", "", {}, "sha512-DCXu6Ifhqcks7TZKY3Hxp3y6qphY5SJZmrWMDrKcERSOXWQdMhU9Ig/PYrzyw/ul9jOIyh0N4M0tbC5hodg8dw=="], + "fast-safe-stringify": ["fast-safe-stringify@2.1.1", "", {}, "sha512-W+KJc2dmILlPplD/H4K9l9LcAHAfPtP6BY84uVLXQ6Evcz9Lcg33Y2z1IVblT6xdY54PXYVHEv+0Wpq8Io6zkA=="], + "fast-string-truncated-width": ["fast-string-truncated-width@3.0.3", "", {}, "sha512-0jjjIEL6+0jag3l2XWWizO64/aZVtpiGE3t0Zgqxv0DPuxiMjvB3M24fCyhZUO4KomJQPj3LTSUnDP3GpdwC0g=="], "fast-string-width": ["fast-string-width@3.0.2", "", { "dependencies": { "fast-string-truncated-width": "^3.0.2" } }, "sha512-gX8LrtNEI5hq8DVUfRQMbr5lpaS4nMIWV+7XEbXk2b8kiQIizgnlr12B4dA3ZEx3308ze0O4Q1R+cHts8kyUJg=="], @@ -1122,6 +1143,10 @@ "flatted": ["flatted@3.4.2", "", {}, "sha512-PjDse7RzhcPkIJwy5t7KPWQSZ9cAbzQXcafsetQoD7sOJRQlGikNbx7yZp2OotDnJyrDcbyRq3Ttb18iYOqkxA=="], + "form-data": ["form-data@4.0.6", "", { "dependencies": { "asynckit": "^0.4.0", "combined-stream": "^1.0.8", "es-set-tostringtag": "^2.1.0", "hasown": "^2.0.4", "mime-types": "^2.1.35" } }, "sha512-vKatAh4SlVfgbv+YtmhiRjhEMJsYpsG1Y2rMQtR+SVSbytsSD1YGzDIcrAJmdFec88u/+VoGmxnl+80gL1tRCQ=="], + + "formidable": ["formidable@3.5.4", "", { "dependencies": { "@paralleldrive/cuid2": "^2.2.2", "dezalgo": "^1.0.4", "once": "^1.4.0" } }, "sha512-YikH+7CUTOtP44ZTnUhR7Ic2UASBPOqmaRkRKxRbywPTe5VxF7RRCck4af9wutiZ/QKM5nME9Bie2fFaPz5Gug=="], + "forwarded": ["forwarded@0.2.0", "", {}, "sha512-buRG0fpBtRHSTCOASe6hD258tEubFoRLb4ZNA6NxMVHNw2gOcwHo9wyablzMzOA5z9xA9L1KNjk/Nt6MT9aYow=="], "fraction.js": ["fraction.js@5.3.4", "", {}, "sha512-1X1NTtiJphryn/uLQz3whtY6jK3fTqoE3ohKs0tT+Ujr1W59oopxmoEh7Lu5p6vBaPbgoM0bzveAW4Qi5RyWDQ=="], @@ -1164,7 +1189,9 @@ "has-symbols": ["has-symbols@1.1.0", "", {}, "sha512-1cDNdwJ2Jaohmb3sg4OmKaMBwuC48sYni5HUw2DvsC8LjGTLK9h+eb1X6RyuOHe4hT0ULCW68iomhjUoKUqlPQ=="], - "hasown": ["hasown@2.0.3", "", { "dependencies": { "function-bind": "^1.1.2" } }, "sha512-ej4AhfhfL2Q2zpMmLo7U1Uv9+PyhIZpgQLGT1F9miIGmiCJIoCgSmczFdrc97mWT4kVY72KA+WnnhJ5pghSvSg=="], + "has-tostringtag": ["has-tostringtag@1.0.2", "", { "dependencies": { "has-symbols": "^1.0.3" } }, "sha512-NqADB8VjPFLM2V0VvHUewwwsw0ZWBaIdgo+ieHtK3hasLz4qeCRjYcqfB6AQrBggRKppKF8L52/VqdVsO47Dlw=="], + + "hasown": ["hasown@2.0.4", "", { "dependencies": { "function-bind": "^1.1.2" } }, "sha512-T2UbfbBEF32wiepXIsMlTW9+dDYC6wMh/t/vYA4tuOMKqWz/n3vr1NFSxQiyP+zk2mXsoMA/i/7qV6LKut1t1A=="], "headers-polyfill": ["headers-polyfill@5.0.1", "", { "dependencies": { "@types/set-cookie-parser": "^2.4.10", "set-cookie-parser": "^3.0.1" } }, "sha512-1TJ6Fih/b8h5TIcv+1+Hw0PDQWJTKDKzFZzcKOiW1wJza3XoAQlkCuXLbymPYB8+ZQyw8mHvdw560e8zVFIWyA=="], @@ -1334,6 +1361,10 @@ "merge-descriptors": ["merge-descriptors@2.0.0", "", {}, "sha512-Snk314V5ayFLhp3fkUREub6WtjBfPdCPY1Ln8/8munuLuiYhsABgBVWsozAG+MWMbVEvcdcpbi9R7ww22l9Q3g=="], + "methods": ["methods@1.1.2", "", {}, "sha512-iclAHeNqNm68zFtnZ0e+1L2yUIdvzNoauKU4WBA3VvH/vPFieF7qfRlwUZU+DA9P9bPXIS90ulxoUoCH23sV2w=="], + + "mime": ["mime@2.6.0", "", { "bin": { "mime": "cli.js" } }, "sha512-USPkMeET31rOMiarsBNIHZKLGgvKc/LrjofAnBlOttf5ajRvqiRA8QsenbcooctK6d6Ts6aqZXBA+XbkKthiQg=="], + "mime-db": ["mime-db@1.54.0", "", {}, "sha512-aU5EJuIN2WDemCcAp2vFBfp/m4EAhWJnUNSSw0ixs7/kXbd6Pg64EmwJkNdFhB8aWt1sH2CTXrLxo/iAGV3oPQ=="], "mime-types": ["mime-types@3.0.2", "", { "dependencies": { "mime-db": "^1.54.0" } }, "sha512-Lbgzdk0h4juoQ9fCKXW4by0UJqj+nOOrI9MJ1sSj4nI8aI2eo1qmvQEie4VD1glsS250n15LsWsYtCugiStS5A=="], @@ -1614,6 +1645,10 @@ "strip-indent": ["strip-indent@3.0.0", "", { "dependencies": { "min-indent": "^1.0.0" } }, "sha512-laJTa3Jb+VQpaC6DseHhF7dXVqHTfJPCRDaEbid/drOhgitgYku/letMUqOXFoWV0zIIUbjpdH2t+tYj4bQMRQ=="], + "superagent": ["superagent@10.3.0", "", { "dependencies": { "component-emitter": "^1.3.1", "cookiejar": "^2.1.4", "debug": "^4.3.7", "fast-safe-stringify": "^2.1.1", "form-data": "^4.0.5", "formidable": "^3.5.4", "methods": "^1.1.2", "mime": "2.6.0", "qs": "^6.14.1" } }, "sha512-B+4Ik7ROgVKrQsXTV0Jwp2u+PXYLSlqtDAhYnkkD+zn3yg8s/zjA2MeGayPoY/KICrbitwneDHrjSotxKL+0XQ=="], + + "supertest": ["supertest@7.2.2", "", { "dependencies": { "cookie-signature": "^1.2.2", "methods": "^1.1.2", "superagent": "^10.3.0" } }, "sha512-oK8WG9diS3DlhdUkcFn4tkNIiIbBx9lI2ClF8K+b2/m8Eyv47LSawxUzZQSNKUrVb2KsqeTDCcjAAVPYaSLVTA=="], + "supports-color": ["supports-color@10.2.2", "", {}, "sha512-SS+jx45GF1QjgEXQx4NJZV9ImqmO2NPz5FNsIHrsDjh2YsHnawpan7SNQ1o8NuhrbHZy9AZhIoCUiCeaW/C80g=="], "symbol-tree": ["symbol-tree@3.2.4", "", {}, "sha512-9QNk5KwDF+Bvz+PyObkmSYjI5ksVUYtjW7AU22r2NKcfLJcXp96hkDWU3+XndOsUb+AQ9QhfzfCT2O+CNWT5Tw=="], @@ -1866,6 +1901,10 @@ "d3-shape/d3-path": ["d3-path@1.0.9", "", {}, "sha512-VLaYcn81dtHVTjEHd8B+pbe9yHWpXKZUC87PzoFmsFrJqgFwDe/qxfp5MlfsfM1V5E/iVt0MmEbWQ7FVIXh/bg=="], + "form-data/mime-types": ["mime-types@2.1.35", "", { "dependencies": { "mime-db": "1.52.0" } }, "sha512-ZDY+bPm5zTTF+YpCrAU9nK0UgICYPT0QtT1NZWFv4s++TNkcgVaT0g6+4R2uI4MjQjzysHB1zxuWL50hzaeXiw=="], + + "get-intrinsic/hasown": ["hasown@2.0.3", "", { "dependencies": { "function-bind": "^1.1.2" } }, "sha512-ej4AhfhfL2Q2zpMmLo7U1Uv9+PyhIZpgQLGT1F9miIGmiCJIoCgSmczFdrc97mWT4kVY72KA+WnnhJ5pghSvSg=="], + "import-fresh/resolve-from": ["resolve-from@4.0.0", "", {}, "sha512-pb/MYmXstAkysRFx8piNI1tGFNQIFA3vkE3Gq4EuA1dF6gHp/+vgZqsCGJapvy8N3Q+4o7FwvquPJcnZ7RYy4g=="], "istanbul-lib-report/supports-color": ["supports-color@7.2.0", "", { "dependencies": { "has-flag": "^4.0.0" } }, "sha512-qpCAvRl9stuOHveKsn7HncJRvv501qIacKzQlO/+Lwxc9+0q2wLyv4Dfvt80/DPn2pqOBsJdDiogXGR9+OvwRw=="], @@ -1958,6 +1997,8 @@ "d3-sankey/d3-array/internmap": ["internmap@1.0.1", "", {}, "sha512-lDB5YccMydFBtasVtxnZ3MRBHuaoE8GKsppq+EchKL2U4nK/DmEpPHNH8MZe5HkMtpSiTSOZwfN0tzYjO/lJEw=="], + "form-data/mime-types/mime-db": ["mime-db@1.52.0", "", {}, "sha512-sPU4uV7dYlvtWJxwwxHD0PuihVNiE7TyAbQ5SWxDCB9mUYvOgroQOwYQQOKPJ8CIbE+1ETVlOoK1UC2nU3gYvg=="], + "lazystream/readable-stream/string_decoder": ["string_decoder@1.1.1", "", { "dependencies": { "safe-buffer": "~5.1.0" } }, "sha512-n/ShnvDi6FHbbVfviro+WojiFzv+s8MPMHBczVePfUpDJLwoLT0ht1l4YwBCbi8pJAveEEdnkHyPyTP/mzRfwg=="], "msw/yargs/cliui": ["cliui@8.0.1", "", { "dependencies": { "string-width": "^4.2.0", "strip-ansi": "^6.0.1", "wrap-ansi": "^7.0.0" } }, "sha512-BSeNnyus75C4//NQ9gQt1/csTXyo/8Sb+afLAkzAptFuMsod9HFokGNudZpi/oQV73hnVK+sR+5PVRMd+Dr7YQ=="], diff --git a/openapi.yaml b/openapi.yaml index 890e4264..dc4cad16 100644 --- a/openapi.yaml +++ b/openapi.yaml @@ -6200,7 +6200,7 @@ paths: type: object properties: batch_id: - type: string + type: integer total: type: integer imported: @@ -6222,7 +6222,7 @@ paths: type: object properties: batch_id: - type: string + type: integer requires_review: type: boolean match_source_counts: