diff --git a/TODO.md b/TODO.md index 30d532ab..3fdd1b5d 100644 --- a/TODO.md +++ b/TODO.md @@ -450,7 +450,7 @@ look-changing one. - Transactions are bucketed solely by `recipient_id`; amounts go through `.abs()` before averaging, with no sign/category partitioning. A recipient who both pays and is paid by the user gets both directions merged into one averaged "pattern" that matches neither real flow. - Fix: partition each recipient's transactions by sign (or category) before interval/amount detection. -- [ ] **Route/service boundary (ADR-067) is bypassed via direct DB access in several route files, undetected by the lint gate meant to prevent exactly this** πŸ”Ό πŸ”Ž partial-0b9d64b 2026-07-13 (the shared name-resolver extraction removed `plannedTransactions.js`'s only `database/connection.js` import; `transactions.js` and `attachments.js` still import it directly, and the lint rule still doesn't flag connection.js imports under routes/**) +- [x] **Route/service boundary (ADR-067) is bypassed via direct DB access in several route files, undetected by the lint gate meant to prevent exactly this** πŸ”Ό πŸ”Ž partial-0b9d64b 2026-07-13 βœ… 2026-07-27 Β· b4995be (verified 2026-07-27: the LEFT clause was fully resolved by b4995be (#117, 2026-07-22) β€” transactions.js bulk/create/hard-delete SQL moved to transactionBulkService/transactionService/attachmentCleanup, attachments.js rows behind attachmentRecordService, and the eslint rule's ImportDeclaration visitor now reports any `/database/` import under routes/** (fire-proof re-run 2026-07-27: probe import β†’ error, exit 1). Sole remaining routes/** connection.js import is admin.js:18 with a documented inline ADR-067 exemption) - β†ͺ _from: Codebase audit 2026-06-30 Β· Correctness β€” Backend Β· Architecture / route-service boundary / dead code (backend)_ - `routes/transactions.js:8,171-203,318-485`, `routes/plannedTransactions.js:8,45-75`, `routes/attachments.js:22,61` import `query`/`withTransaction` straight from `database/connection.js` and run raw SQL in route handlers. The custom ESLint rule `no-repo-direct-from-route` only blocks `/repositories/` imports, not `database/connection.js` β€” this larger bypass passes lint clean (verified directly by running `npx eslint` on all three files: exit 0, zero warnings). - Fix: move these queries into `transactionService`/`plannedTransactionService`; extend the lint rule to also flag `database/connection.js` imports under `routes/**`. @@ -563,11 +563,17 @@ look-changing one. - In the sign-based model (βˆ’ expense / + income) a 0-amount row is meaningless and pollutes aggregations. - Fix: reject `amountValue === 0` client-side; tighten backend to `Number.isFinite` + non-zero. -- [ ] **Raw English backend error messages leak into toasts on every failed mutation** πŸ”½ πŸ”Ž verified-present 2026-07-11 +- [x] **Raw English backend error messages leak into toasts on every failed mutation** πŸ”½ πŸ”Ž verified-present 2026-07-11 βœ… 2026-07-28 Β· 632cf8b (half 1 was already fixed since verification: lib/api/errorMessage.ts apiErrorToMessage(error, t) maps ApiClientError codes β†’ apiError.* keys and is used by 33 files incl. all 7 useTransactions mutations β€” zero raw error.message toasts remained. Half 2 landed here: GlobalMutationErrorToaster subscribes to the mutation cache and toasts mapped copy for the 9 mutations that had no onError at all (silent failures: attachment upload, watchlist delete, research mapping dialogs…), skipping mutations with options.onError or meta.suppressErrorToast; RebalancePage inline raw message also mapped; 5 tests pin copy + both silence gates) - β†ͺ _from: Correctness research 2026-07-02 Β· Wave 2a_ - `hooks/useTransactions.ts:78-81` (toast description = raw `error.message`); no global `MutationCache` onError in `App.tsx:93-101`, so any mutation without a hook-level handler silently swallows errors (none confirmed, hooks unaudited β€” see residue) - Fix: map backend error codes to i18n keys at the toast layer; add a global MutationCache onError as backstop. +- [ ] **Residual raw-error surfaces: query-context inline errors show raw `error.message`; AddTransactionDialog branches on error-message substring** ⏬ + - β†ͺ _from: Orchestration session 2026-07-28 Β· error-toast fix pass (noticed, out of that finding's mutation-toast scope)_ + - Query (not mutation) inline error surfaces still render raw `error.message`: `pages/RecipientsPage.tsx:340`, `pages/CategoriesPage.tsx:118`, `pages/TransactionsPage.tsx:415`, `pages/ImportReviewPage.tsx` (load error), `NetWorthPage.tsx:132`, `PortfolioImportPage.tsx:141`, plus `AdminErrorState`/`ErrorBoundary` diagnostics (arguably fine as developer surfaces). And `AddTransactionDialog.tsx:120` branches on `error.message.includes('Duplicate')` β€” fragile string matching that should key off `ApiClientError.code`/status (control flow, not copy). + - Fix: route the inline query-error surfaces through `apiErrorToMessage`; replace the Duplicate substring check with a code/status check. + + - [x] **Chart tooltips + two pages format via browser locale, not the app language setting** πŸ”½ βœ… 2026-07-13 Β· d9e4bf9 (#84, Area/Line/ComposedChart fallbacks use formatDateWithAppSettings(appSettings.dateFormat); ToolResultCard and DbMaintenancePage use numberFormatToLocale/formatDateTimeWithAppSettings) - β†ͺ _from: Correctness research 2026-07-02 Β· Wave 2a_ - `components/charts/AreaChart.tsx:564`, `LineChart.tsx:481`, `ComposedChart.tsx:346` (no-arg `toLocaleDateString()` fallback); `features/ai-chat/ToolResultCard.tsx:68`, `pages/DbMaintenancePage.tsx:87,91` (no-arg `toLocaleString()`) @@ -750,21 +756,54 @@ look-changing one. - **Caveat: the intended sign convention for non-loan planned payments is undocumented β€” confirm intent before fixing.** - Fix: add an explicit income/expense toggle to the form and negate on save for expenses, once intent is confirmed. -- [ ] **RecurringDetectionPanel hardcodes expense sign β€” a detected recurring income becomes a negative planned payment that can never auto-match** πŸ”Ό +- [x] **RecurringDetectionPanel hardcodes expense sign β€” a detected recurring income becomes a negative planned payment that can never auto-match** πŸ”Ό βœ… 2026-07-27 Β· f44de00 (panel now recombines |latestAmount| with the pattern's server-emitted `direction` β€” no payload change needed since detection already groups per recipient+sign; MSW end-to-end tests pin the POST body sign both directions; adversarially verified incl. plannedMatchService sign predicate accepting the income row) - β†ͺ _from: Orchestration session 2026-07-27 Β· planned-sign fix verification_ - `apps/frontend/src/components/planned/RecurringDetectionPanel.tsx:150` β€” `amount: pattern.latestAmount * -1`, and detected amounts are `.abs()`'d server-side (`recurringDetectionService.js:240-242`), so sign information is lost before the panel and the panel assumes expense. Salary/rent-received patterns become negative planned payments; `plannedMatchService` then rejects the sign mismatch against the real positive txn β€” same bug class as the fixed form finding, different producer. - Fix: carry the dominant sign of the detected pattern through the detection payload (or infer from the source transactions) and use it when creating the planned payment. -- [ ] **Loan PATCH keeps a stale client `amount` when the repayment schedule is regenerated** πŸ”½ +- [x] **Loan PATCH keeps a stale client `amount` when the repayment schedule is regenerated** πŸ”½ βœ… 2026-07-27 Β· 7c9ed65 (the regeneration branch now sets amount = -|regular_payment_amount| unconditionally, matching POST; a loan PATCH touching no schedule input still honors the client amount β€” pinned by three route tests against the real amortization math; adversarially verified incl. sign convention and no-rounding-divergence vs POST) - β†ͺ _from: Orchestration session 2026-07-27 Β· planned-sign fix verification_ - `apps/node-backend/src/routes/plannedTransactions.js:236-240` β€” editing `loan_principal`/rate/term regenerates the schedule and updates `loan_regular_payment_amount`, but a defined client `fields.amount` is kept as-is (only an *undefined* amount is re-derived), so `amount` desyncs from `loan_regular_payment_amount`. Same on convert-to-loan via PATCH. Pre-existing behavior, unchanged in magnitude by the sign-control fix (which strictly improved the convert case). - Fix: when `is_loan` and any schedule input changed, always re-derive `amount = -|regular_payment_amount|` server-side, ignoring the client value. -- [ ] **Planned-payment form state is sticky across consecutive "New" opens β€” including the new visible direction toggle** πŸ”½ +- [x] **Planned-payment form state is sticky across consecutive "New" opens β€” including the new visible direction toggle** πŸ”½ βœ… 2026-07-27 Β· 8f4ebbb (createFormKey counter bumped on each New open keys fresh creates so the form remounts blank; integration test pins direction-back-to-Expense + empty name on reopen; no visual change) - β†ͺ _from: Orchestration session 2026-07-27 Β· planned-sign fix verification_ - `apps/frontend/src/pages/PlannedPaymentsPage.tsx:467-473` β€” the form is keyed `editing?.id ?? "new"`, so back-to-back creates never remount and all `useState` initializers survive (name, amount, notes… pre-existing; the direction toggle is now the most visible sticky field: create an income row, reopen "New", toggle still says Income). - Fix: reset key per open (e.g. an incrementing counter when opening in create mode) or reset state on `open` transition. +- [x] **AI watchlist insights tool reads `current_price` off bare repository rows β€” the field is never selected, so the tool's currentPrice is always null** πŸ”½ βœ… 2026-07-27 Β· 92729f8 (tool now merges live prices via marketLookupService.getQuotes(symbols, true) β€” the same cached service behind the quote endpoint β€” with quote failures degrading to null; nuance found during the fix: the enrichment the tool was missing lives in the frontend WatchlistPage merge, not a backend route as filed; 4 tests pin merge/skip/degradation) + - β†ͺ _from: Orchestration session 2026-07-27 Β· checkJs ratchet pass (noticed while typing watchlistRepository)_ + - `apps/node-backend/src/services/aiChat/tools/insights.js:219` reads `item.current_price` from `watchlistRepository.getAllWithCount()` rows, but the repository never selects that column β€” it is merged in by the watchlist *route* from the live quote, a step the AI tool skips. The tool therefore always reports `currentPrice: null`. (`FormattedWatchlistRow` now types it as an optional merged-in field, so the type gate stays honest; the runtime gap remains.) + - Fix: have the insights tool fetch quotes the same way the route does (or call a shared enrichment helper) before reporting watchlist prices. + +- [x] **`sanitizeIsolatedDailyInvestmentSpikes` recomputes netWorth without liabilities β€” corrected spike days get an inflated net-worth point** πŸ”½ βœ… 2026-07-27 Β· da15e4a (recompute now liquid + liabilities + correctedInvestments, matching the builder formula at infoRepositoryNetWorth.js:289; test pins exact figures incl. the old buggy 1504.99; adversarially verified β€” snapshot shape traced from the single call site, ADR-092 negative-liability convention confirmed from the builder code, no consumer relied on the inflated value, zero-liability installs degenerate to the old formula exactly) + - β†ͺ _from: Orchestration session 2026-07-27 Β· checkJs ratchet pass (noticed while typing infoRepositoryHelpers)_ + - `apps/node-backend/src/repositories/infoRepositoryHelpers.js:~292` β€” when an isolated investment spike is corrected, the recomputation is `netWorth = liquid + correctedInvestments`, dropping the `liabilities` term that every other net-worth computation includes (`liquid + liabilities + investments`). On a ledger with liability accounts, each corrected spike day's netWorth point is overstated by |liabilities|. + - Fix: include `liabilities` in the corrected recomputation, matching the standard formula; pin with a unit test that has a nonzero liabilities balance on a spike day. + +- [x] **Migration 0076 regressed the account-sync trigger's ON CONFLICT arbiter β€” onboarding any NEW `bank_account` label fails with 42P10 at schema head** ⏫ βœ… 2026-07-28 Β· df0314f (new migration 0083 CREATE OR REPLACEs the trigger function with 0076's exact body + the corrected (lower(btrim(name))) arbiter for already-migrated installs; 0076 fixed in place for installs below it, duplication documented in both files; both 0076 behaviors preserved verbatim. Proven both ways: scratch already-migrated install β€” 42P10 reproduced, then 0082β†’0083 upgrade only, then two casings of a new label onboard exactly one trimmed account β€” and fresh DB via test:db 3175/3175 incl. the flipped pinned test) + - β†ͺ _from: Orchestration session 2026-07-28 Β· real-DB harness migration (mock suites could never catch this β€” the arbiter is only validated when the trigger executes on the real schema)_ + - `alembic/versions/0066_normalized_account_identity.py` dropped `uq_accounts_name` for the expression index `uq_accounts_name_norm` on `lower(btrim(name))` and retargeted the sync trigger to `ON CONFLICT (lower(btrim(name)))`; `alembic/versions/0076_account_trigger_blank_and_case.py:77` rewrote the trigger and regressed the arbiter back to `ON CONFLICT (name)` β€” which matches NO unique index at head. INSERTing a transaction whose `bank_account` has no existing account raises `there is no unique or exclusion constraint matching the ON CONFLICT specification`. Existing labels (any casing) still resolve β€” only first-seen-label onboarding (e.g. the first CSV import for a new account) blows up. Pinned by the clearly-marked test in `tests/transactionRepository.db.test.js` ("0076 ON CONFLICT regression"). + - Fix: new migration re-creating the trigger function with the `(lower(btrim(name)))` arbiter (the function lives in the DB, so already-migrated installs need the CREATE OR REPLACE β€” editing 0076 in place is not enough); flip the pinned test to expect successful onboarding. + +- [x] **`transactionRepository.update()` RETURNING enrichment resolves only 2 category levels β€” response disagrees with an immediate getById on alias-recipient rows** πŸ”½ βœ… 2026-07-28 Β· a029b5d (both enrichment queries now use the shared 3-level join fragments and return effective_category_id; flipped DB pin asserts deep equality with an immediate getById through both the fields and fields+tags PATCH paths) + - β†ͺ _from: Orchestration session 2026-07-28 Β· real-DB harness migration_ + - `update()`'s RETURNING join omits the primary-recipient default-category fallback (and `COALESCE(pr.name, r.name)`): updating a row categorised via its primary's default returns `category_name: null` / the alias name, while `getById`/`getAll`/`create` return the 3-level resolution. Pinned in `tests/transactionRepository.db.test.js` ("update() response resolves only 2 category levels"). + - Fix: align update()'s RETURNING join with the 3-level pattern used by the read paths. + +- [x] **Statistics breakdown/pivot resolve categories 2-level β€” alias rows categorised via their primary are UNCATEGORISED in the breakdown and entirely absent from the pivot** πŸ”Ό βœ… 2026-07-28 Β· 0ffaed6 (breakdown + pivot now 3-level β€” pivot's COALESCE fixed consistently in all four embedded sites; the mv_category_totals fast path embedded the same 2-level expression, so its definition is corrected and migration 0084 drops the stale MV on existing installs (live query serves until the rebuild populates β€” no wrong-answer window); flipped DB pins assert breakdown/pivot agree with the transactions list; verified via test:db 3175/3175) + - β†ͺ _from: Orchestration session 2026-07-28 Β· real-DB harness migration_ + - `getCategoryBreakdown` and `getCategoryPivot` (infoRepositoryStatistics.js) use `COALESCE(t.category_id, r.default_category_id)` with no primary-recipient fallback: a row recorded under an alias whose primary carries the default is categorised in the transactions list but UNCATEGORISED in the breakdown, and absent from the pivot (its WHERE requires the 2-level COALESCE non-NULL). Corollary: the mock suite's "missing category_id β†’ Uncategorised" pivot case is unreachable through the real query. Pinned in `tests/infoRepoStatistics.db.test.js`. + - Fix: extend both queries' COALESCE with the primary-recipient default (matching transactionRepository's 3-level pattern); flip the pinned test. + +- [ ] **Remaining 2-level category-resolution surfaces: monthly summary (live + MV), recurring detection, sankey** πŸ”½ + - β†ͺ _from: Orchestration session 2026-07-28 Β· category-resolution fix pass (same pattern, outside that pass's scope)_ + - `services/materializedViewService.js:71` (`mv_monthly_summary` definition) and the monthly live path `repositories/infoRepositoryMonthly.js:184`, `services/recurringDetectionService.js:157`, and `services/calculations/aggregation/sankey.js:51,73` all still use the 2-level `COALESCE(t.category_id, r.default_category_id)` β€” alias rows categorised via their primary's default are treated as uncategorised on these surfaces, now inconsistent with the fixed transactions/breakdown/pivot 3-level resolution. + - Fix: extend each to the canonical 3-level pattern; the MV change needs the same drop-and-rebuild migration treatment as 0084 (mv_monthly_summary drop). + + + + - [x] **AddInvestmentDialog: initial purchase silently dropped (success toast) β€” or guaranteed 400 after the investment row already exists** ⏫ βœ… 2026-07-11 Β· 750022d - β†ͺ _from: Correctness research 2026-07-02 Β· Wave 2a (residue, closed 2026-07-03)_ - Buy leg only sent `if (amount > 0)` (`AddInvestmentDialog.tsx:101-119`): amount empty/0 with "add initial purchase" ON β†’ investment created, buy skipped, success toast `:121` (amount labeled required at `InvestmentFormFields.tsx:226` but no `required` attr `:228-237`); unit-based with units empty β†’ amount-only buy fails `portfolioTxRepo.common.js:109-116` β†’ raw 400 with the investment already committed (see the enrichment on the duplicate-investment finding above). `initialDate` clearable to `''` β†’ 400 (`investmentController.js:404-406`). @@ -831,7 +870,7 @@ look-changing one. - `EditPortfolioTxnDialog.tsx:150,290-298` (`min="0"` permits 0) vs `portfolioTxRepo.common.js:164-166` ("must be positive"). - Fix: change the field's `min` to just above 0 (or validate `> 0` client-side). -- [ ] **Currency free-text class: multiple surfaces let malformed currency codes reach a raw 400/500** πŸ”½ πŸ”Ž verified-present 2026-07-11 πŸ”Ž partial-453662d 2026-07-14 (ISO-4217 assertCurrency now wired into POST/PATCH /api/transactions so free-typed codes 400 instead of 500; other currency-accepting surfaces still open) πŸ”Ž partial-6566efa 2026-07-14 (currency now validated (ISO shape) on portfolio-tx + investment create/update too; remaining non-transaction surfaces still open) πŸ”Ž partial-96f1122 2026-07-14 (watchlist currency now normalized to uppercase on write (matches assertCurrency); the multi-surface class still has other surfaces open) +- [ ] **Currency free-text class: multiple surfaces let malformed currency codes reach a raw 400/500** πŸ”½ πŸ”Ž verified-present 2026-07-11 πŸ”Ž partial-453662d 2026-07-14 (ISO-4217 assertCurrency now wired into POST/PATCH /api/transactions so free-typed codes 400 instead of 500; other currency-accepting surfaces still open) πŸ”Ž partial-6566efa 2026-07-14 (currency now validated (ISO shape) on portfolio-tx + investment create/update too; remaining non-transaction surfaces still open) πŸ”Ž partial-96f1122 2026-07-14 (watchlist currency uppercased on write) πŸ”Ž partial-9843677 2026-07-28 (full-surface sweep with per-surface verdicts: planned POST/PATCH, portfolio-tx PATCH, generic+vision CSV adapters (shared normalizeIsoCurrency β€” free-text cells no longer 500 whole commits at the 0046 CHECK), portfolio-import new-holding + repo VALIDATION_ERRORβ†’400 translation all fixed; accounts/investments/watchlist/reports/info confirmed already-validated. LEFT by design: compute-only rebalance/forecast currency params (degrade to logged 1:1 fallback, no write), bank-specific machine-export adapters, dbEditor, and the conservative app_settings blob guard) - β†ͺ _from: Correctness research 2026-07-02 Β· Wave 2a (residue, closed 2026-07-03)_ - AddAccountDialog: `<3` letters/digits β†’ `accountService.js:48-51` `^[A-Z]{3}$` raw 400 (`AddAccountDialog.tsx:194-199`; the `|| "EUR"` at `:101` rescues empty only). Transaction PATCH: free text, no check either side in code (`TransactionInfoDialog.tsx:78-79,158`) β€” the 0046 DB CHECK turns `"euro"`/`"€"` into a raw 500. `AddTransactionDialog.tsx:119` `maxLength={10}` vs `transactions.currency` VARCHAR(3) (`0001_initial_database_schema.py:190`) + ISO CHECK (`0046_currency_integrity.py:65`) β†’ 4-10 chars = PG 22001/CHECK 500. - Fix: validate ISO-4217 client-side (regex or a fixed currency list) before submit on all three surfaces. @@ -841,7 +880,7 @@ look-changing one. - No max client-side (`AddToWatchlistDialog.tsx:276-284`, `WatchlistChartDialog.tsx:165-172`) or server-side (`watchlist.js:20-24`, `min: 0` inclusive β€” a 0 alert target is meaningless for the at/below check `WatchlistPage.tsx:149-150`); column `0001…py:542` caps ~1e12; `1e999` β†’ `Infinity` β†’ `parseDecimal` fallback 0 β†’ PATCH sets 0 with a success toast (`lib/decimal.ts:13-17`, `WatchlistChartDialog.tsx:95`). Negative target: frontend allows it, backend 400s. - Fix: reject `≀0` and cap at the column's max both sides; reject non-finite instead of falling back to 0. -- [ ] **Investments API hygiene: unknown asset_class 500s, symbol uniqueness only enforced on update, empty-name Save silently no-ops** πŸ”½ πŸ”Ž verified-present 2026-07-11 πŸ”Ž partial-08c8250 2026-07-14 (unknown asset_class on create now throws a validation error β†’ 400 (test strengthened); LEFT: symbol-uniqueness-on-update and empty-name Save feedback are frontend/other, not addressed here) πŸ”Ž partial-453662d 2026-07-14 (create() now rejects empty name and trims/uppercases symbol to match update(); symbol-uniqueness-on-create still not enforced) +- [x] **Investments API hygiene: unknown asset_class 500s, symbol uniqueness only enforced on update, empty-name Save silently no-ops** πŸ”½ πŸ”Ž verified-present 2026-07-11 πŸ”Ž partial-08c8250 2026-07-14 (unknown asset_class on create now throws a validation error β†’ 400 (test strengthened); LEFT: symbol-uniqueness-on-update and empty-name Save feedback are frontend/other, not addressed here) πŸ”Ž partial-453662d 2026-07-14 (create() rejects empty name, trims/uppercases symbol) βœ… 2026-07-28 Β· 9843677 (symbol uniqueness now enforced on create via the same ensureSymbolIsUnique as update β€” identical error shape/status, pinned by repo tests; empty-name Save feedback fixed in both dialogs with explicit toasts β€” the global mutation toaster alone could NOT close it because both dialogs bare-return before any mutation fires and HTML required misses whitespace-only names) - β†ͺ _from: Correctness research 2026-07-02 Β· Wave 2a (residue, closed 2026-07-03)_ - Unknown `asset_class` β†’ plain `Error` β†’ 500 not 400 (`investmentRepository.js:220`; legacy view-schema installs insert arbitrary classes `:476-509`). Symbol uniqueness + trim/uppercase normalization enforced on **update only** (`:537-541` vs create `:417`; no DB unique index on symbol; `EditInvestmentDialog.tsx:94` uppercases, `AddInvestmentDialog.tsx:79` doesn't β€” compounds the filed duplicate-on-retry bug). Empty-name Save silently no-ops (`EditInvestmentDialog.tsx:85` bare `return`, no feedback; backend would accept `''` β€” `investmentRepository.js:95-110` has no non-empty check). - Fix: validate `asset_class` against the enum before insert (400 not 500), apply the same normalization at create time as update, and give empty-name Save an explicit rejection with feedback. @@ -851,7 +890,7 @@ look-changing one. - `PortfolioImportPage.tsx:211-217` (`min="0"` attr only; `parseInt || 0` keeps negatives) and `portfolioImportRoutes.js:100`; csv-parse throws `Invalid Option: from must be a positive integer` (verified by executing csv-parse). - Fix: clamp to `Math.max(0, n)` client-side and/or validate server-side before passing to csv-parse. -- [ ] **Backend has ZERO length validation anywhere β€” `sanitizeString` is dead code** πŸ”½ πŸ”Ž verified-present 2026-07-11 πŸ”Ž partial-453662d 2026-07-14 (throwing assertMaxLength added and wired to the VARCHAR(100) bank_account mirror on create/PATCH; broad systemic length coverage still open) πŸ”Ž partial-6566efa 2026-07-14 (assertMaxLength now also on watchlist + investment string fields; broad systemic length coverage still open) +- [ ] **Backend has ZERO length validation anywhere β€” `sanitizeString` is dead code** πŸ”½ πŸ”Ž verified-present 2026-07-11 πŸ”Ž partial-453662d 2026-07-14 (throwing assertMaxLength added and wired to the VARCHAR(100) bank_account mirror on create/PATCH; broad systemic length coverage still open) πŸ”Ž partial-6566efa 2026-07-14 (assertMaxLength on watchlist + investment string fields) πŸ”Ž partial-9843677 2026-07-28 (all remaining VARCHAR write surfaces now capped to their actual column widths: recipient_bank_accounts.account_numberβ†’34, investment price_provider_idβ†’200, provider urlsβ†’500, provider pathsβ†’300. Systematic survey result: every other directive-named field (transactions memo/comment, recipients, categories, planned strings, accounts name/display_name/institution, tags, splits note) is a TEXT column with zero app-level-cap precedent in the codebase β€” deliberately left; closing this finding fully would need a decided TEXT-cap policy first) - β†ͺ _from: Correctness research 2026-07-02 Β· Wave 2a (residue, closed 2026-07-03)_ - `middleware/validation.js:78` `sanitizeString` has no call sites (`validateDateString` likewise dead) β€” an intended-but-unwired sanitization layer; the only real protection is frontend `maxLength` + the DB column width. Real exposure is narrow (core text columns are TEXT): `manual_raw_transactions.bank_account` VARCHAR(100) (`0001:431`) β€” frontend cap is exact-match, so an API-length overflow 500s the raw-mirror insert *after* the main insert already succeeded (mid-operation failure; `transactions.bank_account` is TEXT, so the sinks diverge); `watchlist.name/symbol/price_provider_id` (200/20/200, `0001:539-545`) and `investments.name` (200, `0001:473`) are provider-/market-prefilled β€” HTML `maxLength` doesn't clamp programmatic values. - Fix: wire `sanitizeString`/`validateDateString` into the routes they were built for, or delete them if genuinely superseded. @@ -866,7 +905,7 @@ look-changing one. - Restore flow asks for a "wachtwoord" where backup set a "wachtzin" (same secret, two different names, in a security-critical flow: `settings.restore.passphrase*` vs `settings.backup.passphrase.*`) Β· `importReview.toast.persistDefaultFailed` "Standaard ontvanger opslaan mislukt" β€” wrong referent (reads "failed to save the default recipient"; sibling `importReview.persistDefault` gets it right) Β· `recurring.loading` "Terugkerende detectie" (modifier on the wrong noun) Β· `transactions.memo` "Omschrijving" collides with the Description label (also "Omschrijving") Β· `dashboard.greetingAfternoon` "Goedenmiddag" should be "Goedemiddag". - Fix: correct each string; unify the backup/restore passphrase terminology first (security-critical UX). -- [ ] **Backlog batch: assorted low-severity validation/consistency gaps across planned/portfolio-tx/splits/watchlist/accounts/tax** ⏬ πŸ”Ž verified-present 2026-07-11 πŸ”Ž partial-e501502 2026-07-14 (splits/batch sub-item fixed: all-dropped input now returns 400 instead of 201 {total:0}; the other planned/portfolio-tx/watchlist/accounts/tax sub-items in this bundle remain open) πŸ”Ž partial-8fe6720/7b8ed98 2026-07-14 (two more sub-items fixed: planned is_recurring-without-pattern now 400; watchlist empty-name + added_price-on-PATCH now rejected β€” bundle still has other open sub-items) πŸ”Ž partial-453662d 2026-07-14 (splits sub-item fixed: POST / and /batch validate transaction_id/recipient_id/amount before Postgres (was FK/type 500); other planned/portfolio-tx/watchlist/accounts/tax sub-items remain) πŸ”Ž partial-6566efa 2026-07-14 (planned (reminder_days_before PATCH whitelist + 0..365 guard, zero/absurd-amount reject), portfolio-tx (type + recurrence_interval whitelist, recurrence_end_date>=date, clear-stale-on-off) and account statement_balance bound sub-items fixed; splits/tax + other sub-items remain) +- [ ] **Backlog batch: assorted low-severity validation/consistency gaps across planned/portfolio-tx/splits/watchlist/accounts/tax** ⏬ πŸ”Ž verified-present 2026-07-11 πŸ”Ž partial-e501502 2026-07-14 (splits/batch sub-item fixed: all-dropped input now returns 400 instead of 201 {total:0}; the other planned/portfolio-tx/watchlist/accounts/tax sub-items in this bundle remain open) πŸ”Ž partial-8fe6720/7b8ed98 2026-07-14 (two more sub-items fixed: planned is_recurring-without-pattern now 400; watchlist empty-name + added_price-on-PATCH now rejected β€” bundle still has other open sub-items) πŸ”Ž partial-453662d 2026-07-14 (splits sub-item fixed: POST / and /batch validate transaction_id/recipient_id/amount before Postgres (was FK/type 500); other planned/portfolio-tx/watchlist/accounts/tax sub-items remain) πŸ”Ž partial-6566efa 2026-07-14 (planned reminder/amount guards, portfolio-tx whitelists, account statement_balance bounds) πŸ”Ž partial-9843677 2026-07-28 (full sub-item audit table produced; fixed this pass: planned PATCH is_recurring-without-pattern guard (only when the PATCH touches recurrence fields, so legacy broken rows stay editable β€” pinned), planned PATCH amount bounds (0/1e15/Infinity/null/'' previously reached the DB), CSV mapper duplicate-column non-blocking warning, phantom `balance` removed from frontend update type + openapi + handleUpdate, inline amount-clear no longer parses '' to 0 (VirtualDataTable keeps raw string while editing). Confirmed fixed-by-earlier-pass: splits batch all-or-nothing, watchlist PATCH guards, tax mortgageStartYear + profileConfigured. LEFT: the nl copy/terminology sweep sub-item only (subjective bulk copy calls, cleanly separable)) - β†ͺ _from: Correctness research 2026-07-02 Β· Wave 2a (residue, closed 2026-07-03)_ - Planned: API-only `is_recurring:true` without a pattern stores and is perpetually due after execution (`plannedTransactions.js:211` guard fires only when a pattern is present; `recurrence.js:55`) Β· `reminder_days_before` creatable + returned but missing from the PATCH whitelist β†’ updates silently dropped (`validation.js:26-33`) Β· zero/absurd amounts end-to-end (`PlannedPaymentForm.tsx:58` blocks only empty; `plannedTransactions.js:181` null-check only; zero at least excluded from auto-match, `plannedMatchService.js:63`). - Portfolio-tx: `type`/`currency`/`recurrence_interval` have no backend whitelist or DB CHECK (`type:'banana'` inserts and is invisible to units replay, `common.js:222-266`) Β· recurrence fields are stored-but-inert metadata β€” grep finds NO backend consumer of `is_recurring`/`recurrence_interval`/`recurrence_end_date` for portfolio txns (**CONFIRMED 2026-07-10 (D9): badge-only IS the intended design β€” do not file the missing consumer as a bug; remaining work is hygiene only: whitelist interval values, validate `end β‰₯ start`, clear stale interval/end-date on recurrence-off**) Β· turning recurrence off leaves stale interval/end-date stored (`EditPortfolioTxnDialog.tsx:153-155`). @@ -901,7 +940,7 @@ look-changing one. - `assertDashboardSettingsValue` (`routes/settings.js:57-60`) checks `typeof value !== 'object' || Array.isArray(value)` but not `null` (the theme validator at `:33` does it right); `null.excludedCategoryIds` throws a TypeError. - Fix: add an explicit `value === null` check alongside the existing type/array checks. -- [ ] **Settings write route validates only 5 keys β€” every other settings key accepts arbitrary JSON** πŸ”½ πŸ”Ž partial 2026-07-11 (belgian_tax_profile* blobs now validated in validateSettingValue; app_settings/backup_settings/widget_visibility/onboarding_complete/unknown keys still accept arbitrary JSON) πŸ”Ž partial-453662d 2026-07-14 (onboarding_complete/app_settings/backup_settings/widget_visibility now type-guarded in validateSettingValue; unknown keys still accept arbitrary JSON) +- [x] **Settings write route validates only 5 keys β€” every other settings key accepts arbitrary JSON** πŸ”½ πŸ”Ž partial 2026-07-11 (belgian_tax_profile* blobs now validated in validateSettingValue; app_settings/backup_settings/widget_visibility/onboarding_complete/unknown keys still accept arbitrary JSON) πŸ”Ž partial-453662d 2026-07-14 (onboarding_complete/app_settings/backup_settings/widget_visibility type-guarded) βœ… 2026-07-28 Β· 9843677 (unknown keys now rejected with a 400 naming the known-key list on single and bulk PUT β€” decision grounded in a grep of every writer: all 15 frontend saveSetting call sites, the electron backup_settings writer and the repository-side fx flag use fixed keys, no dynamic-key writer exists; three first-party keys missing from validation entirely (dismissed_recurring_patterns, portfolio_tax_adjustments_v1, portfolio_tax_classifications_v1) got shape schemas so they are known AND type-guarded; GET/DELETE stay unrestricted so restored-backup legacy keys remain listable/cleanable; unknown-key 400 + real-payload acceptance tests added) - β†ͺ _from: Correctness research 2026-07-02 Β· Wave 2b (residue, closed 2026-07-03)_ - `validateSettingValue` (`routes/settings.js:190-207`) covers dashboard/theme/cost-basis/includeTransfers/rebalance_plans; `app_settings`, `backup_settings`, `widget_visibility`, `onboarding_complete`, tax-profile blobs, and ANY unknown key accept arbitrary JSON (≀1MB body, key ≀100 chars, unbounded key count). - No backend consumer ingests settings into SQL/math unguarded (`plannedMatchService.js:92-95` tolerant, `portfolioSummaryService` set-membership, `getIncludeTransfers === true`), so exposure today is garbage-in for frontend blobs (e.g. `defaultPageSize:"abc"` survives the `migrateAppSettings` spread) β€” low severity on a single-user install, but the Belgian-tax-profile finding above shows the concrete cost when a settings blob DOES feed real math. @@ -981,13 +1020,13 @@ look-changing one. - In packaged mode every `composeStartOrUp` path returns `built:false` (`:1287` β€” `!skipBuild && (!app.isPackaged || useRepoMode)`), so an image-pull update polls with the 60s budget, not the 3-minute build budget. A migration pushing backend-listen past ~56s β†’ `pollReady` rejects β†’ `loadErrorPage()` + blocking "taking longer than expected" dialog *mid-migration*; when alembic later finishes, nothing re-navigates (watchdog only starts after a successful `pollReady`, `:1128`) β€” the user is stranded on the error page until manual Retry. - Fix: treat "new image pulled / pending migrations" as `building`, or expose a backend "migrating" signal (splash status line) and extend the poll budget while it's set. -- [ ] **Transaction search requests are never aborted β€” superseded keystrokes leave the expensive OR-chain scans running to completion server-side, and nothing gates 1-character queries client-side** ⏫ πŸ”Ž verified-present 2026-07-11 πŸ”Ž partial-c6b2327 2026-07-14 (superseded transaction-search requests now abort via the React Query signal threaded through the api client; the client-side 1-char min-length gate was left (a visible behavior change needing sign-off)) +- [ ] **Transaction search requests are never aborted β€” superseded keystrokes leave the expensive OR-chain scans running to completion server-side, and nothing gates 1-character queries client-side** ⏫ πŸ”Ž verified-present 2026-07-11 πŸ”Ž partial-c6b2327 2026-07-14 (abort signal threaded) βœ… 2026-07-28 Β· f5fb531 (user signed off with a THREE-character threshold: shared VirtualDataTable forwards the trimmed term only at >=3 chars (SERVER_SEARCH_MIN_LENGTH) and resets to '' below β€” unfiltered, never stale, input text preserved; RecipientsPage's server search gets the same gate; cross-referenced with backend MIN_SEARCH_LENGTH for the planned server-side companion) - β†ͺ _from: Performance research 2026-07-09 Β· Wave F3 (frontend residues)_ - `apps/frontend/src/features/transactions/hooks/useTransactionListData.ts:102-121` (main query), `:149-168` (loadMore) β€” `queryFn` never receives/forwards React Query's abort `signal`; `apps/frontend/src/lib/api/transactions.ts:16-52` (`getTransactions`) has no `signal` param even though `client.ts:229-259` fully supports `options.signal` - Each debounced (300ms) keystroke changes the queryKey and starts a fresh backend search; React Query drops the stale query client-side but the HTTP request is NOT aborted, so the known-unindexable OR-chain scan (see the filed ⏫ search finding) runs to completion for every intermediate term β€” fast typing stacks N concurrent full scans server-side, competing for the 10-connection pool. Compounding it, `VirtualDataTable.tsx:164-175` forwards ANY non-empty value with no minimum-length gate, so typing "a" issues a search matching nearly every row (the filed backend search finding already lists a server-side min-length as part of its fix β€” the client half is missing too). - Fix: thread React Query's `signal` into `queryFn` and add a `signal?: AbortSignal` param to `getTransactions` forwarded to `client.ts` (which already honors it); gate `handleSearchInput` on `value.trim().length >= 2`, resetting via `onSearchChange("")` below the threshold. -- [ ] **Migration 0050 rewrites the entire `transactions` heap (account_id backfill on every row) and builds two non-concurrent indexes on it β€” the most expensive migration in the recent update window** ⏫ πŸ”Ž verified-present 2026-07-11 +- [x] **Migration 0050 rewrites the entire `transactions` heap (account_id backfill on every row) and builds two non-concurrent indexes on it β€” the most expensive migration in the recent update window** ⏫ πŸ”Ž verified-present 2026-07-11 πŸ”Ž partial-7eed4c7 2026-07-27 (the migration half is DONE: backfill UPDATE batched in 50k id-ranges inside an autocommit_block (resumable via the account_id IS NULL guard), both transactions indexes now built after the backfill with CREATE INDEX CONCURRENTLY incl. invalid-leftover recovery; env.py already ran transaction_per_migration so the change is scoped inside 0050; verified on fresh DB (test:db green, 3112 tests) and a populated pre-0050 DB (120k rows, sparse ids: upgrade/idempotent re-run/downgrade/invalid-index recovery). LEFT at that point: the appended dialog-overlay blur finding) βœ… 2026-07-28 Β· 395dd5e (dialog-blur half closed with the user-chosen fx-tier middle path: modal-overlay class renders a flat background/0.6 dim at reduced/standard visual-effects tiers and the pixel-identical backdrop-blur-md frost at fx-enhanced via the ADR-075 root-class mechanism β€” the rich look is a settings toggle away; overlays in dialog/alert-dialog/sheet all converted, aurora and modal content glass untouched) - β†ͺ _from: Performance research 2026-07-09 Β· Wave F1_ - `alembic/versions/0050_add_accounts_entity.py:122-131` (3Γ— `CREATE INDEX`, incl. partial B-tree `idx_transactions_account_date_active` over all active rows), `:167-183` (`UPDATE transactions SET account_id = a.id ... WHERE account_id IS NULL` β€” touches every row), `:134-164` (several full `DISTINCT`/`DISTINCT ON` scans) - On a 500k-row table: full heap rewrite (500k dead + 500k new tuples, full WAL, maintenance of the just-built indexes) plus two full index builds under a write-blocking lock β€” inside the single boot transaction of the finding above. Anyone updating from a pre-ADR-088 version crosses this. @@ -3660,7 +3699,7 @@ look-changing one. - evidence: 204+empty: `routes/watchlist.js:89`, `routes/savedCharts.js:169`, `routes/ai.js:217`, `routes/importRoutes.js:234`, `routes/portfolioImportRoutes.js:299`, `controllers/investmentController.js:377,483`. 200+`{message,…}`: `routes/transactions.js:646`, `routes/recipients.js:94`, `routes/categories.js:82`, `routes/plannedTransactions.js:321`, `routes/tags.js:47`, `routes/accounts.js:49`, `routes/recipientBankAccounts.js:60`, `routes/splits.js:204`. 200+`{deleted:true}`: `routes/attachments.js:165`, `routes/settings.js:242`. 200+`{removed}`: `routes/research.js:235`. 200+`{ok:true}`: `routes/transactions.js:233`. 200+`{patternId}`: `routes/recipients.js:199`. Same operation, 6 contracts β€” a generic delete-mutation hook on the frontend can't exist. - fix: pick one convention (204 for hard delete, 200+entity for soft-delete/deactivate is a defensible split) and codify it in `docs/reference/code-patterns.md`; migrate outliers opportunistically since the frontend is the only consumer today. -- [ ] **List-response key drift: `items` vs `batches` vs bare arrays vs per-route domain keys** ⏫ πŸ”Ž verified-present 2026-07-11 πŸ”Ž partial-1581a34 2026-07-27 (the finding's own first targets are done: both {batches,…} endpoints β†’ {items,total,limit,offset} and all 7 bare-array-as-data GETs β†’ {items,total} (parser lists now share lib/parserConfigRoutes.js β€” one edit covers both; ai conversations; saved charts; admin providers/health, endpoints, endpoint-liveness), with clients unwrapping at the api boundary, openapi/generated/MSW/contract tests aligned (two spec drifts + a phantom {charts} MSW shape closed en route). Convention decision recorded: {items,total(,limit,offset)} in the data body is the wire convention β€” do NOT migrate counts into meta.pagination; the packages/types api.js meta.pagination doc and its sole emitter (routes/info/netWorth.js:61) are the sibling pagination finding's problem. LEFT: the domain-key endpoints (quotes/articles/points marketLookup.js; mappings/providers research.js; models ai.js:200; banks/adapters info/statistics.js), plannedTransactions.js's array-in-data + counts-in-meta pattern, and the uncited bare-array GET /api/admin/metrics/requests (admin.js:311) spotted during this pass β€” same one-line change as its migrated neighbors) +- [x] **List-response key drift: `items` vs `batches` vs bare arrays vs per-route domain keys** ⏫ πŸ”Ž verified-present 2026-07-11 πŸ”Ž partial-1581a34 2026-07-27 βœ… 2026-07-27 Β· b60cf41 (round 1 β€” 1581a34: both {batches,…} endpoints β†’ {items,total,limit,offset} and all 7 bare-array-as-data GETs β†’ {items,total} (parser lists now share lib/parserConfigRoutes.js β€” one edit covers both; ai conversations; saved charts; admin providers/health, endpoints, endpoint-liveness), with clients unwrapping at the api boundary, openapi/generated/MSW/contract tests aligned (two spec drifts + a phantom {charts} MSW shape closed en route). Convention decision recorded: {items,total(,limit,offset)} in the data body is the wire convention β€” do NOT migrate counts into meta.pagination; the packages/types api.js meta.pagination doc and its sole emitter (routes/info/netWorth.js:61) are the sibling pagination finding's problem. Round 2 β€” this pass closed the whole LEFT list: quotes/articles/points (marketLookupService payload builders; chart keeps symbol/currency alongside {items,total}); mappings/providers in research.js (both GET+the POST /mappings and PUT /provider-keys mutations that echo the same list, so one response type survives); models ai.js; banks/adapters info/statistics.js (total_count retired); plannedTransactions due-soon + match-suggestions (counts moved from meta INTO the data body; due-soon keeps `days` alongside); and bare-array GET /api/admin/metrics/requests. Frontend clients unwrap to rows at the api boundary (market chart re-surfaces items as `points` client-side), MSW handlers/contract tests/live-contracts/openapi+generated types/docs all aligned. Still domain-keyed by design, out of the finding's scope: research chart/news/quote data payloads ({points}/{articles} under the provenance meta contract) and GET /api/market/search + /api/splits/owed ({items} without total)) - β†ͺ _from: Architecture & code design 2026-07-06 Β· Wave W1 (REST API design)_ - evidence: canonical `{items,total,limit,offset}` in transactions/recipients/categories/planned/watchlist/investments (`routes/transactions.js:256-262`, `controllers/investmentController.js:189-195`). But: `{batches,total,limit,offset}` (`routes/importRoutes.js:362`, `routes/portfolioImportRoutes.js:307`); bare array as `data`: `routes/importRoutes.js:193` (GET /parsers), `routes/portfolioImportRoutes.js:261`, `routes/ai.js:166` (GET /conversations), `routes/savedCharts.js:83`, `routes/admin.js:294,312,317`; array-in-data + counts moved to envelope `meta`: `routes/plannedTransactions.js:231` (`res.ok(items,{days,total})`), `:245`; domain keys: `quotes` (`routes/marketLookup.js:279`), `articles` (`:367`), `points` (`:305`), `mappings` (`routes/research.js:198`), `providers` (`:254`), `models` (`routes/ai.js:149`), `banks`/`adapters` (`routes/info/statistics.js:27,36`). - fix: standardize on `{items, total}` (+`limit/offset` when paginated) for every collection GET; treat `batches`β†’`items` and the bare-array endpoints as the first migration targets since they also block ever adding pagination non-breakingly. @@ -3830,7 +3869,7 @@ look-changing one. - evidence: adding a section touches ~6 places: `sections/.js`; reports/index.js **three** spots (import :15-34, renderer map :414/:435/:454, default-order list :424/:444/:464); the matching data fetcher (dataFetcher*.js); the frontend's independently hand-maintained mirror lists (ExportDialog.tsx:58-90 `FINANCIAL/PORTFOLIO/TAX_SECTIONS`); i18n `export.section.*` en+nl. There is no `GET /sections` catalog, and unknown IDs are *silently filtered* server-side (`requested.filter(id => id in RENDERERS)` β€” reports/index.js:480,:503,:526), so a backend/frontend ID typo yields a PDF that quietly omits the section rather than erroring. - fix: per report type, export one `[{id, render, default}]` array from reports/index.js (derive renderer map + default list from it) and either expose it via a small catalog endpoint for ExportDialog or at least make generateReport reject unknown section IDs instead of silently dropping them. -- [ ] **Real-DB test harness built but adoption stalled at 1 suite; TEST_DATABASE_URL set nowhere, so its tests silently skip everywhere** ⏫ πŸ”Ž verified-present 2026-07-11 πŸ”Ž partial-c2152e5 2026-07-27 (the DECIDED wiring is done and was executed against a live PG16 cluster stood up in-session: CI test-backend now runs a postgres:18 service + alembic migration with TEST_DATABASE_URL set (via new scripts/db-migrate.js, which delegates to runMigrations() because bare `alembic upgrade head` cannot migrate a fresh DB β€” VARCHAR(32) version column vs longer revision ids); local `bun run test:db` runs a disposable tmpfs container. Turning it on exposed that the one 'adopted' suite had NEVER run β€” 3 of aggregationRefresh's 4 DB cases asserted pre-squash schema artifacts (fixed from the real schema) and 2 cashflowForecastMethods tests passed only when Postgres was unreachable (repo now mocked with the documented 42P01 fallback). First migrated suite landed: transferReconciliation.db.test.js, 24 live-DB tests for the previously-untested 254-line service. Also fixed: .gitignore `*.db*` silently swallowed any *.db.test.js file. 3108 pass with DB (deterministic across runs), 3080 without. LEFT: migrating the remaining SQL-order-choreography mock suites (infoRepo*, transactionRepository*) onto the harness, incrementally as the fix line says) +- [ ] **Real-DB test harness built but adoption stalled at 1 suite; TEST_DATABASE_URL set nowhere, so its tests silently skip everywhere** ⏫ πŸ”Ž verified-present 2026-07-11 πŸ”Ž partial-c2152e5 2026-07-27 (the DECIDED wiring is done and was executed against a live PG16 cluster stood up in-session: CI test-backend now runs a postgres:18 service + alembic migration with TEST_DATABASE_URL set (via new scripts/db-migrate.js, which delegates to runMigrations() because bare `alembic upgrade head` cannot migrate a fresh DB β€” VARCHAR(32) version column vs longer revision ids); local `bun run test:db` runs a disposable tmpfs container. Turning it on exposed that the one 'adopted' suite had NEVER run β€” 3 of aggregationRefresh's 4 DB cases asserted pre-squash schema artifacts (fixed from the real schema) and 2 cashflowForecastMethods tests passed only when Postgres was unreachable (repo now mocked with the documented 42P01 fallback). First migrated suite landed: transferReconciliation.db.test.js, 24 live-DB tests for the previously-untested 254-line service. Also fixed: .gitignore `*.db*` silently swallowed any *.db.test.js file. 3108 pass with DB (deterministic across runs), 3080 without. LEFT at that point: infoRepo*/transactionRepository* mock suites) πŸ”Ž partial-c155d78 2026-07-28 (second increment: transactionRepository (34 live-DB tests) + infoRepoStatistics (8) migrated; DB suites now serialize via a pg advisory lock in tests/setup/db.js (parallel workers deleted each other's fixtures once a 2nd suite existed) and with-test-db.sh probes readiness over TCP (socket probe raced initdb's temporary server). The real DB exposed 3 mock-masked discrepancies, pinned + filed as findings below β€” incl. the ⏫ 0076 ON CONFLICT onboarding regression. test:db 3175/3175. LEFT: infoRepoBanks, infoRepoForecast, infoRepoMonthly, infoRepository (854 lines), infoRepositoryRecipients β€” advisory-lock pattern makes each drop-in) - β†ͺ _from: Architecture & code design 2026-07-06 Β· Wave W4 (test architecture)_ - evidence: `apps/node-backend/tests/setup/db.js:1-53` is a deliberate opt-in real-PG seam ("Phase 0 step 6": `getTestPool()` + `it.skipIf(!hasTestDatabase())`), but only one production suite uses it (`tests/services/aggregationRefresh.test.js:37` `describe.skipIf(!hasTestDatabase())`) plus the harness self-test (`tests/setup/db.test.js`). `TEST_DATABASE_URL` appears in zero workflows and zero package.json scripts β€” the gated cases skip in CI *and* in every default local run. Meanwhile 60 suites mock the pool (`vi.mock('../src/database/connection.js')` 48Γ— + 12Γ— two-levels-deep), with 48 SQL-string assertions across 11 files, and multi-step `mockResolvedValueOnce` choreographies encoding exact query order (e.g. `tests/infoRepoMonthly.test.js:33-50`: MV probe β†’ currency probe β†’ data query, pinned fake clock). DB-bound orchestration like `src/services/transferReconciliationService.js` (254 lines) has no direct test at all β€” only its pure-calc extract is golden-tested (`tests/services/transfers.golden.test.js` β†’ `calculations/transfers.js`, 87 lines) and the service is mocked in 3 bulk-route tests. These are exactly the tests the harness was built for. - fix: **DECIDED 2026-07-10: wire the real-DB harness.** Add a Postgres service to CI (compose service or testcontainers) + a local script wiring `TEST_DATABASE_URL`, then migrate the SQL-order-choreography suites (infoRepo*, transactionRepository*, transferReconciliation) onto it incrementally. The mock-only alternative is rejected β€” do not delete `setup/db.js`. @@ -3880,7 +3919,7 @@ look-changing one. - evidence: `src/lib/api/__tests__/coverage-clients.test.ts` (537 lines) + `-2` (494) + `-3` (339) batch-import dozens of thin fetch-wrapper functions and exercise them against MSW. They aren't worthless (they route through the contract-checked handlers), but the naming is candid: these exist to feed the threshold, and they pad the "statements covered" figure with the easiest code in the app while the harder gaps (features/, contexts/ β€” see the include-list finding) sit unmeasured. - fix: no urgent action; when the coverage include-list is fixed and re-ratcheted, consider folding these into per-module client tests so file names describe behavior, not the metric. -- [ ] **checkJs CI gate is nominally load-bearing: `strict:false` + `noImplicitAny:false` and the data layer is untyped** ⏫ πŸ”Ž verified-present 2026-07-11 πŸ”Ž partial-6345da9 2026-07-27 (the fix's structure is in place: src/types/rows.js with 29 pg-accurate row typedefs (raw vs formatted shapes distinct; NUMERIC/BIGINT=string, DATE=Date β€” no setTypeParser exists), repositories/ annotated 101β†’324 @param / 51β†’206 @returns, and a working noImplicitAny ratchet (tsconfig.check.strict.json + scripts/checkjs-ratchet.js, wired into CI + pre-push + package.json; verified to fail on a removed annotation; a directory-scoped 2nd tsconfig was measured and rejected β€” doesn't narrow the program, breaks ambient types under bun). 24 files ratcheted at 0 errors (src/types + 23 of 40 repositories). Ratchet options were measured: global noImplicitAny = 3349 errors. LEFT: the remaining 17 repositories/ files β€” 335 errors, concentrated in infoRepositoryForecast (55), infoRepositoryHelpers (36), savedChartsRepository (31) β€” then per the fix line, further directories) +- [ ] **checkJs CI gate is nominally load-bearing: `strict:false` + `noImplicitAny:false` and the data layer is untyped** ⏫ πŸ”Ž verified-present 2026-07-11 πŸ”Ž partial-6345da9 2026-07-27 (the fix's structure is in place: src/types/rows.js with 29 pg-accurate row typedefs (raw vs formatted shapes distinct; NUMERIC/BIGINT=string, DATE=Date β€” no setTypeParser exists), repositories/ annotated 101β†’324 @param / 51β†’206 @returns, and a working noImplicitAny ratchet (tsconfig.check.strict.json + scripts/checkjs-ratchet.js, wired into CI + pre-push + package.json; verified to fail on a removed annotation; a directory-scoped 2nd tsconfig was measured and rejected β€” doesn't narrow the program, breaks ambient types under bun). 24 files ratcheted at 0 errors (src/types + 23 of 40 repositories). Ratchet options were measured: global noImplicitAny = 3349 errors. LEFT at that point: 17 repositories/ files) πŸ”Ž partial-a1c3a8d 2026-07-27 (repositories/ COMPLETE: all remaining 17 files annotated to 0 errors (335β†’0; incl. infoRepositoryForecast 55, infoRepositoryHelpers 36), 8 new pg-accurate typedefs in src/types/rows.js verified against the Alembic migrations, and the ratchet list collapsed to directory prefixes src/types/ + src/repositories/ so new repository files are ratcheted from birth β€” ratchet verified to still bite (removed annotation β†’ exit 1). 41 files clean. LEFT: further directories per the fix line (services/, routes/, lib/ remain un-ratcheted)) - β†ͺ _from: Architecture & code design 2026-07-06 Β· Wave W5 (cross-cutting concerns)_ - evidence: `apps/node-backend/tsconfig.check.json:10-11` (`"strict": false, "noImplicitAny": false`), run in CI at `.github/workflows/ci.yml:200`. Under these settings untyped params are silent `any`, so the check only catches gross misuse. Core data layer is where typing is absent: `src/repositories/transactionRepository.js` has 14 exports and **0** `@param`/`@returns`; all of `src/repositories/` (41 files) totals 71 `@param` vs 411 in `services/`. Only 15 `@typedef` sites repo-wide and **no shared domain-row typedefs** (no `TransactionRow`/`PlannedRow` contracts β€” `@vision/types` covers only error codes; `packages/shared-utils/src/*.d.ts` covers money/portfolio helpers). The clean `@ts-ignore` count (0 in src/) reflects the loose checker, not clean types. Net: row shapes flow into services as implicit `any`; typing is decorative exactly where SQL-shape drift is likeliest. - fix: add a shared `src/types/rows.js` (or `.d.ts` in `@vision/types`) with `@typedef` contracts for the ~10 core row shapes; annotate repository returns against them; then ratchet `noImplicitAny: true` per-directory (start `repositories/`), keeping `strict:false` elsewhere. diff --git a/alembic/versions/0050_add_accounts_entity.py b/alembic/versions/0050_add_accounts_entity.py index c2a15b87..e3963f5d 100644 --- a/alembic/versions/0050_add_accounts_entity.py +++ b/alembic/versions/0050_add_accounts_entity.py @@ -31,16 +31,30 @@ checks. Distinct from `recipient_bank_accounts` (counterparty IBANs, ADR-015/ADR-087). Blast radius: one new table + two nullable columns + indexes (no rewrite of existing columns), -plus a bounded backfill (one INSERT over distinct strings, two UPDATEs keyed by the new index). +plus a bounded backfill (one INSERT over distinct strings, UPDATEs to link rows). No existing data is destroyed. Downgrade drops the columns/indexes, the table, and the enum types; the backfilled `account_id` values disappear with the columns (the `bank_account` string is untouched, so re-applying re-derives them). +COST CONTROL (large installs): on a big `transactions` table the naive shape of step 3 β€” a +single full-table UPDATE plus two eager index builds, all inside one migration transaction β€” +is the most expensive step of the whole chain (full heap rewrite + full WAL + write-blocking +index builds). To keep it bounded, the `transactions` half of the backfill and the two +`transactions` indexes run inside an `autocommit_block()`: + * the backfill UPDATE is batched over id ranges, each batch committing on its own, so locks + stay short, WAL is spread out, and vacuum can reclaim dead tuples between batches; + * the indexes are built AFTER the backfill (no index maintenance during the rewrite) and + with CREATE INDEX CONCURRENTLY (no write-blocking lock); + * every statement stays idempotent (`WHERE account_id IS NULL`, drop-invalid-then-create), + so a kill mid-migration resumes cleanly on the next boot β€” the same resume contract + `transaction_per_migration` (alembic/env.py) already establishes per-migration. + NOTE: migrations are not auto-run β€” this is AUTHORED NOT YET APPLIED; the user applies it. """ from typing import Sequence, Union +import sqlalchemy as sa from alembic import op @@ -49,6 +63,45 @@ branch_labels: Union[str, Sequence[str], None] = None depends_on: Union[str, Sequence[str], None] = None +# Rows per committed batch of the transactions backfill UPDATE (id-range keyed, so each +# batch is a cheap PK range scan regardless of how sparse the id space is). +BACKFILL_BATCH_SIZE = 50_000 + + +def _create_index_concurrently(name: str, create_sql: str) -> None: + """CREATE INDEX CONCURRENTLY with the failure caveat handled. + + Must be called inside an autocommit_block(): CONCURRENTLY cannot run in a + transaction. A concurrent build that is interrupted leaves an INVALID index + behind, and `CREATE INDEX IF NOT EXISTS` would silently keep it β€” so instead: + a valid existing index is kept (idempotent re-run, no wasteful rebuild), an + invalid leftover is dropped, and only then is the index (re)built. + """ + valid = ( + op.get_bind() + .execute( + sa.text( + """ + SELECT i.indisvalid + FROM pg_index i + JOIN pg_class c ON c.oid = i.indexrelid + JOIN pg_namespace n ON n.oid = c.relnamespace + WHERE c.relname = :name + AND n.nspname = current_schema() + """ + ), + {"name": name}, + ) + .scalar() + ) + if valid: + return + if valid is not None: # exists but INVALID (interrupted previous concurrent build) + op.execute(f"DROP INDEX IF EXISTS {name}") + # One statement per execute: a multi-statement string would run as a single + # implicit transaction and CONCURRENTLY would refuse. + op.execute(create_sql) + def upgrade() -> None: # ── Enum types for the categorical flags (idempotent, matching the 0001 idiom) ── @@ -109,7 +162,10 @@ def upgrade() -> None: """ ) - # ── account_id FKs (ON DELETE RESTRICT) + indexes ── + # ── account_id FKs (ON DELETE RESTRICT) + planned_transactions index ── + # The two transactions indexes are NOT built here: they are built CONCURRENTLY at the + # end of upgrade(), after the backfill, inside the autocommit block (see docstring). + # planned_transactions is small, so its index stays in the transactional section. op.execute( """ ALTER TABLE transactions @@ -119,12 +175,6 @@ def upgrade() -> None: ADD COLUMN IF NOT EXISTS account_id INTEGER REFERENCES accounts(id) ON DELETE RESTRICT; - CREATE INDEX IF NOT EXISTS idx_transactions_account_id - ON transactions (account_id); - -- Mirrors idx_transactions_bank_date_active for the running-balance window - -- (PARTITION BY account_id ORDER BY date) and the balance-history LATERAL probe. - CREATE INDEX IF NOT EXISTS idx_transactions_account_date_active - ON transactions (account_id, date DESC) WHERE is_active = true; CREATE INDEX IF NOT EXISTS idx_planned_transactions_account_id ON planned_transactions (account_id); """ @@ -189,16 +239,10 @@ def upgrade() -> None: """ ) - # Link existing rows to their account by exact trimmed-name match. + # Link existing planned rows to their account by exact trimmed-name match + # (small table β€” single statement, stays in the migration transaction). op.execute( """ - UPDATE transactions t - SET account_id = a.id - FROM accounts a - WHERE t.account_id IS NULL - AND t.bank_account IS NOT NULL - AND a.name = btrim(t.bank_account); - UPDATE planned_transactions p SET account_id = a.id FROM accounts a @@ -208,6 +252,57 @@ def upgrade() -> None: """ ) + # ── transactions backfill + indexes, outside the migration transaction ── + # autocommit_block() commits everything above (safe: every statement above is + # idempotent), runs the block in autocommit, then resumes a transaction for the + # alembic_version stamp. Semantics of the UPDATE are identical to the original + # single statement (same WHERE, same join) β€” it is merely partitioned by id range, + # and each batch commits on its own. The `account_id IS NULL` guard makes an + # interrupted backfill resume where it stopped on the next boot. + with op.get_context().autocommit_block(): + bind = op.get_bind() + bounds = bind.execute( + sa.text("SELECT min(id) AS lo, max(id) AS hi FROM transactions") + ).one() + if bounds.lo is not None: + batch_lo = bounds.lo + while batch_lo <= bounds.hi: + batch_hi = batch_lo + BACKFILL_BATCH_SIZE - 1 + bind.execute( + sa.text( + """ + UPDATE transactions t + SET account_id = a.id + FROM accounts a + WHERE t.id BETWEEN :lo AND :hi + AND t.account_id IS NULL + AND t.bank_account IS NOT NULL + AND a.name = btrim(t.bank_account) + """ + ), + {"lo": batch_lo, "hi": batch_hi}, + ) + batch_lo = batch_hi + 1 + + # Built AFTER the backfill so the heap rewrite above does not have to + # maintain them row-by-row, and CONCURRENTLY so writers are never blocked. + _create_index_concurrently( + "idx_transactions_account_id", + """ + CREATE INDEX CONCURRENTLY idx_transactions_account_id + ON transactions (account_id) + """, + ) + # Mirrors idx_transactions_bank_date_active for the running-balance window + # (PARTITION BY account_id ORDER BY date) and the balance-history LATERAL probe. + _create_index_concurrently( + "idx_transactions_account_date_active", + """ + CREATE INDEX CONCURRENTLY idx_transactions_account_date_active + ON transactions (account_id, date DESC) WHERE is_active = true + """, + ) + def downgrade() -> None: op.execute( diff --git a/alembic/versions/0076_account_trigger_blank_and_case.py b/alembic/versions/0076_account_trigger_blank_and_case.py index 102fec65..2873f726 100644 --- a/alembic/versions/0076_account_trigger_blank_and_case.py +++ b/alembic/versions/0076_account_trigger_blank_and_case.py @@ -30,6 +30,18 @@ or written by the migration itself. Rollback: downgrade() restores the 0062 function verbatim. + +RETROACTIVE FIX (2026-07-28): as originally shipped, the onboarding INSERT's +arbiter was `ON CONFLICT (name)` β€” but 0066 had already dropped +uq_accounts_name for the expression index uq_accounts_name_norm on +`lower(btrim(name))`, so the arbiter matched NO unique index and every +first-seen-label INSERT raised 42P10. This file is edited in place to use the +correct `(lower(btrim(name)))` arbiter so fresh installs (and installs below +0076) never pass through the broken state; installs that already ran the +broken version are repaired by migration 0083, which re-issues the CREATE OR +REPLACE with the same corrected body. The duplication is deliberate: the +function lives in the database, so editing this file alone cannot reach +already-migrated installs. """ from typing import Sequence, Union @@ -72,9 +84,18 @@ def upgrade() -> None: WHERE lower(btrim(name)) = lower(acct_name) ORDER BY id LIMIT 1; IF resolved_id IS NULL THEN + -- Arbiter = the 0066 unique expression index + -- uq_accounts_name_norm on lower(btrim(name)). As shipped, + -- this said `ON CONFLICT (name)` β€” matching no unique + -- index, since 0066 dropped uq_accounts_name β€” so every + -- first-seen-label INSERT raised 42P10. Fixed here IN + -- PLACE so fresh installs never pass through the broken + -- state; installs that already ran the broken version get + -- the same fix via 0083's CREATE OR REPLACE (the function + -- lives in the DB β€” editing this file cannot reach them). INSERT INTO accounts (name, display_name) VALUES (acct_name, acct_name) - ON CONFLICT (name) DO NOTHING; + ON CONFLICT (lower(btrim(name))) DO NOTHING; SELECT id INTO resolved_id FROM accounts WHERE lower(btrim(name)) = lower(acct_name) ORDER BY id LIMIT 1; diff --git a/alembic/versions/0083_fix_account_onboarding_conflict_arbiter.py b/alembic/versions/0083_fix_account_onboarding_conflict_arbiter.py new file mode 100644 index 00000000..c0496a1c --- /dev/null +++ b/alembic/versions/0083_fix_account_onboarding_conflict_arbiter.py @@ -0,0 +1,170 @@ +"""Repair the account-sync trigger's ON CONFLICT arbiter regressed by 0076. + +Revision ID: 0083_fix_account_onboarding_conflict_arbiter +Revises: 0082_drop_mv_bank_balances +Create Date: 2026-07-28 + +Migration 0066 (ADR-088 addendum, D1) replaced `uq_accounts_name` with the +unique expression index `uq_accounts_name_norm` on `lower(btrim(name))` and +retargeted `sync_account_id_from_bank_account()`'s onboarding INSERT to +`ON CONFLICT (lower(btrim(name)))`. The trigger rewrite in 0076 (blank-label +detach + case-insensitive resolve) regressed that arbiter back to +`ON CONFLICT (name)` β€” which matches NO unique index at head, so the first +INSERT of a transaction whose `bank_account` label has no existing account +raised 42P10 ("there is no unique or exclusion constraint matching the +ON CONFLICT specification"). Existing labels (any casing) still resolved β€” +the resolve path short-circuits before the INSERT β€” but first-seen-label +onboarding (e.g. the first CSV import for a brand-new account) blew up. +Pinned (now flipped) by tests/transactionRepository.db.test.js. + +This migration re-creates the function with 0076's exact body β€” both of its +fixes (blank-on-UPDATE detaches; case-insensitive resolve-before-create) are +preserved β€” changing ONLY the arbiter to `(lower(btrim(name)))`. + +Why this duplicates an in-place edit of 0076: the function lives in the +database, so installs already at/past 0076 only get the fix via this +CREATE OR REPLACE β€” editing 0076 retroactively does nothing for them. 0076 +is ALSO edited in place (same arbiter) so fresh installs and installs below +0076 never pass through the broken state; for those, this migration is an +idempotent re-create of the same function. + +Blast radius: replaces one trigger FUNCTION in place (CREATE OR REPLACE); the +triggers on transactions / planned_transactions stay attached. No data is read +or written by the migration itself. + +Rollback: downgrade() restores the 0076 function verbatim as it originally +shipped (i.e. with the broken `ON CONFLICT (name)` arbiter β€” that IS the 0076 +state; downgrading past this revision reintroduces the onboarding failure). + +NOTE: migrations are not auto-run by the agent β€” authored here; applied on the +next app boot. +""" + +from typing import Sequence, Union + +from alembic import op + + +revision: str = "0083_fix_account_onboarding_conflict_arbiter" +down_revision: Union[str, Sequence[str], None] = "0082_drop_mv_bank_balances" +branch_labels: Union[str, Sequence[str], None] = None +depends_on: Union[str, Sequence[str], None] = None + + +def upgrade() -> None: + # 0076's body verbatim, except the ON CONFLICT arbiter now targets the + # unique expression index uq_accounts_name_norm (0066) instead of the + # dropped uq_accounts_name constraint. + op.execute( + """ + CREATE OR REPLACE FUNCTION sync_account_id_from_bank_account() + RETURNS trigger AS $$ + DECLARE acct_name text; + resolved_id integer; + BEGIN + acct_name := btrim(NEW.bank_account); + IF acct_name IS NULL OR acct_name = '' THEN + -- Blanking the label on UPDATE detaches the account: keeping + -- the stale account_id made the row keep counting toward an + -- account whose label was removed. Leave INSERTs and + -- already-blank UPDATEs alone (account-first writers set + -- account_id directly with no bank_account string). + IF TG_OP = 'UPDATE' AND NEW.bank_account IS DISTINCT FROM OLD.bank_account THEN + NEW.account_id := NULL; + END IF; + RETURN NEW; + END IF; + + IF TG_OP = 'INSERT' THEN + -- Case-insensitive resolve first: "Kbc" must reuse "KBC", + -- not create a duplicate account. Only a label with no + -- casing variant onboards a brand-new account. + SELECT id INTO resolved_id FROM accounts + WHERE lower(btrim(name)) = lower(acct_name) + ORDER BY id LIMIT 1; + IF resolved_id IS NULL THEN + -- Arbiter = the 0066 unique expression index + -- uq_accounts_name_norm. 0076 shipped `ON CONFLICT (name)` + -- here, which matches no unique index since 0066 dropped + -- uq_accounts_name β†’ 42P10 on every first-seen label. + INSERT INTO accounts (name, display_name) + VALUES (acct_name, acct_name) + ON CONFLICT (lower(btrim(name))) DO NOTHING; + SELECT id INTO resolved_id FROM accounts + WHERE lower(btrim(name)) = lower(acct_name) + ORDER BY id LIMIT 1; + END IF; + NEW.account_id := resolved_id; + ELSIF NEW.bank_account IS DISTINCT FROM OLD.bank_account + OR NEW.account_id IS NULL THEN + -- UPDATE: resolve ONLY against existing accounts (0062 + -- semantics β€” never create); now case-insensitively. + SELECT id INTO resolved_id FROM accounts + WHERE lower(btrim(name)) = lower(acct_name) + ORDER BY id LIMIT 1; + IF resolved_id IS NOT NULL THEN + NEW.account_id := resolved_id; + END IF; + END IF; + RETURN NEW; + END; + $$ LANGUAGE plpgsql; + """ + ) + + +def downgrade() -> None: + # Restore the 0076 function verbatim as originally shipped β€” including its + # broken `ON CONFLICT (name)` arbiter, because that IS the 0076 state. + op.execute( + """ + CREATE OR REPLACE FUNCTION sync_account_id_from_bank_account() + RETURNS trigger AS $$ + DECLARE acct_name text; + resolved_id integer; + BEGIN + acct_name := btrim(NEW.bank_account); + IF acct_name IS NULL OR acct_name = '' THEN + -- Blanking the label on UPDATE detaches the account: keeping + -- the stale account_id made the row keep counting toward an + -- account whose label was removed. Leave INSERTs and + -- already-blank UPDATEs alone (account-first writers set + -- account_id directly with no bank_account string). + IF TG_OP = 'UPDATE' AND NEW.bank_account IS DISTINCT FROM OLD.bank_account THEN + NEW.account_id := NULL; + END IF; + RETURN NEW; + END IF; + + IF TG_OP = 'INSERT' THEN + -- Case-insensitive resolve first: "Kbc" must reuse "KBC", + -- not create a duplicate account. Only a label with no + -- casing variant onboards a brand-new account. + SELECT id INTO resolved_id FROM accounts + WHERE lower(btrim(name)) = lower(acct_name) + ORDER BY id LIMIT 1; + IF resolved_id IS NULL THEN + INSERT INTO accounts (name, display_name) + VALUES (acct_name, acct_name) + ON CONFLICT (name) DO NOTHING; + SELECT id INTO resolved_id FROM accounts + WHERE lower(btrim(name)) = lower(acct_name) + ORDER BY id LIMIT 1; + END IF; + NEW.account_id := resolved_id; + ELSIF NEW.bank_account IS DISTINCT FROM OLD.bank_account + OR NEW.account_id IS NULL THEN + -- UPDATE: resolve ONLY against existing accounts (0062 + -- semantics β€” never create); now case-insensitively. + SELECT id INTO resolved_id FROM accounts + WHERE lower(btrim(name)) = lower(acct_name) + ORDER BY id LIMIT 1; + IF resolved_id IS NOT NULL THEN + NEW.account_id := resolved_id; + END IF; + END IF; + RETURN NEW; + END; + $$ LANGUAGE plpgsql; + """ + ) diff --git a/alembic/versions/0084_mv_category_totals_alias_category.py b/alembic/versions/0084_mv_category_totals_alias_category.py new file mode 100644 index 00000000..587c04b8 --- /dev/null +++ b/alembic/versions/0084_mv_category_totals_alias_category.py @@ -0,0 +1,54 @@ +"""Rebuild mv_category_totals with the 3-level effective-category resolution. + +Revision ID: 0084_mv_category_totals_alias_category +Revises: 0083_fix_account_onboarding_conflict_arbiter +Create Date: 2026-07-28 + +`mv_category_totals` (the getCategoryBreakdown fast path) resolved the +effective category over TWO levels only β€” `COALESCE(t.category_id, +r.default_category_id)` β€” while the transactions surfaces resolve THREE +(`…, pr.default_category_id`): a row recorded under an alias recipient whose +PRIMARY carries the default category was categorised in the transactions list +but counted as UNCATEGORISED in the category breakdown. The live fallback in +`infoRepositoryStatistics.getCategoryBreakdown` (and `getCategoryPivot`) is +fixed to the 3-level pattern in the same release; this migration drops the MV +so the fast path is rebuilt from the corrected definition and the two paths +cannot disagree. + +`CREATE MATERIALIZED VIEW IF NOT EXISTS` never redefines an existing view, so +without this drop already-migrated installs would keep the old 2-level SQL +forever (REFRESH re-runs the *stored* definition). Same drop-and-let-boot- +recreate precedent as migration 0045: materializedViewService. +createMaterializedViews runs right after migrations on the same boot +(main.js), recreates the view with the new definition, and the startup / +post-mutation refresh populates it. Until then `mvAvailable()` sees it +unpopulated and getCategoryBreakdown serves the (corrected) live query β€” no +window of wrong answers. + +Downgrade drops the view again so the older code recreates its prior 2-level +definition on next boot. Derived data only; nothing to preserve. +""" + +from typing import Sequence, Union + +from alembic import op + + +revision: str = "0084_mv_category_totals_alias_category" +down_revision: Union[str, Sequence[str], None] = "0083_fix_account_onboarding_conflict_arbiter" +branch_labels: Union[str, Sequence[str], None] = None +depends_on: Union[str, Sequence[str], None] = None + + +def upgrade() -> None: + # destructive-ok: derived data only, rebuilt from transactions by + # materializedViewService.createMaterializedViews on the same boot that applies this + # migration (precedent: 0045) β€” the recreation with the 3-level effective-category + # definition is the point of the drop. Its unique index drops with it via CASCADE. + op.execute("DROP MATERIALIZED VIEW IF EXISTS mv_category_totals CASCADE;") + + +def downgrade() -> None: + # destructive-ok: same rebuild-at-boot contract in reverse β€” older application code + # recreates the previous 2-level definition via createMaterializedViews on next start. + op.execute("DROP MATERIALIZED VIEW IF EXISTS mv_category_totals CASCADE;") diff --git a/apps/frontend/src/App.tsx b/apps/frontend/src/App.tsx index c79a911c..03f17102 100644 --- a/apps/frontend/src/App.tsx +++ b/apps/frontend/src/App.tsx @@ -13,6 +13,7 @@ import { configureCurrencyFormatDefaults, numberFormatToLocale } from "@/utils/c import { lazy, Suspense, useCallback, useEffect, type ReactNode } from "react"; import { ErrorBoundary } from "@/components/shared/ErrorBoundary"; +import { GlobalMutationErrorToaster } from "@/components/shared/GlobalMutationErrorToaster"; import { ScrollToTop } from "@/components/shared/ScrollToTop"; import { StartupRedirect } from "@/components/shared/StartupRedirect"; import { RequireAdmin } from "@/components/auth/RequireAdmin"; @@ -196,6 +197,7 @@ const App = () => { + diff --git a/apps/frontend/src/components/onboarding/__tests__/OnboardingWizard.test.tsx b/apps/frontend/src/components/onboarding/__tests__/OnboardingWizard.test.tsx index 3dcd985e..302a7dbc 100644 --- a/apps/frontend/src/components/onboarding/__tests__/OnboardingWizard.test.tsx +++ b/apps/frontend/src/components/onboarding/__tests__/OnboardingWizard.test.tsx @@ -16,11 +16,11 @@ function stubAdapters() { server.use( http.get(`${API_BASE}/api/info/supported-adapters`, () => ok({ - adapters: [ + items: [ { key: "kbc", name: "KBC", adapter_class: "KbcAdapter" }, { key: "ing", name: "ING", adapter_class: "IngAdapter" }, ], - total_count: 2, + total: 2, }), ), ); diff --git a/apps/frontend/src/components/planned/PlannedPaymentForm.tsx b/apps/frontend/src/components/planned/PlannedPaymentForm.tsx index dc4f9c1d..6212b1e9 100644 --- a/apps/frontend/src/components/planned/PlannedPaymentForm.tsx +++ b/apps/frontend/src/components/planned/PlannedPaymentForm.tsx @@ -153,12 +153,10 @@ export default function PlannedPaymentForm({ open, onOpenChange, onSubmit, initi // If loan is enabled, clear recurrence inputs before submitting - loans drive their own schedule const payload: Record = { name: name.trim(), - // Loans: POST overwrites this from the generated schedule outright, and - // PATCH keeps a defined client value as-is (never re-negates it), so - // there is no double negation on either path. Note PATCH keeping the - // client value means editing a loan's principal/rate/term can leave a - // stale amount vs. the regenerated schedule β€” a pre-existing backend - // behavior, tracked separately in TODO.md. + // Loans: the server owns the amount β€” POST and PATCH both re-derive it + // as -|regular_payment_amount| whenever the repayment schedule is + // (re)generated, ignoring this client value, so there is no double + // negation and no stale amount on either path. amount: signedAmount, currency, due_date: dueDateStr, diff --git a/apps/frontend/src/components/planned/RecurringDetectionPanel.tsx b/apps/frontend/src/components/planned/RecurringDetectionPanel.tsx index 27891c77..258d3e80 100644 --- a/apps/frontend/src/components/planned/RecurringDetectionPanel.tsx +++ b/apps/frontend/src/components/planned/RecurringDetectionPanel.tsx @@ -147,7 +147,15 @@ export function RecurringDetectionPanel({ onCreatePlanned }: Props) { planned_date: pattern.predictedNext, recipient_id: pattern.recipientId, memo: t('recurring.autoDetectedMemo', { name: pattern.recipientName }), - amount: pattern.latestAmount * -1, + // Detected amounts are .abs()'d server-side; the pattern's + // `direction` carries the dominant sign of the source + // transactions. Planned sign convention: money out negative, + // money in positive β€” hardcoding the expense sign here turned a + // detected salary into a negative planned payment that + // plannedMatchService could never auto-match (sign mismatch). + amount: pattern.direction === "income" + ? Math.abs(pattern.latestAmount) + : -Math.abs(pattern.latestAmount), currency: pattern.currency, category_id: pattern.categoryId ?? undefined, bank_account: pattern.bankAccount ?? undefined, diff --git a/apps/frontend/src/components/planned/__tests__/RecurringDetectionPanel.test.tsx b/apps/frontend/src/components/planned/__tests__/RecurringDetectionPanel.test.tsx new file mode 100644 index 00000000..04fb39cd --- /dev/null +++ b/apps/frontend/src/components/planned/__tests__/RecurringDetectionPanel.test.tsx @@ -0,0 +1,104 @@ +// @vitest-environment jsdom +import { beforeEach, describe, expect, it } from "vitest"; +import { screen, waitFor } from "@testing-library/react"; +import userEvent from "@testing-library/user-event"; +import { http } from "msw"; +import { renderWithApp } from "@/test/renderWithApp"; +import { server } from "@/test/msw/server"; +import { ok } from "@/test/msw/handlers"; +import { RecurringDetectionPanel } from "@/components/planned/RecurringDetectionPanel"; + +const API_BASE = "http://localhost:3002"; + +/** + * Sign convention pin (planned-payments): money OUT is negative, money IN is + * positive. The detection service .abs()'s every amount server-side and carries + * the flow sign only in `direction` β€” the panel must recombine them, or a + * detected income (salary, rent received) becomes a negative planned payment + * that plannedMatchService can never auto-match against the real positive + * transaction. + */ +function patternFixture(direction: "income" | "expense") { + return { + recipientId: direction === "income" ? 7 : 8, + recipientName: direction === "income" ? "Employer NV" : "Landlord SA", + direction, + detectedPattern: "monthly", + intervalDays: 30, + consistency: 95, + occurrences: 6, + averageAmount: 2500, // .abs()'d server-side β€” always positive on the wire + latestAmount: 2500, + currency: "EUR", + categoryId: null, + categoryName: null, + bankAccount: "BE12", + firstSeen: "2025-01-28", + lastSeen: "2025-06-28", + predictedNext: "2025-07-28", + amountChanges: [], + isAlreadyPlanned: false, + confidence: 90, + }; +} + +function servePatternsAndCapturePost(direction: "income" | "expense") { + const captured: { body: Record | null } = { body: null }; + server.use( + http.get(`${API_BASE}/api/info/recurring-patterns`, () => + ok({ patterns: [patternFixture(direction)], total: 1 }), + ), + http.post(`${API_BASE}/api/planned-transactions`, async ({ request }) => { + captured.body = (await request.json()) as Record; + return ok({ id: 99 }); + }), + ); + return captured; +} + +async function clickTrack(user: ReturnType) { + const trackBtn = await screen.findByRole("button", { name: /track/i }); + await user.click(trackBtn); +} + +describe("RecurringDetectionPanel β€” detected sign carried into the planned payment", () => { + beforeEach(() => { + window.localStorage.clear(); + }); + + it("creates a POSITIVE planned amount for a detected income pattern", async () => { + const user = userEvent.setup(); + const captured = servePatternsAndCapturePost("income"); + + renderWithApp(); + + expect(await screen.findByText("Employer NV")).toBeInTheDocument(); + await clickTrack(user); + + await waitFor(() => expect(captured.body).not.toBeNull()); + expect(captured.body).toMatchObject({ + amount: 2500, // income β†’ positive, auto-matchable against the credit + recipient_id: 7, + currency: "EUR", + planned_date: "2025-07-28", + is_recurring: true, + recurrence_pattern: "monthly", + }); + }); + + it("creates a NEGATIVE planned amount for a detected expense pattern", async () => { + const user = userEvent.setup(); + const captured = servePatternsAndCapturePost("expense"); + + renderWithApp(); + + expect(await screen.findByText("Landlord SA")).toBeInTheDocument(); + await clickTrack(user); + + await waitFor(() => expect(captured.body).not.toBeNull()); + expect(captured.body).toMatchObject({ + amount: -2500, // expense β†’ negative, auto-matchable against the debit + recipient_id: 8, + }); + }); +}); diff --git a/apps/frontend/src/components/portfolio/AddInvestmentDialog.tsx b/apps/frontend/src/components/portfolio/AddInvestmentDialog.tsx index 40b915ab..944ae6ee 100644 --- a/apps/frontend/src/components/portfolio/AddInvestmentDialog.tsx +++ b/apps/frontend/src/components/portfolio/AddInvestmentDialog.tsx @@ -80,7 +80,13 @@ export function AddInvestmentDialog({ allowedAssetClasses }: Props) { const handleSubmit = async (e: React.FormEvent) => { e.preventDefault(); - if (!form.assetClass || !form.name.trim()) return; + if (!form.assetClass) return; + // The input's `required` blocks a truly empty field, but a whitespace-only + // name passed it and this guard then silently no-op'd the Create. + if (!form.name.trim()) { + toast.error(t('addInv.nameRequired')); + return; + } // Validate the initial purchase BEFORE creating the investment. The old // flow created the row first, then either silently skipped the buy (empty diff --git a/apps/frontend/src/components/portfolio/AddToWatchlistDialog.tsx b/apps/frontend/src/components/portfolio/AddToWatchlistDialog.tsx index 0aff1685..dcb518a5 100644 --- a/apps/frontend/src/components/portfolio/AddToWatchlistDialog.tsx +++ b/apps/frontend/src/components/portfolio/AddToWatchlistDialog.tsx @@ -98,8 +98,8 @@ export function AddToWatchlistDialog({ open, onOpenChange, prefill }: AddToWatch queryFn: async () => { if (!selectedAsset?.symbol) return null; try { - const data = await getMarketQuotes(selectedAsset.symbol, { detail: "basic" }); - return data.quotes?.[0] ?? null; + const quotes = await getMarketQuotes(selectedAsset.symbol, { detail: "basic" }); + return quotes[0] ?? null; } catch { return null; } diff --git a/apps/frontend/src/components/portfolio/EditInvestmentDialog.tsx b/apps/frontend/src/components/portfolio/EditInvestmentDialog.tsx index 9883db9d..56054afc 100644 --- a/apps/frontend/src/components/portfolio/EditInvestmentDialog.tsx +++ b/apps/frontend/src/components/portfolio/EditInvestmentDialog.tsx @@ -71,7 +71,12 @@ export function EditInvestmentDialog({ investment, trigger }: Props) { const handleSubmit = async (e: React.FormEvent) => { e.preventDefault(); - if (!form.name.trim()) return; + // The input's `required` blocks a truly empty field, but a whitespace-only + // name passed it and this guard then silently no-op'd the Save. + if (!form.name.trim()) { + toast.error(t('invEdit.nameRequired')); + return; + } if (unitBased && !form.symbol.trim()) { toast.error(t('invEdit.symbolRequired')); return; diff --git a/apps/frontend/src/components/portfolio/PortfolioNewsFeed.tsx b/apps/frontend/src/components/portfolio/PortfolioNewsFeed.tsx index 254010e4..03222813 100644 --- a/apps/frontend/src/components/portfolio/PortfolioNewsFeed.tsx +++ b/apps/frontend/src/components/portfolio/PortfolioNewsFeed.tsx @@ -29,7 +29,7 @@ export function PortfolioNewsFeed({ symbols }: PortfolioNewsFeedProps) { enabled: isOnline, }); - const articles = data?.articles ?? []; + const articles = data ?? []; return ( diff --git a/apps/frontend/src/components/portfolio/PortfolioTicker.tsx b/apps/frontend/src/components/portfolio/PortfolioTicker.tsx index 3bc2d3aa..11e3bbc0 100644 --- a/apps/frontend/src/components/portfolio/PortfolioTicker.tsx +++ b/apps/frontend/src/components/portfolio/PortfolioTicker.tsx @@ -123,7 +123,7 @@ export function PortfolioTicker({ items }: PortfolioTickerProps) { const entries = useMemo(() => { const bySymbol = new Map(); - for (const q of data?.quotes ?? []) { + for (const q of data ?? []) { if (q.symbol) bySymbol.set(q.symbol.toUpperCase(), q); } if (bySymbol.size === 0) return []; diff --git a/apps/frontend/src/components/portfolio/WatchlistChartDialog.tsx b/apps/frontend/src/components/portfolio/WatchlistChartDialog.tsx index ed014bd6..26133b43 100644 --- a/apps/frontend/src/components/portfolio/WatchlistChartDialog.tsx +++ b/apps/frontend/src/components/portfolio/WatchlistChartDialog.tsx @@ -74,7 +74,7 @@ export function WatchlistChartDialog({ item, open, onOpenChange }: WatchlistChar queryFn: async () => { if (!item?.symbol) return null; try { - const { quotes } = await apiClient.getMarketQuotes(item.symbol, { detail: "basic" }); + const quotes = await apiClient.getMarketQuotes(item.symbol, { detail: "basic" }); return quotes[0] ?? null; } catch { return null; diff --git a/apps/frontend/src/components/portfolio/__tests__/AddPortfolioTxnDialog.test.tsx b/apps/frontend/src/components/portfolio/__tests__/AddPortfolioTxnDialog.test.tsx index a1e9e3cc..8060f150 100644 --- a/apps/frontend/src/components/portfolio/__tests__/AddPortfolioTxnDialog.test.tsx +++ b/apps/frontend/src/components/portfolio/__tests__/AddPortfolioTxnDialog.test.tsx @@ -329,7 +329,7 @@ describe("AddPortfolioTxnDialog", () => { /** The Radix overlay β€” clicking it is the "stray click next to the dialog". */ const overlay = () => - document.querySelector(".fixed.inset-0.backdrop-blur-md")!; + document.querySelector(".fixed.inset-0.modal-overlay")!; it("keeps typed input when dismissed with Escape", async () => { // Arrange diff --git a/apps/frontend/src/components/portfolio/__tests__/EditInvestmentDialog.test.tsx b/apps/frontend/src/components/portfolio/__tests__/EditInvestmentDialog.test.tsx index 49013507..4633117f 100644 --- a/apps/frontend/src/components/portfolio/__tests__/EditInvestmentDialog.test.tsx +++ b/apps/frontend/src/components/portfolio/__tests__/EditInvestmentDialog.test.tsx @@ -3,6 +3,7 @@ import { describe, expect, it, afterEach, vi } from "vitest"; import { screen, waitFor } from "@testing-library/react"; import userEvent from "@testing-library/user-event"; import { http } from "msw"; +import { toast } from "sonner"; import { renderWithApp } from "@/test/renderWithApp"; import { server } from "@/test/msw/server"; import { err } from "@/test/msw/handlers"; @@ -159,6 +160,27 @@ describe("EditInvestmentDialog", () => { ); }); + it("whitespace-only name shows an error toast instead of silently no-oping the Save", async () => { + const toastError = vi.spyOn(toast, "error"); + const user = userEvent.setup(); + renderWithApp(); + + await user.click(await screen.findByRole("button", { name: /^edit$/i })); + await screen.findByRole("dialog"); + + // `required` on the input blocks a truly EMPTY submit natively, but + // whitespace passed it and the guard used to bare-return silently. + const nameInput = screen.getByDisplayValue("MSCI World ETF"); + await user.clear(nameInput); + await user.type(nameInput, " "); + + await user.click(screen.getByRole("button", { name: /^save$/i })); + + // Dialog stays open and the user gets explicit feedback. + await waitFor(() => expect(toastError).toHaveBeenCalledWith("Name is required")); + expect(screen.getByRole("dialog")).toBeInTheDocument(); + }); + it("Cancel button closes dialog", async () => { // Arrange const user = userEvent.setup(); @@ -214,7 +236,7 @@ describe("EditInvestmentDialog", () => { /** The Radix overlay β€” clicking it is the "stray click next to the dialog". */ const overlay = () => - document.querySelector(".fixed.inset-0.backdrop-blur-md")!; + document.querySelector(".fixed.inset-0.modal-overlay")!; it("keeps unsaved edits when dismissed by an outside click", async () => { // Arrange diff --git a/apps/frontend/src/components/portfolio/__tests__/EditPortfolioTxnDialog.test.tsx b/apps/frontend/src/components/portfolio/__tests__/EditPortfolioTxnDialog.test.tsx index 58379903..a298f01b 100644 --- a/apps/frontend/src/components/portfolio/__tests__/EditPortfolioTxnDialog.test.tsx +++ b/apps/frontend/src/components/portfolio/__tests__/EditPortfolioTxnDialog.test.tsx @@ -360,7 +360,7 @@ describe("EditPortfolioTxnDialog", () => { /** The Radix overlay β€” clicking it is the "stray click next to the dialog". */ const overlay = () => - document.querySelector(".fixed.inset-0.backdrop-blur-md")!; + document.querySelector(".fixed.inset-0.modal-overlay")!; it("keeps unsaved edits when dismissed by an outside click", async () => { // Arrange diff --git a/apps/frontend/src/components/portfolio/__tests__/WatchlistChartDialog.test.tsx b/apps/frontend/src/components/portfolio/__tests__/WatchlistChartDialog.test.tsx index 66e19d08..4ef15c32 100644 --- a/apps/frontend/src/components/portfolio/__tests__/WatchlistChartDialog.test.tsx +++ b/apps/frontend/src/components/portfolio/__tests__/WatchlistChartDialog.test.tsx @@ -29,15 +29,17 @@ const ITEM: WatchlistItem = { const CHART_RESPONSE = { symbol: "AAPL", currency: "USD", - points: [ + items: [ { time: 1700000000000, close: 190 }, { time: 1700086400000, close: 192 }, ], + total: 2, }; /** Quote data payload β€” wrapped in the ADR-026 success envelope by `ok()`. */ const QUOTE_RESPONSE = { - quotes: [{ symbol: "AAPL", price: 195.5, change: 1.2, changePercent: 0.6 }], + items: [{ symbol: "AAPL", price: 195.5, change: 1.2, changePercent: 0.6 }], + total: 1, }; function setupDefaultChartHandlers() { @@ -77,7 +79,7 @@ describe("WatchlistChartDialog", () => { it("fetches chart data on open and displays chart or no-data message", async () => { // Arrange β€” empty chart data server.use( - http.get(`${API_BASE}/api/market/chart`, () => ok({ symbol: "AAPL", currency: "USD", points: [] })), + http.get(`${API_BASE}/api/market/chart`, () => ok({ symbol: "AAPL", currency: "USD", items: [], total: 0 })), http.get(`${API_BASE}/api/market/quote`, () => ok(QUOTE_RESPONSE)), ); const onOpenChange = vi.fn(); @@ -91,8 +93,8 @@ describe("WatchlistChartDialog", () => { it("unwraps the API envelope so chart points and live quote render (regression)", async () => { // Regression guard for the envelope bug: the market endpoints return - // { ok, data: { points | quotes } }. Reading res.json() directly (the old - // raw-fetch path) left points/quote undefined β€” empty chart + no price. + // { ok, data: { items, total } }. Reading res.json() directly (the old + // raw-fetch path) left the rows undefined β€” empty chart + no price. // Arrange β€” realistic enveloped responses with non-empty data. setupDefaultChartHandlers(); renderWithApp( @@ -100,10 +102,10 @@ describe("WatchlistChartDialog", () => { ); await screen.findByRole("dialog"); - // Assert β€” live quote (195.5) is unwrapped from data.quotes[0] and rendered + // Assert β€” live quote (195.5) is unwrapped from data.items[0] and rendered // (locale-tolerant on the decimal separator); not the loading skeleton. expect(await screen.findByText(/195[.,]5/)).toBeInTheDocument(); - // Assert β€” chart points are unwrapped from data.points, so the empty-state + // Assert β€” chart points are unwrapped from data.items, so the empty-state // fallback must not appear. expect(screen.queryByText(/no chart data/i)).not.toBeInTheDocument(); }); @@ -130,7 +132,7 @@ describe("WatchlistChartDialog", () => { http.get(`${API_BASE}/api/market/chart`, ({ request }) => { const url = new URL(request.url); capturedRange = url.searchParams.get("range"); - return ok({ symbol: "AAPL", currency: "USD", points: [] }); + return ok({ symbol: "AAPL", currency: "USD", items: [], total: 0 }); }), http.get(`${API_BASE}/api/market/quote`, () => ok(QUOTE_RESPONSE)), ); @@ -156,7 +158,7 @@ describe("WatchlistChartDialog", () => { // Arrange let patchBody: unknown = null; server.use( - http.get(`${API_BASE}/api/market/chart`, () => ok({ symbol: "AAPL", currency: "USD", points: [] })), + http.get(`${API_BASE}/api/market/chart`, () => ok({ symbol: "AAPL", currency: "USD", items: [], total: 0 })), http.get(`${API_BASE}/api/market/quote`, () => ok(QUOTE_RESPONSE)), http.patch(`${API_BASE}/api/watchlist/1`, async ({ request }) => { patchBody = await request.json(); @@ -197,7 +199,7 @@ describe("WatchlistChartDialog", () => { // with a success toast. A 0 target must now be rejected client-side. let patched = false; server.use( - http.get(`${API_BASE}/api/market/chart`, () => ok({ symbol: "AAPL", currency: "USD", points: [] })), + http.get(`${API_BASE}/api/market/chart`, () => ok({ symbol: "AAPL", currency: "USD", items: [], total: 0 })), http.get(`${API_BASE}/api/market/quote`, () => ok(QUOTE_RESPONSE)), http.patch(`${API_BASE}/api/watchlist/1`, () => { patched = true; @@ -268,7 +270,7 @@ describe("WatchlistChartDialog", () => { it("renders gracefully when chart history API returns 5xx error", async () => { server.use( http.get(`${API_BASE}/api/market/chart`, () => err(500, "history unavailable")), - http.get(`${API_BASE}/api/market/quote`, () => ok({ quotes: [] })), + http.get(`${API_BASE}/api/market/quote`, () => ok({ items: [], total: 0 })), ); const errSpy = vi.spyOn(console, "error").mockImplementation(() => {}); renderWithApp( diff --git a/apps/frontend/src/components/research/ResearchMappingDialog.tsx b/apps/frontend/src/components/research/ResearchMappingDialog.tsx index d42c333a..7b9904b3 100644 --- a/apps/frontend/src/components/research/ResearchMappingDialog.tsx +++ b/apps/frontend/src/components/research/ResearchMappingDialog.tsx @@ -62,7 +62,7 @@ export function ResearchMappingDialog({ enabled: open && !!instrumentKey, staleTime: 60_000, }); - const existing = existingResult?.data.mappings ?? []; + const existing = existingResult?.data.items ?? []; // Resolve proposals (auto-propose per provider). Does not persist. const resolveMutation = useMutation({ diff --git a/apps/frontend/src/components/settings/ResearchKeysSection.tsx b/apps/frontend/src/components/settings/ResearchKeysSection.tsx index 74804467..e1e61d43 100644 --- a/apps/frontend/src/components/settings/ResearchKeysSection.tsx +++ b/apps/frontend/src/components/settings/ResearchKeysSection.tsx @@ -53,7 +53,7 @@ export function ResearchKeysSection() {

{t('settings.research.hint')}

- {(data?.providers ?? []).map((status) => ( + {(data?.items ?? []).map((status) => ( { - const { quotes } = await apiClient.getMarketQuotes(debouncedTicker, { detail: "basic" }); + const quotes = await apiClient.getMarketQuotes(debouncedTicker, { detail: "basic" }); return quotes[0] ?? null; }, enabled: open && debouncedTicker.length >= 1, diff --git a/apps/frontend/src/components/shared/GlobalMutationErrorToaster.tsx b/apps/frontend/src/components/shared/GlobalMutationErrorToaster.tsx new file mode 100644 index 00000000..7e5bde9d --- /dev/null +++ b/apps/frontend/src/components/shared/GlobalMutationErrorToaster.tsx @@ -0,0 +1,78 @@ +/** + * Global backstop toaster for mutation errors nobody else surfaces. + * + * Every mutation hook that cares about errors already toasts localized copy via + * `apiErrorToMessage` in its own `onError`. But a mutation defined WITHOUT an + * `onError` used to fail silently β€” the user clicked, nothing happened, and the + * only trace was a rejected promise in devtools (e.g. the attachment upload in + * `AttachmentPanel`). + * + * This component subscribes to the app QueryClient's `MutationCache` and toasts + * the mapped, localized message for exactly those unhandled failures. Mounted + * UNDER `LanguageProvider` (same pattern as `AppSettingsSaveErrorToaster`) + * because the module-scope QueryClient in App.tsx cannot reach `t`. + * + * Double-toast prevention β€” the backstop stays quiet when: + * β€’ `mutation.options.onError` is set: the hook handles (and usually toasts) + * the error itself. Call-site-only handlers (`mutate(vars, { onError })`) + * are invisible here, but every such call site in this repo pairs with a + * hook-level `onError`, so they are covered by this check too. + * β€’ `mutation.meta.suppressErrorToast` is set: the call site surfaces errors + * through another channel (`mutateAsync` + try/catch toasts, or an inline + * error rendering) that the cache cannot see. + * + * The raw error is logged for developers; the toast shows mapped copy only. + */ + +import { useEffect, useRef } from 'react'; +import { useQueryClient } from '@tanstack/react-query'; +import { toast } from 'sonner'; + +import { useLanguage } from '@/contexts/LanguageContext'; +import { apiErrorToMessage } from '@/lib/api/errorMessage'; +import logger from '@/lib/logger'; + +// App-wide typing for mutation `meta` (TanStack Query's Register pattern). +declare module '@tanstack/react-query' { + interface Register { + mutationMeta: { + /** + * Set on mutations whose errors are surfaced outside `onError` + * (`mutateAsync` + catch, inline error rendering) so the global + * backstop does not toast the same failure twice. + */ + suppressErrorToast?: boolean; + }; + } +} + +export function GlobalMutationErrorToaster() { + const queryClient = useQueryClient(); + const { t } = useLanguage(); + + // Latest-t ref so the cache subscription (established once) always + // translates with the active locale instead of the mount-time one. + const tRef = useRef(t); + useEffect(() => { + tRef.current = t; + }, [t]); + + useEffect(() => { + return queryClient.getMutationCache().subscribe((event) => { + if (event.type !== 'updated' || event.action.type !== 'error') return; + const { mutation } = event; + if (mutation.options.onError || mutation.meta?.suppressErrorToast) return; + + const error: unknown = event.action.error; + // Keep the raw message for devtools/logs β€” the toast never shows it. + logger.error('Unhandled mutation error:', error); + + const translate = tRef.current; + toast.error(translate('common.error'), { + description: apiErrorToMessage(error, translate), + }); + }); + }, [queryClient]); + + return null; +} diff --git a/apps/frontend/src/components/shared/VirtualDataTable.tsx b/apps/frontend/src/components/shared/VirtualDataTable.tsx index 364311f2..31014420 100644 --- a/apps/frontend/src/components/shared/VirtualDataTable.tsx +++ b/apps/frontend/src/components/shared/VirtualDataTable.tsx @@ -23,6 +23,22 @@ export type { Column }; export type SortDirection = "asc" | "desc" | null; +/** + * Minimum trimmed length before a server-mode search is forwarded to + * `search.onChange`. Below this the table forwards "" so the list shows + * UNFILTERED results (never stale ones) while the input keeps the user's + * text. Companion of the server-side `MIN_SEARCH_LENGTH` + * (apps/node-backend/src/lib/filterBuilder.js): the backend ignores + * sub-length terms anyway, so forwarding them only refetches for nothing. + */ +export const SERVER_SEARCH_MIN_LENGTH = 3; + +/** The query actually forwarded to the server for a given raw input value. */ +function gateServerSearch(value: string): string { + const trimmed = value.trim(); + return trimmed.length >= SERVER_SEARCH_MIN_LENGTH ? trimmed : ""; +} + /** * Server-driven data operations, grouped by concern. Presence of a group turns * that concern over to the server; omit the whole object for a fully local table. @@ -207,7 +223,7 @@ export function VirtualDataTable>({ isTypingRef.current = true; if (debounceRef.current) clearTimeout(debounceRef.current); debounceRef.current = setTimeout(() => { - onSearchChange!(value); + onSearchChange!(gateServerSearch(value)); debounceRef.current = null; isTypingRef.current = false; }, SEARCH_DEBOUNCE_MS); @@ -224,7 +240,15 @@ export function VirtualDataTable>({ useEffect(() => { if (!isServerSearch) return; const externalQuery = searchValue ?? ""; - if (!isTypingRef.current && externalQuery !== localSearchQuery) { + // `gateServerSearch(localSearchQuery)` is what this table last forwarded + // for the current input. When the external value matches it, this is our + // own (gated/trimmed) echo coming back β€” not an outside change β€” and + // syncing would wipe the sub-threshold text the user is still typing. + if ( + !isTypingRef.current && + externalQuery !== localSearchQuery && + externalQuery !== gateServerSearch(localSearchQuery) + ) { setLocalSearchQuery(externalQuery); } }, [isServerSearch, searchValue, localSearchQuery]); @@ -527,7 +551,20 @@ export function VirtualDataTable>({ const saveEditing = (sourceIndex: number, row: T) => { if (onRowUpdate) { - const updatedRow = { ...row, ...editValues } as T; + // Number columns keep the raw string while editing (so a decimal + // separator can be typed) and are parsed here. A cleared number + // field keeps the original value instead of saving a legit-looking + // 0.00 (`parseDecimal("") β†’ 0`). + const values: Record = { ...editValues }; + for (const col of columns) { + if (!col.editable || col.type !== "number" || !(col.key in values)) continue; + const raw = values[col.key]; + if (typeof raw === "string") { + if (raw.trim() === "") delete values[col.key]; + else values[col.key] = parseDecimal(raw); + } + } + const updatedRow = { ...row, ...values } as T; onRowUpdate(sourceIndex, updatedRow); } setEditingRow(null); @@ -847,11 +884,12 @@ export function VirtualDataTable>({ type={col.type || "text"} value={String(editValues[col.key] ?? "")} onChange={(e) => + // Raw string for every type; number columns are parsed + // at save time (saveEditing) so clearing the field does + // not silently become a saved 0.00. setEditValues((prev) => ({ ...prev, - [col.key]: col.type === "number" - ? parseDecimal(e.target.value) - : e.target.value, + [col.key]: e.target.value, })) } onKeyDown={(e) => { diff --git a/apps/frontend/src/components/shared/__tests__/GlobalMutationErrorToaster.test.tsx b/apps/frontend/src/components/shared/__tests__/GlobalMutationErrorToaster.test.tsx new file mode 100644 index 00000000..f3fe3570 --- /dev/null +++ b/apps/frontend/src/components/shared/__tests__/GlobalMutationErrorToaster.test.tsx @@ -0,0 +1,162 @@ +// @vitest-environment jsdom +/** + * Global mutation-error backstop (GlobalMutationErrorToaster). + * + * Verifies the three contracts: + * 1. a mutation WITHOUT `onError` gets a localized toast (mapped copy, never + * the raw transport/backend message for machine-generated shapes), + * 2. a mutation WITH a hook-level `onError` is left alone (no double toast), + * 3. `meta.suppressErrorToast` silences the backstop for call sites that + * surface errors themselves (mutateAsync + catch, inline rendering). + */ +import { describe, it, expect, vi, afterEach } from "vitest"; +import { renderHook, act, waitFor } from "@testing-library/react"; +import { createElement, type ReactNode } from "react"; +import { useMutation } from "@tanstack/react-query"; + +import { createLanguageQueryWrapper } from "@/test/queryWrapper"; +import { ApiClientError } from "@/lib/api/client"; +import { ApiErrorCode } from "@vision/types"; +import en from "@/locales/en"; +import { GlobalMutationErrorToaster } from "@/components/shared/GlobalMutationErrorToaster"; + +vi.mock("sonner", () => ({ + toast: { success: vi.fn(), error: vi.fn(), warning: vi.fn(), info: vi.fn() }, +})); +import { toast } from "sonner"; + +/** LanguageQueryWrapper with the backstop toaster mounted alongside the hook under test. */ +function makeWrapper() { + const Base = createLanguageQueryWrapper(); + return function BackstopWrapper({ children }: { children: ReactNode }) { + return createElement(Base, null, createElement(GlobalMutationErrorToaster), children); + }; +} + +afterEach(() => { + vi.restoreAllMocks(); + vi.mocked(toast.error).mockClear(); +}); + +/** Title + description of the most recent `toast.error(...)` call. */ +function lastErrorToast(): { title: unknown; description: unknown } { + const call = vi.mocked(toast.error).mock.calls.at(-1); + return { title: call?.[0], description: (call?.[1] as { description?: unknown } | undefined)?.description }; +} + +describe("GlobalMutationErrorToaster", () => { + it("toasts mapped localized copy when an onError-less mutation fails with a 5xx", async () => { + const { result } = renderHook( + () => + useMutation({ + mutationFn: () => + Promise.reject( + new ApiClientError({ + status: 500, + code: ApiErrorCode.INTERNAL_SERVER_ERROR, + message: "Traceback (most recent call last): boom", + }), + ), + }), + { wrapper: makeWrapper() }, + ); + + await act(async () => { + result.current.mutate(undefined); + }); + + await waitFor(() => expect(toast.error).toHaveBeenCalledTimes(1)); + const { title, description } = lastErrorToast(); + expect(title).toBe(en["common.error"]); + expect(description).toBe(en["apiError.server"]); + expect(description).not.toContain("Traceback"); + }); + + it("shows humanized network copy instead of the browser's raw fetch error", async () => { + const { result } = renderHook( + () => + useMutation({ + mutationFn: () => Promise.reject(new TypeError("Failed to fetch")), + }), + { wrapper: makeWrapper() }, + ); + + await act(async () => { + result.current.mutate(undefined); + }); + + await waitFor(() => expect(toast.error).toHaveBeenCalledTimes(1)); + const { description } = lastErrorToast(); + expect(description).toBe(en["apiError.network"]); + expect(description).not.toBe("Failed to fetch"); + }); + + it("passes an authored backend 4xx detail through as the description", async () => { + const authored = "statement_balance_date is required when a statement balance is set"; + const { result } = renderHook( + () => + useMutation({ + mutationFn: () => + Promise.reject( + new ApiClientError({ + status: 400, + code: ApiErrorCode.VALIDATION_ERROR, + message: authored, + }), + ), + }), + { wrapper: makeWrapper() }, + ); + + await act(async () => { + result.current.mutate(undefined); + }); + + await waitFor(() => expect(toast.error).toHaveBeenCalledTimes(1)); + expect(lastErrorToast().description).toBe(authored); + }); + + it("stays quiet when the mutation has a hook-level onError (no double toast)", async () => { + const onError = vi.fn(); + const { result } = renderHook( + () => + useMutation({ + mutationFn: () => Promise.reject(new Error("boom")), + onError, + }), + { wrapper: makeWrapper() }, + ); + + await act(async () => { + result.current.mutate(undefined); + }); + + await waitFor(() => expect(result.current.isError).toBe(true)); + expect(onError).toHaveBeenCalledTimes(1); + expect(toast.error).not.toHaveBeenCalled(); + }); + + it("stays quiet for meta.suppressErrorToast mutations handled via mutateAsync + catch", async () => { + const { result } = renderHook( + () => + useMutation({ + mutationFn: () => Promise.reject(new Error("boom")), + meta: { suppressErrorToast: true }, + }), + { wrapper: makeWrapper() }, + ); + + let caught: unknown; + await act(async () => { + try { + await result.current.mutateAsync(undefined); + } catch (err) { + caught = err; + } + }); + + await waitFor(() => expect(result.current.isError).toBe(true)); + expect(caught).toEqual(new Error("boom")); + expect(toast.error).not.toHaveBeenCalled(); + }); +}); diff --git a/apps/frontend/src/components/shared/__tests__/VirtualDataTable.test.tsx b/apps/frontend/src/components/shared/__tests__/VirtualDataTable.test.tsx index adca4036..3748fdb1 100644 --- a/apps/frontend/src/components/shared/__tests__/VirtualDataTable.test.tsx +++ b/apps/frontend/src/components/shared/__tests__/VirtualDataTable.test.tsx @@ -3,7 +3,8 @@ import { describe, it, expect, vi, afterEach } from "vitest"; import { screen, fireEvent, waitFor } from "@testing-library/react"; import userEvent from "@testing-library/user-event"; import { renderWithApp } from "@/test/renderWithApp"; -import { VirtualDataTable } from "@/components/shared/VirtualDataTable"; +import { useState } from "react"; +import { VirtualDataTable, SERVER_SEARCH_MIN_LENGTH } from "@/components/shared/VirtualDataTable"; import { ContextMenuContent, ContextMenuItem } from "@/components/ui/context-menu"; import type { Column } from "@/types/dataTable"; import { SEARCH_DEBOUNCE_MS } from "@/hooks/useDebounce"; @@ -225,6 +226,86 @@ describe("VirtualDataTable β€” server-side search", () => { await vi.advanceTimersByTimeAsync(SEARCH_DEBOUNCE_MS - 1); expect(onSearchChange).not.toHaveBeenCalled(); }); + + it(`forwards "" (unfiltered) for input shorter than ${SERVER_SEARCH_MIN_LENGTH} characters and never issues a filtered search`, async () => { + vi.useFakeTimers(); + const onSearchChange = vi.fn(); + renderTable({ serverMode: { search: { onChange: onSearchChange } } }); + + const input = screen.getByPlaceholderText("Search database..."); + fireEvent.change(input, { target: { value: "ab" } }); + await vi.advanceTimersByTimeAsync(SEARCH_DEBOUNCE_MS); + + expect(onSearchChange).toHaveBeenCalledWith(""); + expect(onSearchChange).not.toHaveBeenCalledWith("ab"); + // The input keeps the user's typed text β€” only the forwarded search resets. + expect(input).toHaveValue("ab"); + }); + + it("whitespace-padded input below the threshold counts as too short", async () => { + vi.useFakeTimers(); + const onSearchChange = vi.fn(); + renderTable({ serverMode: { search: { onChange: onSearchChange } } }); + + fireEvent.change( + screen.getByPlaceholderText("Search database..."), + { target: { value: " ab " } }, + ); + await vi.advanceTimersByTimeAsync(SEARCH_DEBOUNCE_MS); + + expect(onSearchChange).toHaveBeenCalledWith(""); + expect(onSearchChange).not.toHaveBeenCalledWith("ab"); + }); + + it(`forwards the trimmed term once the input reaches ${SERVER_SEARCH_MIN_LENGTH} characters`, async () => { + vi.useFakeTimers(); + const onSearchChange = vi.fn(); + renderTable({ serverMode: { search: { onChange: onSearchChange } } }); + + fireEvent.change( + screen.getByPlaceholderText("Search database..."), + { target: { value: " abc " } }, + ); + await vi.advanceTimersByTimeAsync(SEARCH_DEBOUNCE_MS); + + expect(onSearchChange).toHaveBeenCalledWith("abc"); + }); + + it("resets the forwarded search to \"\" when a filtered query is shortened below the threshold, keeping the typed text", async () => { + vi.useFakeTimers(); + // Controlled harness mirroring the real call sites (TransactionsPage / + // RecipientsPage): the forwarded value loops back in as search.value. + const onSearchChange = vi.fn(); + function Harness() { + const [search, setSearch] = useState(""); + return ( + + title="My Table" + columns={COLUMNS} + data={DATA} + serverMode={{ + search: { + onChange: (q) => { onSearchChange(q); setSearch(q); }, + value: search, + }, + }} + /> + ); + } + renderWithApp(); + + const input = screen.getByPlaceholderText("Search database..."); + fireEvent.change(input, { target: { value: "abcd" } }); + await vi.advanceTimersByTimeAsync(SEARCH_DEBOUNCE_MS); + expect(onSearchChange).toHaveBeenLastCalledWith("abcd"); + + fireEvent.change(input, { target: { value: "ab" } }); + await vi.advanceTimersByTimeAsync(SEARCH_DEBOUNCE_MS); + // Below the threshold: unfiltered (never stale "abcd" results) … + expect(onSearchChange).toHaveBeenLastCalledWith(""); + // … while the echoed-back "" must not wipe what the user typed. + expect(input).toHaveValue("ab"); + }); }); // --------------------------------------------------------------------------- @@ -318,6 +399,44 @@ describe("VirtualDataTable β€” inline editing", () => { ), ); }); + + it("keeps the original number when the field is cleared (no silent 0.00 save)", async () => { + const user = userEvent.setup(); + const onRowUpdate = vi.fn(); + renderTable({ columns: EDITABLE_COLUMNS, onRowUpdate }); + + await user.dblClick(screen.getByText("Alpha")); + const valueInput = screen.getByRole("spinbutton"); + await user.clear(valueInput); + await user.keyboard("{Enter}"); + + // Row 0's value is 100 β€” a cleared field must not become 0. + await waitFor(() => + expect(onRowUpdate).toHaveBeenCalledWith( + 0, + expect.objectContaining({ value: 100 }), + ), + ); + }); + + it("parses an edited number value at save time", async () => { + const user = userEvent.setup(); + const onRowUpdate = vi.fn(); + renderTable({ columns: EDITABLE_COLUMNS, onRowUpdate }); + + await user.dblClick(screen.getByText("Alpha")); + const valueInput = screen.getByRole("spinbutton"); + await user.clear(valueInput); + await user.type(valueInput, "12.5"); + await user.keyboard("{Enter}"); + + await waitFor(() => + expect(onRowUpdate).toHaveBeenCalledWith( + 0, + expect.objectContaining({ value: 12.5 }), + ), + ); + }); }); // --------------------------------------------------------------------------- diff --git a/apps/frontend/src/components/ui/__tests__/modal-overlay.test.tsx b/apps/frontend/src/components/ui/__tests__/modal-overlay.test.tsx new file mode 100644 index 00000000..aa676a18 --- /dev/null +++ b/apps/frontend/src/components/ui/__tests__/modal-overlay.test.tsx @@ -0,0 +1,135 @@ +// @vitest-environment jsdom +/** + * Modal overlay backdrop β€” tier contract (ADR-075). + * + * The dialog / alert-dialog / sheet overlays used to hard-code + * `backdrop-blur-md`, re-blurring the entire viewport on every vsync while + * any modal was open. They now carry the semantic `modal-overlay` class, + * whose per-tier styling lives in index.css: a flat dim at every tier, with + * the frosted blur restored only under `:root.fx-enhanced` (the class + * VisualEffectsController stamps on when the effective + * visual-effects tier is 'enhanced'). + * + * jsdom neither loads index.css nor evaluates @supports/@media, so the tier + * toggle is pinned in two halves: + * - DOM: each overlay renders the semantic class and no blur utility; + * - CSS: index.css scopes the blur to `:root.fx-enhanced` and keeps the + * base `.modal-overlay` rule blur-free. + */ +import { readFileSync } from "node:fs"; +import { join } from "node:path"; +import { describe, it, expect } from "vitest"; +import { render, cleanup } from "@testing-library/react"; +import { Dialog, DialogContent, DialogTitle, DialogDescription } from "@/components/ui/dialog"; +import { + AlertDialog, + AlertDialogContent, + AlertDialogTitle, + AlertDialogDescription, +} from "@/components/ui/alert-dialog"; +import { Sheet, SheetContent, SheetTitle, SheetDescription } from "@/components/ui/sheet"; + +// vitest runs with apps/frontend as the project root. +const indexCss = readFileSync(join(process.cwd(), "src/index.css"), "utf8"); + +function overlayEl(): HTMLElement { + const el = document.querySelector(".fixed.inset-0.z-50"); + expect(el).not.toBeNull(); + return el!; +} + +/** The body of the first CSS rule for `selector` (naive but sufficient here). */ +function ruleBody(selector: string): string { + const start = indexCss.indexOf(`${selector} {`); + expect(start, `rule "${selector}" exists in index.css`).toBeGreaterThan(-1); + const open = indexCss.indexOf("{", start); + const close = indexCss.indexOf("}", open); + return indexCss.slice(open + 1, close); +} + +describe("modal overlay β€” rendered classes (default tier = dim)", () => { + it.each([ + [ + "dialog", + () => ( + + + t + d + + + ), + false, + ], + [ + "alert-dialog", + () => ( + + + t + d + + + ), + true, + ], + [ + "sheet", + () => ( + + + t + d + + + ), + false, + ], + ])("%s overlay uses the tiered dim class, not an unconditional blur", (_name, ui, strong) => { + render(ui()); + const overlay = overlayEl(); + expect(overlay.className).toContain("modal-overlay"); + expect(overlay.className.includes("modal-overlay-strong")).toBe(strong); + // The unconditional full-viewport blur must be gone at the default tier. + expect(overlay.className).not.toContain("backdrop-blur"); + expect(overlay.className).not.toContain("bg-background/"); + cleanup(); + }); +}); + +describe("modal overlay β€” index.css tier rules", () => { + it("base tier is a flat dim with no backdrop blur", () => { + const base = ruleBody(" .modal-overlay"); + expect(base).toContain("hsl(var(--background) / 0.6)"); + expect(base).not.toContain("backdrop-filter"); + }); + + it("the frosted blur is scoped to the enhanced tier root class", () => { + const enhanced = ruleBody(":root.fx-enhanced .modal-overlay"); + // Pixel-identical to the pre-tier overlay: blur-md (12px) over /40. + expect(enhanced).toContain("backdrop-filter: blur(12px)"); + expect(enhanced).toContain("hsl(var(--background) / 0.4)"); + }); + + it("alert dialogs keep their stronger scrim at the enhanced tier", () => { + const strong = ruleBody(":root.fx-enhanced .modal-overlay-strong"); + expect(strong).toContain("hsl(var(--background) / 0.5)"); + }); + + it("no rule blurs .modal-overlay outside the fx-enhanced scope", () => { + // Every `.modal-overlay` rule that sets a backdrop-filter must sit + // under :root.fx-enhanced β€” the class VisualEffectsController stamps + // on when the user's visual-effects setting is 'enhanced'. + const ruleRe = /^[^\S\n]*([^\n{}]*\.modal-overlay[^\n{}]*)\{([^}]*)\}/gm; + let match: RegExpExecArray | null; + let count = 0; + while ((match = ruleRe.exec(indexCss)) !== null) { + count += 1; + const [, selector, body] = match; + if (body.includes("backdrop-filter")) { + expect(selector).toContain(":root.fx-enhanced"); + } + } + expect(count).toBeGreaterThanOrEqual(2); + }); +}); diff --git a/apps/frontend/src/components/ui/alert-dialog.tsx b/apps/frontend/src/components/ui/alert-dialog.tsx index 5ceb88e3..4a1d0dd8 100644 --- a/apps/frontend/src/components/ui/alert-dialog.tsx +++ b/apps/frontend/src/components/ui/alert-dialog.tsx @@ -17,7 +17,10 @@ const AlertDialogOverlay = React.forwardRef< >(({className, ...props}, ref) => ( (({className, ...props}, ref) => ( { expect(screen.getAllByText("Buy").length).toBeGreaterThan(0); expect(screen.getByText("Sell")).toBeInTheDocument(); }); + + it("warns when the same CSV column is mapped to multiple fields", async () => { + // amount + fees both mapped to "Amount" β†’ every row would get fees = amount. + renderMapper(null, baseConfig({ dateColumn: "Date", amountColumn: "Amount", feesColumn: "Amount" })); + + expect(await screen.findByRole("alert")).toHaveTextContent( + /The same CSV column is mapped to multiple fields: Amount/, + ); + }); + + it("shows no duplicate warning when every mapped column is distinct", async () => { + renderMapper(null, baseConfig({ dateColumn: "Date", amountColumn: "Amount", feesColumn: "Fees" })); + + expect(await screen.findByText(/Required\. Map a date column/)).toBeInTheDocument(); + expect(screen.queryByRole("alert")).not.toBeInTheDocument(); + }); }); diff --git a/apps/frontend/src/features/imports/PortfolioCsvColumnMapper.tsx b/apps/frontend/src/features/imports/PortfolioCsvColumnMapper.tsx index 253fbb53..fcd9b7f0 100644 --- a/apps/frontend/src/features/imports/PortfolioCsvColumnMapper.tsx +++ b/apps/frontend/src/features/imports/PortfolioCsvColumnMapper.tsx @@ -70,6 +70,18 @@ export function PortfolioCsvColumnMapper({ file, separator, config, onChange }: onChange({ ...config, typeMapping: next }); }; + // CSV columns mapped to more than one field. Legit for e.g. one "Instrument" + // column serving both symbol and name, but a plausible-corruption trap for + // amount/fees (every row would get fees = amount) β€” so warn, don't block. + const duplicateColumns = useMemo(() => { + const counts = new Map(); + for (const [key] of PORTFOLIO_COLUMN_FIELDS) { + const v = String(config[key] ?? "").trim(); + if (v) counts.set(v, (counts.get(v) ?? 0) + 1); + } + return [...counts.entries()].filter(([, n]) => n > 1).map(([col]) => col); + }, [config]); + return (
{/* Defaults */} @@ -115,6 +127,12 @@ export function PortfolioCsvColumnMapper({ file, separator, config, onChange }: ))}
+ {duplicateColumns.length > 0 && ( +

+ {t("portfolioImport.duplicateColumnsWarning", { columns: duplicateColumns.join(", ") })} +

+ )} + {/* Type-value mapping */} {config.typeColumn && distinctTypeValues.length > 0 && (
diff --git a/apps/frontend/src/features/imports/useAdapters.ts b/apps/frontend/src/features/imports/useAdapters.ts index 71122dac..a144d5e0 100644 --- a/apps/frontend/src/features/imports/useAdapters.ts +++ b/apps/frontend/src/features/imports/useAdapters.ts @@ -30,5 +30,5 @@ export function useAdapters(enabled = true) { if (isError) toast.error(t('importPage.toast.parsersError')); }, [isError, t]); - return { adapters: data?.adapters ?? [], loading: isLoading }; + return { adapters: data ?? [], loading: isLoading }; } diff --git a/apps/frontend/src/hooks/__tests__/useQueryHooks.test.tsx b/apps/frontend/src/hooks/__tests__/useQueryHooks.test.tsx index 644c76c9..81967f6f 100644 --- a/apps/frontend/src/hooks/__tests__/useQueryHooks.test.tsx +++ b/apps/frontend/src/hooks/__tests__/useQueryHooks.test.tsx @@ -35,16 +35,16 @@ afterEach(() => vi.restoreAllMocks()); describe("useBankAccounts", () => { it("starts in loading state", () => { - vi.spyOn(apiClient, "getDistinctBankAccounts").mockResolvedValue({ banks: [] }); + vi.spyOn(apiClient, "getDistinctBankAccounts").mockResolvedValue([]); const { result } = renderHook(() => useBankAccounts(), { wrapper: makeQueryWrapper() }); expect(result.current.isLoading).toBe(true); }); it("returns bank accounts on success", async () => { - vi.spyOn(apiClient, "getDistinctBankAccounts").mockResolvedValue({ banks: ["BE123", "BE456"] }); + vi.spyOn(apiClient, "getDistinctBankAccounts").mockResolvedValue(["BE123", "BE456"]); const { result } = renderHook(() => useBankAccounts(), { wrapper: makeQueryWrapper() }); await waitFor(() => expect(result.current.isSuccess).toBe(true)); - expect(result.current.data?.banks).toEqual(["BE123", "BE456"]); + expect(result.current.data).toEqual(["BE123", "BE456"]); }); it("exposes error on failure", async () => { diff --git a/apps/frontend/src/hooks/useMarketQuotesQuery.ts b/apps/frontend/src/hooks/useMarketQuotesQuery.ts index dcea39d4..7d059506 100644 --- a/apps/frontend/src/hooks/useMarketQuotesQuery.ts +++ b/apps/frontend/src/hooks/useMarketQuotesQuery.ts @@ -20,7 +20,7 @@ export function useMarketQuotesQuery( queryFn: () => symbols ? apiClient.getMarketQuotes(symbols, { detail: 'basic' }) - : Promise.resolve({ quotes: [] as Q[] }), + : Promise.resolve([] as Q[]), enabled: !!symbols && isOnline, staleTime: options?.staleTime, refetchInterval: isOnline ? 60_000 : false, diff --git a/apps/frontend/src/index.css b/apps/frontend/src/index.css index 777fa474..805e8c23 100644 --- a/apps/frontend/src/index.css +++ b/apps/frontend/src/index.css @@ -470,6 +470,41 @@ } } + /* ------------------------------------------------------------------ + * Modal overlay backdrop (ui/dialog, ui/alert-dialog, ui/sheet). + * + * A frosted full-viewport overlay re-blurs every glass surface plus the + * drifting aurora on every vsync for as long as a modal is open β€” the + * most expensive standing state in the app. The frost is therefore an + * fx-enhanced opt-in (ADR-075); every other tier gets a flat dim the + * compositor can cache. The modal content's own glass is untouched. + * ------------------------------------------------------------------ */ + + .modal-overlay { + background: hsl(var(--background) / 0.6); + } + + @supports ((-webkit-backdrop-filter: blur(1px)) or (backdrop-filter: blur(1px))) { + /* Gated to no-preference so the frost yields to + prefers-reduced-transparency, like the other fx-enhanced opt-ins. */ + @media (prefers-reduced-transparency: no-preference) { + /* blur(12px) = the former Tailwind backdrop-blur-md; alphas match + the former bg-background/40 (dialog, sheet) β€” the enhanced tier + must look exactly like the pre-tier overlay. */ + :root.fx-enhanced .modal-overlay { + background: hsl(var(--background) / 0.4); + -webkit-backdrop-filter: blur(12px); + backdrop-filter: blur(12px); + } + + /* Alert dialogs keep their slightly stronger scrim (was + bg-background/50). */ + :root.fx-enhanced .modal-overlay-strong { + background: hsl(var(--background) / 0.5); + } + } + } + /* ------------------------------------------------------------------ * Shell surfaces * ------------------------------------------------------------------ */ diff --git a/apps/frontend/src/lib/api/__tests__/ai.test.ts b/apps/frontend/src/lib/api/__tests__/ai.test.ts index 5aa5aef6..1b64cc31 100644 --- a/apps/frontend/src/lib/api/__tests__/ai.test.ts +++ b/apps/frontend/src/lib/api/__tests__/ai.test.ts @@ -31,16 +31,16 @@ describe("ai conversation API client", () => { expect((await getOllamaStatus()).ok).toBe(true); }); - it("getOllamaModels returns the models array", async () => { + it("getOllamaModels unwraps the rows from { items, total }", async () => { server.use( - http.get(`${API_BASE}/api/ai/models`, () => ok({ models: [{ name: "llama3" }] })), + http.get(`${API_BASE}/api/ai/models`, () => ok({ items: [{ name: "llama3" }], total: 1 })), ); const models = await getOllamaModels(); expect(models).toHaveLength(1); expect(models[0].name).toBe("llama3"); }); - it("getOllamaModels defaults to [] when models is absent", async () => { + it("getOllamaModels defaults to [] when items is absent", async () => { server.use(http.get(`${API_BASE}/api/ai/models`, () => ok({}))); expect(await getOllamaModels()).toEqual([]); }); diff --git a/apps/frontend/src/lib/api/__tests__/coverage-clients-3.test.ts b/apps/frontend/src/lib/api/__tests__/coverage-clients-3.test.ts index f39c78fe..4c7bed10 100644 --- a/apps/frontend/src/lib/api/__tests__/coverage-clients-3.test.ts +++ b/apps/frontend/src/lib/api/__tests__/coverage-clients-3.test.ts @@ -269,7 +269,7 @@ describe("market API client", () => { server.use( http.get(`${API_BASE}/api/market/news`, ({ request }) => { url = request.url; - return ok({ articles: [] }); + return ok({ items: [], total: 0 }); }), ); await getMarketNews(["AAPL", "MSFT"], 5); @@ -282,7 +282,7 @@ describe("market API client", () => { server.use( http.get(`${API_BASE}/api/market/news`, ({ request }) => { url = request.url; - return ok({ articles: [] }); + return ok({ items: [], total: 0 }); }), ); await getMarketNews(); @@ -295,7 +295,7 @@ describe("market API client", () => { server.use( http.get(`${API_BASE}/api/market/quote`, ({ request }) => { urls.push(request.url); - return ok({ quotes: [] }); + return ok({ items: [], total: 0 }); }), ); await getMarketQuotes("AAPL"); @@ -309,7 +309,7 @@ describe("market API client", () => { server.use( http.get(`${API_BASE}/api/market/chart`, ({ request }) => { url = request.url; - return ok({ points: [] }); + return ok({ items: [], total: 0 }); }), ); await getMarketChart("AAPL", "1mo", "1d"); diff --git a/apps/frontend/src/lib/api/__tests__/coverage-clients.test.ts b/apps/frontend/src/lib/api/__tests__/coverage-clients.test.ts index 8427ea9f..ac15762f 100644 --- a/apps/frontend/src/lib/api/__tests__/coverage-clients.test.ts +++ b/apps/frontend/src/lib/api/__tests__/coverage-clients.test.ts @@ -318,7 +318,7 @@ describe("planned-transactions API client", () => { it("getPlannedMatchSuggestions returns suggestion list", async () => { server.use( http.get(`${API_BASE}/api/planned-transactions/match-suggestions`, () => - ok([{ planned: { id: 1 }, candidates: [] }]), + ok({ items: [{ planned: { id: 1 }, candidates: [] }], total: 1 }), ), ); const res = await getPlannedMatchSuggestions(); diff --git a/apps/frontend/src/lib/api/__tests__/info.test.ts b/apps/frontend/src/lib/api/__tests__/info.test.ts index 3a94299e..3b817108 100644 --- a/apps/frontend/src/lib/api/__tests__/info.test.ts +++ b/apps/frontend/src/lib/api/__tests__/info.test.ts @@ -26,20 +26,20 @@ function ok(data: T, init?: ResponseInit) { afterEach(() => server.resetHandlers()); describe("info API client", () => { - it("getSupportedParsers fetches the adapter list", async () => { + it("getSupportedParsers unwraps the adapter rows from { items, total }", async () => { server.use( http.get(`${API_BASE}/api/info/supported-adapters`, () => - ok({ adapters: [{ key: "kbc", name: "KBC" }], total_count: 1 }), + ok({ items: [{ key: "kbc", name: "KBC" }], total: 1 }), ), ); const res = await getSupportedParsers(); - expect(res.total_count).toBe(1); - expect(res.adapters[0].key).toBe("kbc"); + expect(res).toHaveLength(1); + expect(res[0].key).toBe("kbc"); }); - it("getDistinctBankAccounts fetches the banks endpoint", async () => { - server.use(http.get(`${API_BASE}/api/info/banks`, () => ok({ banks: ["acc1"] }))); - expect((await getDistinctBankAccounts()).banks).toEqual(["acc1"]); + it("getDistinctBankAccounts unwraps the rows from { items, total }", async () => { + server.use(http.get(`${API_BASE}/api/info/banks`, () => ok({ items: ["acc1"], total: 1 }))); + expect(await getDistinctBankAccounts()).toEqual(["acc1"]); }); it("getTransactionCount fetches the count", async () => { diff --git a/apps/frontend/src/lib/api/admin.ts b/apps/frontend/src/lib/api/admin.ts index 6f7c08e6..8c78d75b 100644 --- a/apps/frontend/src/lib/api/admin.ts +++ b/apps/frontend/src/lib/api/admin.ts @@ -89,8 +89,10 @@ export function probeProvider(provider: string): Promise { // ── Request Metrics ─────────────────────────────────────────────────────────── -export function getRequestMetrics(): Promise { - return apiRequest('/api/admin/metrics/requests'); +/** Canonical `{items, total}` collection body β€” callers only need the rows. */ +export async function getRequestMetrics(): Promise { + const { items } = await apiRequest<{ items: RouteMetric[]; total: number }>('/api/admin/metrics/requests'); + return items; } // ── Endpoint Manifest ───────────────────────────────────────────────────────── diff --git a/apps/frontend/src/lib/api/ai.ts b/apps/frontend/src/lib/api/ai.ts index 80023dd5..78c22dec 100644 --- a/apps/frontend/src/lib/api/ai.ts +++ b/apps/frontend/src/lib/api/ai.ts @@ -53,9 +53,10 @@ export function getOllamaStatus(): Promise { return apiRequest('/api/ai/status'); } +/** Canonical `{items, total}` collection body β€” callers only need the rows. */ export async function getOllamaModels(): Promise { const response = await apiRequest('/api/ai/models'); - return response.models ?? []; + return response.items ?? []; } /** Canonical `{items, total}` collection body β€” callers only need the rows. */ diff --git a/apps/frontend/src/lib/api/info.ts b/apps/frontend/src/lib/api/info.ts index 7b8e9d4a..8ca55cc7 100644 --- a/apps/frontend/src/lib/api/info.ts +++ b/apps/frontend/src/lib/api/info.ts @@ -6,15 +6,24 @@ import type { NetWorthResponse } from '@/lib/api/types'; // (Removed getStatistics β€” legacy GET /api/info was deleted in the Phase 9 // cutover; it had no callers. Use the aggregations endpoints instead.) -export function getSupportedParsers(): Promise<{ - adapters: Array<{ key: string; name: string; adapter_class?: string }>; - total_count: number; -}> { - return apiRequest('/api/info/supported-adapters'); +export interface SupportedAdapter { + key: string; + name: string; + adapter_class?: string; +} + +/** Canonical `{items, total}` collection body β€” callers only need the rows. */ +export async function getSupportedParsers(): Promise { + const { items } = await apiRequest<{ items: SupportedAdapter[]; total: number }>( + '/api/info/supported-adapters', + ); + return items; } -export function getDistinctBankAccounts(): Promise<{ banks: string[] }> { - return apiRequest('/api/info/banks'); +/** Canonical `{items, total}` collection body β€” callers only need the rows. */ +export async function getDistinctBankAccounts(): Promise { + const { items } = await apiRequest<{ items: string[]; total: number }>('/api/info/banks'); + return items; } // (Removed getTransactionSummary β€” legacy GET /api/info/transaction-summary was diff --git a/apps/frontend/src/lib/api/market.ts b/apps/frontend/src/lib/api/market.ts index c3c904ae..3547e758 100644 --- a/apps/frontend/src/lib/api/market.ts +++ b/apps/frontend/src/lib/api/market.ts @@ -5,14 +5,19 @@ import type { MarketNewsArticle } from '@/lib/api/types'; export type { MarketNewsArticle }; -export function getMarketNews( +/** Canonical `{items, total}` collection body β€” callers only need the rows. */ +export async function getMarketNews( symbols?: string[], count?: number, -): Promise<{ articles: MarketNewsArticle[] }> { +): Promise { const params: Record = {}; if (symbols?.length) params.symbols = symbols.join(','); if (count) params.count = count; - return requestWithQuery('/api/market/news', params); + const { items } = await requestWithQuery<{ items: MarketNewsArticle[]; total: number }>( + '/api/market/news', + params, + ); + return items; } export interface MarketQuote { @@ -32,12 +37,16 @@ export interface MarketQuote { * benchmark strip and watchlist; omit it (default 'full') when the rich * fundamentals/analyst fields are rendered. */ -export function getMarketQuotes( +export async function getMarketQuotes( symbols: string, opts?: { detail?: 'basic' | 'full' }, -): Promise<{ quotes: Q[] }> { +): Promise { const detail = opts?.detail === 'basic' ? '&detail=basic' : ''; - return apiRequest(`/api/market/quote?symbols=${encodeURIComponent(symbols)}${detail}`); + // Canonical `{items, total}` collection body β€” callers only need the rows. + const { items } = await apiRequest<{ items: Q[]; total: number }>( + `/api/market/quote?symbols=${encodeURIComponent(symbols)}${detail}`, + ); + return items; } export interface MarketChartPoint { @@ -54,12 +63,23 @@ export interface MarketChartResponse

{ points: P[]; } -export function getMarketChart

( +/** + * The wire body is the canonical `{items, total}` collection (with `symbol` and + * `currency` alongside); the series is re-surfaced as `points` here so chart + * consumers keep a domain-named field. + */ +export async function getMarketChart

( symbol: string, range: string, interval: string, ): Promise> { - return requestWithQuery('/api/market/chart', { symbol, range, interval }); + const { items, ...rest } = await requestWithQuery<{ + symbol?: string; + currency?: string; + items: P[]; + total: number; + }>('/api/market/chart', { symbol, range, interval }); + return { symbol: rest.symbol, currency: rest.currency, points: items }; } export interface MarketSearchResult { diff --git a/apps/frontend/src/lib/api/planned.ts b/apps/frontend/src/lib/api/planned.ts index 11dc6163..9cc6e080 100644 --- a/apps/frontend/src/lib/api/planned.ts +++ b/apps/frontend/src/lib/api/planned.ts @@ -82,7 +82,11 @@ export interface PlannedMatchSuggestion { /** * Planned payments with recent unlinked transactions within match tolerance * that were not auto-cleared (ambiguous matches, or auto-clear disabled). + * Canonical `{items, total}` collection body β€” callers only need the rows. */ -export function getPlannedMatchSuggestions(): Promise { - return apiRequest('/api/planned-transactions/match-suggestions'); +export async function getPlannedMatchSuggestions(): Promise { + const { items } = await apiRequest<{ items: PlannedMatchSuggestion[]; total: number }>( + '/api/planned-transactions/match-suggestions', + ); + return items; } diff --git a/apps/frontend/src/locales/en.ts b/apps/frontend/src/locales/en.ts index befb066b..9e1b39da 100644 --- a/apps/frontend/src/locales/en.ts +++ b/apps/frontend/src/locales/en.ts @@ -207,6 +207,7 @@ const en: Record = { 'addInv.label.ticker': 'Ticker / Symbol', 'addInv.label.totalCost': 'Total Cost', 'addInv.label.units': 'Units / Shares', + 'addInv.nameRequired': 'Name is required', 'addInv.placeholder.cadastralIncome': 'e.g. 1250', 'addInv.placeholder.jsonEndpoint': 'https://api.example.com/price', 'addInv.placeholder.location': 'e.g. Brussels, Belgium', @@ -1047,6 +1048,7 @@ const en: Record = { 'invDetail.unitsAt': '{units} units @ {price}', 'invDetail.unitsHeld': 'Units Held', 'invDetail.unrealizedGain': 'Unrealized Gain', + 'invEdit.nameRequired': 'Name is required', 'invEdit.symbolRequired': 'Ticker / symbol is required', 'invEdit.title': 'Edit Investment', 'invEdit.toast.updated': 'Investment "{name}" updated', @@ -1654,6 +1656,7 @@ const en: Record = { 'portfolioImport.defaultAssetClass': 'Default asset class', 'portfolioImport.defaultType': 'Default type', 'portfolioImport.desc': 'Import trades (buys, sells, dividends, fees) from a broker or exchange CSV. Map the columns to your portfolio fields.', + 'portfolioImport.duplicateColumnsWarning': 'The same CSV column is mapped to multiple fields: {columns}. Every mapped field will copy that column\'s value β€” double-check this is intended.', 'portfolioImport.newCustom': 'New custom parser', 'portfolioImport.parserNamePlaceholder': 'e.g. Degiro, Binance', 'portfolioImport.parserSource': 'Parser', diff --git a/apps/frontend/src/locales/nl.ts b/apps/frontend/src/locales/nl.ts index 584a11f2..393404c8 100644 --- a/apps/frontend/src/locales/nl.ts +++ b/apps/frontend/src/locales/nl.ts @@ -207,6 +207,7 @@ const nl: Record = { 'addInv.label.ticker': 'Ticker / Symbool', 'addInv.label.totalCost': 'Totale kostprijs', 'addInv.label.units': 'Eenheden / Aandelen', + 'addInv.nameRequired': 'Naam is verplicht', 'addInv.placeholder.cadastralIncome': 'bijv. 1250', 'addInv.placeholder.jsonEndpoint': 'https://api.example.com/price', 'addInv.placeholder.location': 'bijv. Brussel, BelgiΓ«', @@ -1047,6 +1048,7 @@ const nl: Record = { 'invDetail.unitsAt': '{units} eenheden @ {price}', 'invDetail.unitsHeld': 'Gehouden eenheden', 'invDetail.unrealizedGain': 'Ongerealiseerde winst', + 'invEdit.nameRequired': 'Naam is verplicht', 'invEdit.symbolRequired': 'Ticker / symbool is verplicht', 'invEdit.title': 'Belegging bewerken', 'invEdit.toast.updated': 'Belegging "{name}" bijgewerkt', @@ -1654,6 +1656,7 @@ const nl: Record = { 'portfolioImport.defaultAssetClass': 'Standaard activaklasse', 'portfolioImport.defaultType': 'Standaardtype', 'portfolioImport.desc': 'Importeer transacties (aankopen, verkopen, dividenden, kosten) uit een CSV van een broker of beurs. Wijs de kolommen toe aan je portefeuillevelden.', + 'portfolioImport.duplicateColumnsWarning': 'Dezelfde CSV-kolom is aan meerdere velden gekoppeld: {columns}. Elk gekoppeld veld neemt de waarde van die kolom over β€” controleer of dit de bedoeling is.', 'portfolioImport.newCustom': 'Nieuwe aangepaste parser', 'portfolioImport.parserNamePlaceholder': 'bijv. Degiro, Binance', 'portfolioImport.parserSource': 'Parser', diff --git a/apps/frontend/src/pages/ImportReviewPage.tsx b/apps/frontend/src/pages/ImportReviewPage.tsx index 23fc3b97..d55c45a6 100644 --- a/apps/frontend/src/pages/ImportReviewPage.tsx +++ b/apps/frontend/src/pages/ImportReviewPage.tsx @@ -157,19 +157,25 @@ export default function ImportReviewPage() { const newAccountCount = accountDisclosure.filter((e) => e.isNew).length; + // These three are driven via `mutateAsync` and toast their own localized + // copy in the callers' catch blocks β€” the meta flag keeps the global + // mutation-error backstop from toasting the same failure twice. const overrideMutation = useMutation({ mutationFn: ({ rowId, recipientId }: { rowId: number; recipientId: number | null }) => apiClient.overrideImportRow(batchId, rowId, recipientId), + meta: { suppressErrorToast: true }, }); const categoryOverrideMutation = useMutation({ mutationFn: ({ rowId, categoryId }: { rowId: number; categoryId: number | null }) => apiClient.overrideImportRowCategory(batchId, rowId, categoryId), + meta: { suppressErrorToast: true }, }); const persistDefaultMutation = useMutation({ mutationFn: ({ recipientId, categoryId }: { recipientId: number; categoryId: number | null }) => apiClient.updateRecipient(recipientId, { default_category_id: categoryId }), + meta: { suppressErrorToast: true }, }); const commitMutation = useMutation({ diff --git a/apps/frontend/src/pages/PlannedPaymentsPage.tsx b/apps/frontend/src/pages/PlannedPaymentsPage.tsx index 88412b4c..3355783a 100644 --- a/apps/frontend/src/pages/PlannedPaymentsPage.tsx +++ b/apps/frontend/src/pages/PlannedPaymentsPage.tsx @@ -84,6 +84,10 @@ export default function PlannedPaymentsPage() { const { confirm, ConfirmDialog } = useConfirmDialog(); const [formOpen, setFormOpen] = useState(false); const [editing, setEditing] = useState(); + // Bumped on every "New" open so the create form's key changes and it + // remounts blank β€” a constant "new" key kept all useState initializers + // (name, amount, the direction toggle, …) from the previous create. + const [createFormKey, setCreateFormKey] = useState(0); const [actionLoading, setActionLoading] = useState(false); const [linkDialogOpen, setLinkDialogOpen] = useState(false); const [paymentToLink, setPaymentToLink] = useState(null); @@ -431,7 +435,7 @@ export default function PlannedPaymentsPage() { {showAll ? : } {showAll ? t('plannedPage.showingAll') : t('plannedPage.activeOnly')} - @@ -469,7 +473,7 @@ export default function PlannedPaymentsPage() { onOpenChange={(open) => { setFormOpen(open); if (!open) setEditing(undefined); }} onSubmit={handleSubmit} initial={editing} - key={editing?.id ?? "new"} + key={editing?.id ?? `new-${createFormKey}`} /> { ), http.get(`${API_BASE}/api/market/quote`, () => { quoteFetched = true; - return ok({ quotes: [appleQuote] }); + return ok({ items: [appleQuote], total: 1 }); }), http.get(`${API_BASE}/api/market/chart`, () => - ok({ symbol: "AAPL", currency: "USD", points: [] }), + ok({ symbol: "AAPL", currency: "USD", items: [], total: 0 }), ), - http.get(`${API_BASE}/api/market/news`, () => ok({ articles: [] })), + http.get(`${API_BASE}/api/market/news`, () => ok({ items: [], total: 0 })), ); renderWithApp(); @@ -135,12 +135,12 @@ describe("MarketLookupPage (integration)", () => { }), ), http.get(`${API_BASE}/api/market/quote`, () => - ok({ quotes: [appleQuote] }), + ok({ items: [appleQuote], total: 1 }), ), http.get(`${API_BASE}/api/market/chart`, () => - ok({ symbol: "AAPL", currency: "USD", points: [] }), + ok({ symbol: "AAPL", currency: "USD", items: [], total: 0 }), ), - http.get(`${API_BASE}/api/market/news`, () => ok({ articles: [] })), + http.get(`${API_BASE}/api/market/news`, () => ok({ items: [], total: 0 })), ); renderWithApp(); @@ -156,11 +156,11 @@ describe("MarketLookupPage (integration)", () => { it("shows quote card when symbol is supplied via URL query param", async () => { server.use( - http.get(`${API_BASE}/api/market/quote`, () => ok({ quotes: [appleQuote] })), + http.get(`${API_BASE}/api/market/quote`, () => ok({ items: [appleQuote], total: 1 })), http.get(`${API_BASE}/api/market/chart`, () => - ok({ symbol: "AAPL", currency: "USD", points: [] }), + ok({ symbol: "AAPL", currency: "USD", items: [], total: 0 }), ), - http.get(`${API_BASE}/api/market/news`, () => ok({ articles: [] })), + http.get(`${API_BASE}/api/market/news`, () => ok({ items: [], total: 0 })), ); renderWithApp(, { initialEntries: ["/?symbol=AAPL"] }); @@ -171,11 +171,11 @@ describe("MarketLookupPage (integration)", () => { it("shows Price Chart section when a symbol is loaded", async () => { server.use( - http.get(`${API_BASE}/api/market/quote`, () => ok({ quotes: [appleQuote] })), + http.get(`${API_BASE}/api/market/quote`, () => ok({ items: [appleQuote], total: 1 })), http.get(`${API_BASE}/api/market/chart`, () => - ok({ symbol: "AAPL", currency: "USD", points: [] }), + ok({ symbol: "AAPL", currency: "USD", items: [], total: 0 }), ), - http.get(`${API_BASE}/api/market/news`, () => ok({ articles: [] })), + http.get(`${API_BASE}/api/market/news`, () => ok({ items: [], total: 0 })), ); renderWithApp(, { initialEntries: ["/?symbol=AAPL"] }); @@ -186,11 +186,11 @@ describe("MarketLookupPage (integration)", () => { it("shows Latest News section when a symbol is loaded", async () => { server.use( - http.get(`${API_BASE}/api/market/quote`, () => ok({ quotes: [appleQuote] })), + http.get(`${API_BASE}/api/market/quote`, () => ok({ items: [appleQuote], total: 1 })), http.get(`${API_BASE}/api/market/chart`, () => - ok({ symbol: "AAPL", currency: "USD", points: [] }), + ok({ symbol: "AAPL", currency: "USD", items: [], total: 0 }), ), - http.get(`${API_BASE}/api/market/news`, () => ok({ articles: [] })), + http.get(`${API_BASE}/api/market/news`, () => ok({ items: [], total: 0 })), ); renderWithApp(, { initialEntries: ["/?symbol=AAPL"] }); @@ -202,9 +202,9 @@ describe("MarketLookupPage (integration)", () => { it("shows No news message when the news tab has no articles", async () => { const user = userEvent.setup({ delay: null }); server.use( - http.get(`${API_BASE}/api/market/quote`, () => ok({ quotes: [appleQuote] })), + http.get(`${API_BASE}/api/market/quote`, () => ok({ items: [appleQuote], total: 1 })), http.get(`${API_BASE}/api/market/chart`, () => - ok({ symbol: "AAPL", currency: "USD", points: [] }), + ok({ symbol: "AAPL", currency: "USD", items: [], total: 0 }), ), // News now comes from the research aggregator in the Details card's // News tab. Empty articles (source live) β†’ "No news available". @@ -224,11 +224,11 @@ describe("MarketLookupPage (integration)", () => { it("shows time range buttons when quote is loaded", async () => { server.use( - http.get(`${API_BASE}/api/market/quote`, () => ok({ quotes: [appleQuote] })), + http.get(`${API_BASE}/api/market/quote`, () => ok({ items: [appleQuote], total: 1 })), http.get(`${API_BASE}/api/market/chart`, () => - ok({ symbol: "AAPL", currency: "USD", points: [] }), + ok({ symbol: "AAPL", currency: "USD", items: [], total: 0 }), ), - http.get(`${API_BASE}/api/market/news`, () => ok({ articles: [] })), + http.get(`${API_BASE}/api/market/news`, () => ok({ items: [], total: 0 })), ); renderWithApp(, { initialEntries: ["/?symbol=AAPL"] }); @@ -242,9 +242,9 @@ describe("MarketLookupPage (integration)", () => { server.use( http.get(`${API_BASE}/api/market/quote`, () => err(500, "Server error")), http.get(`${API_BASE}/api/market/chart`, () => - ok({ symbol: "AAPL", currency: "USD", points: [] }), + ok({ symbol: "AAPL", currency: "USD", items: [], total: 0 }), ), - http.get(`${API_BASE}/api/market/news`, () => ok({ articles: [] })), + http.get(`${API_BASE}/api/market/news`, () => ok({ items: [], total: 0 })), ); renderWithApp(, { initialEntries: ["/?symbol=AAPL"] }); diff --git a/apps/frontend/src/pages/__tests__/PlannedPaymentsPage.integration.test.tsx b/apps/frontend/src/pages/__tests__/PlannedPaymentsPage.integration.test.tsx index a4ac8461..66f41461 100644 --- a/apps/frontend/src/pages/__tests__/PlannedPaymentsPage.integration.test.tsx +++ b/apps/frontend/src/pages/__tests__/PlannedPaymentsPage.integration.test.tsx @@ -153,6 +153,31 @@ describe("PlannedPaymentsPage (integration)", () => { expect(screen.queryByRole("dialog")).not.toBeInTheDocument(); }); + it("resets the create form (direction toggle, name) between consecutive New opens", async () => { + const user = userEvent.setup(); + renderWithApp(); + + // First "New" open: dirty the sticky fields β€” flip the direction + // toggle to Income and type a name. + await user.click(await screen.findByRole("button", { name: /new payment/i })); + await screen.findByRole("dialog"); + await user.click(screen.getByRole("radio", { name: /income/i })); + expect(screen.getByRole("radio", { name: /income/i })).toHaveAttribute("aria-checked", "true"); + await user.type(screen.getByLabelText("Name *"), "Salary"); + + // Close without saving, reopen "New". + await user.keyboard("{Escape}"); + expect(screen.queryByRole("dialog")).not.toBeInTheDocument(); + await user.click(screen.getByRole("button", { name: /new payment/i })); + await screen.findByRole("dialog"); + + // The form remounted blank: direction back on its Expense default + // (the sign-owning field), name cleared. + expect(screen.getByRole("radio", { name: /expense/i })).toHaveAttribute("aria-checked", "true"); + expect(screen.getByRole("radio", { name: /income/i })).toHaveAttribute("aria-checked", "false"); + expect(screen.getByLabelText("Name *")).toHaveValue(""); + }); + it("shows payment name in table row when MSW returns a planned payment", async () => { server.use( http.get(`${API_BASE}/api/planned-transactions`, () => diff --git a/apps/frontend/src/pages/portfolio/RebalancePage.tsx b/apps/frontend/src/pages/portfolio/RebalancePage.tsx index 69a9ab79..be80ff01 100644 --- a/apps/frontend/src/pages/portfolio/RebalancePage.tsx +++ b/apps/frontend/src/pages/portfolio/RebalancePage.tsx @@ -26,6 +26,7 @@ import { useLanguage } from "@/contexts/LanguageContext"; import { useAppSettings } from "@/contexts/AppSettingsContext"; import { useCurrencyFormatter } from "@/hooks/useCurrencyFormatter"; import { apiClient } from "@/lib/api"; +import { apiErrorToMessage } from "@/lib/api/errorMessage"; import { cn } from "@/lib/utils"; import { useRebalancePlans } from "@/hooks/useRebalancePlans"; import type { ModelPortfolio, RebalanceResponse } from "@/lib/api/crossWorkspace"; @@ -120,6 +121,9 @@ export default function RebalancePage() { const availableCashArg = useCashCap ? resolveCap(cashCapInput, availableCash) : undefined; return apiClient.computeRebalance({ targetWeights, currency, availableCash: availableCashArg }); }, + // Errors render inline below the form (compute.isError) β€” keep the global + // mutation-error backstop from also toasting the same failure. + meta: { suppressErrorToast: true }, }); const result: RebalanceResponse | undefined = compute.data; @@ -349,7 +353,7 @@ export default function RebalancePage() { )} {compute.isError && ( -

{(compute.error as Error)?.message}

+

{apiErrorToMessage(compute.error, t)}

)} {result && ( diff --git a/apps/frontend/src/pages/research/MarketLookupPage.tsx b/apps/frontend/src/pages/research/MarketLookupPage.tsx index 2b27c783..f71d9962 100644 --- a/apps/frontend/src/pages/research/MarketLookupPage.tsx +++ b/apps/frontend/src/pages/research/MarketLookupPage.tsx @@ -183,7 +183,7 @@ export default function MarketLookupPage() { const { data: quoteData, isFetching: isQuoteLoading } = useQuery({ queryKey: ["market-quote", effectiveSelectedSymbol], queryFn: async () => { - const { quotes } = await apiClient.getMarketQuotes(effectiveSelectedSymbol!, { detail: "basic" }); + const quotes = await apiClient.getMarketQuotes(effectiveSelectedSymbol!, { detail: "basic" }); return quotes[0] ?? null; }, enabled: useYahoo && isOnline, diff --git a/apps/frontend/src/pages/research/MarketOverviewPage.tsx b/apps/frontend/src/pages/research/MarketOverviewPage.tsx index 207a8159..a477a8d5 100644 --- a/apps/frontend/src/pages/research/MarketOverviewPage.tsx +++ b/apps/frontend/src/pages/research/MarketOverviewPage.tsx @@ -978,7 +978,7 @@ export default function MarketOverviewPage() { const { data } = useMarketQuotesQuery(["market-overview", region, sector], symbols, { staleTime: 60_000 }); const pctMap = useMemo( - () => new Map((data?.quotes ?? []).map((q) => [q.symbol, q.changePercent])), + () => new Map((data ?? []).map((q) => [q.symbol, q.changePercent])), [data], ); diff --git a/apps/frontend/src/pages/research/ResearchHomePage.tsx b/apps/frontend/src/pages/research/ResearchHomePage.tsx index 4b50258e..7b40e14a 100644 --- a/apps/frontend/src/pages/research/ResearchHomePage.tsx +++ b/apps/frontend/src/pages/research/ResearchHomePage.tsx @@ -55,7 +55,7 @@ export default function ResearchHomePage() { // Live benchmark strip. 60s polling mirrors the watchlist quote cadence. const { data: benchmarkData } = useMarketQuotesQuery(["research-benchmarks", BENCHMARK_SYMBOLS], BENCHMARK_SYMBOLS, { staleTime: 60_000 }); const benchmarkMap = useMemo( - () => new Map((benchmarkData?.quotes ?? []).map((q) => [q.symbol, q])), + () => new Map((benchmarkData ?? []).map((q) => [q.symbol, q])), [benchmarkData], ); @@ -74,7 +74,7 @@ export default function ResearchHomePage() { ); const { data: watchlistQuotes } = useMarketQuotesQuery(["watchlist-quotes", watchlistSymbols], watchlistSymbols, { staleTime: 60_000 }); const watchlistPriceMap = useMemo( - () => new Map((watchlistQuotes?.quotes ?? []).map((q) => [q.symbol, q])), + () => new Map((watchlistQuotes ?? []).map((q) => [q.symbol, q])), [watchlistQuotes], ); diff --git a/apps/frontend/src/pages/research/WatchlistPage.tsx b/apps/frontend/src/pages/research/WatchlistPage.tsx index fda04067..367a7d1e 100644 --- a/apps/frontend/src/pages/research/WatchlistPage.tsx +++ b/apps/frontend/src/pages/research/WatchlistPage.tsx @@ -50,7 +50,7 @@ export default function WatchlistPage() { const { data: quotesData, isError: quotesError } = useMarketQuotesQuery(["watchlist-quotes", symbols], symbols); const quotesUnavailable = !isOnline || quotesError; - const priceMap = new Map(quotesData?.quotes?.map((q) => [q.symbol, q]) || []); + const priceMap = new Map(quotesData?.map((q) => [q.symbol, q]) || []); const deleteMutation = useMutation({ mutationFn: (id: number) => apiClient.deleteWatchlistItem(id), diff --git a/apps/frontend/src/test/live-contracts/live-contracts.test.ts b/apps/frontend/src/test/live-contracts/live-contracts.test.ts index f5f0361c..e4cd7862 100644 --- a/apps/frontend/src/test/live-contracts/live-contracts.test.ts +++ b/apps/frontend/src/test/live-contracts/live-contracts.test.ts @@ -223,9 +223,13 @@ describe.skipIf(!enabled)("Live backend API contracts (E5)", () => { ); }); - it("GET /api/planned-transactions/due-soon returns array", async () => { + it("GET /api/planned-transactions/due-soon returns { items, total, days }", async () => { const data = await get("/api/planned-transactions/due-soon"); - validate(z.array(z.unknown()), data, "GET /api/planned-transactions/due-soon"); + validate( + collectionSchema().extend({ days: z.number().int().positive() }), + data, + "GET /api/planned-transactions/due-soon", + ); }); it("GET /api/aggregations/category-breakdown returns expected shape", async () => { @@ -343,18 +347,14 @@ describe.skipIf(!enabled)("Live backend API contracts (E5)", () => { ); }); - it("GET /api/info/banks returns banks object", async () => { + it("GET /api/info/banks returns { items, total }", async () => { const data = await get("/api/info/banks"); - validate(z.object({ banks: z.array(z.unknown()) }), data, "GET /api/info/banks"); + validate(collectionSchema(), data, "GET /api/info/banks"); }); - it("GET /api/info/supported-adapters returns adapters object", async () => { + it("GET /api/info/supported-adapters returns { items, total }", async () => { const data = await get("/api/info/supported-adapters"); - validate( - z.object({ adapters: z.array(z.unknown()), total_count: z.number() }), - data, - "GET /api/info/supported-adapters", - ); + validate(collectionSchema(), data, "GET /api/info/supported-adapters"); }); it("GET /api/info/recurring-patterns returns patterns object", async () => { @@ -397,9 +397,9 @@ describe.skipIf(!enabled)("Live backend API contracts (E5)", () => { validate(collectionSchema(), data, "GET /api/admin/providers/health"); }); - it("GET /api/admin/metrics/requests returns array", async () => { + it("GET /api/admin/metrics/requests returns { items, total }", async () => { const data = await get("/api/admin/metrics/requests"); - validate(z.array(z.unknown()), data, "GET /api/admin/metrics/requests"); + validate(collectionSchema(), data, "GET /api/admin/metrics/requests"); }); it("GET /api/admin/endpoints returns { items, total }", async () => { @@ -438,8 +438,8 @@ describe.skipIf(!enabled)("Live backend API contracts (E5)", () => { // (GET /api/info/transaction-summary removed β€” Phase 9 cutover deleted the route.) - it("GET /api/market/news returns articles array", async () => { + it("GET /api/market/news returns { items, total }", async () => { const data = await get("/api/market/news"); - validate(z.object({ articles: z.array(z.unknown()) }), data, "GET /api/market/news"); + validate(collectionSchema(), data, "GET /api/market/news"); }); }); diff --git a/apps/frontend/src/test/msw/contracts.test.ts b/apps/frontend/src/test/msw/contracts.test.ts index 6aafc871..5ff0e116 100644 --- a/apps/frontend/src/test/msw/contracts.test.ts +++ b/apps/frontend/src/test/msw/contracts.test.ts @@ -242,7 +242,11 @@ const NewsArticleSchema = z.object({ thumbnail: z.string().nullable(), relatedSymbols: z.array(z.string()), }); -const MarketNewsSchema = z.object({ articles: z.array(NewsArticleSchema) }); +// Canonical `{ items, total }` collection body. +const MarketNewsSchema = z.object({ + items: z.array(NewsArticleSchema), + total: z.number().int().nonnegative(), +}); // Canonical collection body: { items, total, limit, offset }. const ImportBatchesSchema = z.object({ @@ -494,10 +498,10 @@ describe("Missing GET endpoint contracts (E4)", () => { z.object({ items: z.array(z.unknown()) }), ], [ - "GET /api/market/quote returns expected shape", + "GET /api/market/quote returns { items, total }", "/api/market/quote", "GET /api/market/quote", - z.object({ quotes: z.array(z.unknown()) }), + collectionSchema(), ], [ "GET /api/market/search returns expected shape", @@ -574,16 +578,16 @@ describe("Missing GET endpoint contracts (E4)", () => { z.object({ patterns: z.array(z.unknown()), total: z.number() }), ], [ - "GET /api/info/banks returns { banks }", + "GET /api/info/banks returns { items, total }", "/api/info/banks", "GET /api/info/banks", - z.object({ banks: z.array(z.unknown()) }), + collectionSchema(), ], [ - "GET /api/info/supported-adapters returns { adapters, total_count }", + "GET /api/info/supported-adapters returns { items, total }", "/api/info/supported-adapters", "GET /api/info/supported-adapters", - z.object({ adapters: z.array(z.unknown()), total_count: z.number() }), + collectionSchema(), ], [ "GET /api/info/inflation-rates returns array", @@ -610,10 +614,10 @@ describe("Missing GET endpoint contracts (E4)", () => { collectionSchema(), ], [ - "GET /api/admin/metrics/requests returns array", + "GET /api/admin/metrics/requests returns { items, total }", "/api/admin/metrics/requests", "GET /api/admin/metrics/requests", - z.array(z.unknown()), + collectionSchema(), ], [ "GET /api/admin/endpoints returns { items, total }", @@ -777,10 +781,10 @@ describe("Phase F1: extended GET endpoint contracts", () => { }), ], [ - "GET /api/ai/models returns models array", + "GET /api/ai/models returns { items, total }", "/api/ai/models", "GET /api/ai/models", - z.object({ models: z.array(z.unknown()) }), + collectionSchema(), ], [ "GET /api/attachments/transaction/:id returns items array", @@ -867,10 +871,10 @@ describe("Phase F1: extended GET endpoint contracts", () => { TransactionItemSchema, ], [ - "GET /api/planned-transactions/due-soon returns array", + "GET /api/planned-transactions/due-soon returns { items, total, days }", "/api/planned-transactions/due-soon", "GET /api/planned-transactions/due-soon", - z.array(z.unknown()), + collectionSchema().extend({ days: z.number().int().positive() }), ], ])("%s", async (_name, path, label, schema) => { validate(schema, await getEnvelope(path), label); diff --git a/apps/frontend/src/test/msw/handlers.ts b/apps/frontend/src/test/msw/handlers.ts index cf525c7c..e2d805ba 100644 --- a/apps/frontend/src/test/msw/handlers.ts +++ b/apps/frontend/src/test/msw/handlers.ts @@ -323,7 +323,7 @@ export const defaultHandlers = [ http.get(`${API_BASE}/api/info/exchange-rates`, () => ok({ rates: [], fallback_rates: {}, base: "EUR", date: "2025-01-01" }), ), - http.get(`${API_BASE}/api/market/news`, () => ok({ articles: [] })), + http.get(`${API_BASE}/api/market/news`, () => ok({ items: [], total: 0 })), http.get(`${API_BASE}/api/import/batches`, () => ok({ items: [], total: 0, limit: 50, offset: 0 }), @@ -338,7 +338,7 @@ export const defaultHandlers = [ http.get(`${API_BASE}/api/splits/owed`, () => ok({ items: [] })), - http.get(`${API_BASE}/api/market/quote`, () => ok({ quotes: [] })), + http.get(`${API_BASE}/api/market/quote`, () => ok({ items: [], total: 0 })), http.get(`${API_BASE}/api/market/search`, () => ok({ results: [] })), // Research aggregator endpoints (consumed by the Market Lookup Details tabs // β€” scorecard fires on load, analyst/news on tab-click). Default to an @@ -388,8 +388,8 @@ export const defaultHandlers = [ cashForecast: null, }), ), - http.get(`${API_BASE}/api/info/banks`, () => ok({ banks: [] })), - http.get(`${API_BASE}/api/info/supported-adapters`, () => ok({ adapters: [], total_count: 0 })), + http.get(`${API_BASE}/api/info/banks`, () => ok({ items: [], total: 0 })), + http.get(`${API_BASE}/api/info/supported-adapters`, () => ok({ items: [], total: 0 })), http.get(`${API_BASE}/api/info/inflation-rates`, () => ok([])), http.get(`${API_BASE}/api/admin/endpoint-liveness`, () => ok({ items: [], total: 0 })), @@ -397,7 +397,7 @@ export const defaultHandlers = [ ok({ tables: [], db_size: null }), ), http.get(`${API_BASE}/api/admin/providers/health`, () => ok({ items: [], total: 0 })), - http.get(`${API_BASE}/api/admin/metrics/requests`, () => ok([])), + http.get(`${API_BASE}/api/admin/metrics/requests`, () => ok({ items: [], total: 0 })), http.get(`${API_BASE}/api/admin/endpoints`, () => ok({ items: [], total: 0 })), // ── Mutation stubs ─────────────────────────────────────────────────────── @@ -433,7 +433,7 @@ export const defaultHandlers = [ http.post(`${API_BASE}/api/planned-transactions/:id/execute`, () => ok({ ...PLANNED_TRANSACTION_STUB, is_executed: true }), ), - http.get(`${API_BASE}/api/planned-transactions/due-soon`, () => ok([])), + http.get(`${API_BASE}/api/planned-transactions/due-soon`, () => ok({ items: [], total: 0, days: 7 })), // ── Phase F1: full contract surface coverage ──────────────────────────── @@ -462,7 +462,7 @@ export const defaultHandlers = [ }), ), http.delete(`${API_BASE}/api/ai/conversations/:id`, () => noContent()), - http.get(`${API_BASE}/api/ai/models`, () => ok({ models: [] })), + http.get(`${API_BASE}/api/ai/models`, () => ok({ items: [], total: 0 })), // Attachments http.get(`${API_BASE}/api/attachments/transaction/:id`, () => ok({ items: [] })), @@ -644,11 +644,9 @@ export const defaultHandlers = [ // Recipients delete (already handled above) β€” clusters, etc. covered - // Market chart + // Market chart β€” canonical `{items, total}` collection body with the + // symbol/currency metadata alongside. http.get(`${API_BASE}/api/market/chart`, () => - new Response(JSON.stringify({ symbol: "TEST", currency: "USD", points: [] }), { - status: 200, - headers: { "Content-Type": "application/json" }, - }), + ok({ symbol: "TEST", currency: "USD", items: [], total: 0 }), ), ]; diff --git a/apps/frontend/src/types/aiChat.ts b/apps/frontend/src/types/aiChat.ts index e274d405..708076ab 100644 --- a/apps/frontend/src/types/aiChat.ts +++ b/apps/frontend/src/types/aiChat.ts @@ -130,8 +130,10 @@ export interface OllamaModel { modifiedAt: string | null; } +/** Canonical `{items, total}` collection body of GET /api/ai/models. */ export interface OllamaModelsResponse { - models: OllamaModel[]; + items: OllamaModel[]; + total: number; } export interface CreateConversationBody { diff --git a/apps/frontend/src/types/api.ts b/apps/frontend/src/types/api.ts index fe8300b5..1f1d69f4 100644 --- a/apps/frontend/src/types/api.ts +++ b/apps/frontend/src/types/api.ts @@ -228,6 +228,9 @@ export interface TransactionCreate { // Nullable fields carry PATCH null-to-clear semantics: explicit null clears // the value server-side, undefined (absent key) leaves it unchanged. `?? // undefined` on a cleared value silently dropped the key β€” the clear no-op'd. +// `balance` is deliberately absent: the running balance is bank-stamped import +// data (ADR-094) and the backend PATCH whitelist drops it β€” sending it was a +// silent no-op that made the field look editable. export interface TransactionUpdate { transaction_date?: string; bank_account?: string | null; @@ -236,7 +239,6 @@ export interface TransactionUpdate { memo?: string | null; amount?: number; currency?: string; - balance?: number; category_id?: number | null; category_name?: string; comment?: string | null; diff --git a/apps/frontend/src/types/generated.ts b/apps/frontend/src/types/generated.ts index 05c18d20..625ebc8b 100644 --- a/apps/frontend/src/types/generated.ts +++ b/apps/frontend/src/types/generated.ts @@ -3313,7 +3313,6 @@ export interface components { memo?: string | null; amount?: number; currency?: string; - balance?: number; category_id?: number | null; category_name?: string; comment?: string | null; @@ -5898,14 +5897,18 @@ export interface operations { }; requestBody?: never; responses: { - /** @description Due-soon list */ + /** @description Due-soon list (canonical `{items, total}` body; `days` echoes the effective window) */ 200: { headers: { [name: string]: unknown; }; content: { "application/json": components["schemas"]["Envelope"] & { - data?: components["schemas"]["PlannedTransaction"][]; + data?: { + items: components["schemas"]["PlannedTransaction"][]; + total: number; + days: number; + }; }; }; }; @@ -5920,7 +5923,7 @@ export interface operations { }; requestBody?: never; responses: { - /** @description Match suggestions */ + /** @description Match suggestions (canonical `{items, total}` body) */ 200: { headers: { [name: string]: unknown; @@ -5928,24 +5931,27 @@ export interface operations { content: { "application/json": components["schemas"]["Envelope"] & { data?: { - planned?: { - id?: number; - recipient_id?: number | null; - recipient_name?: string | null; - amount?: number; - planned_date?: string; - currency?: string | null; - is_recurring?: boolean; - }; - candidates?: { - id?: number; - recipient_name?: string | null; - amount?: number; - transaction_date?: string; - currency?: string | null; - memo?: string | null; + total: number; + items: { + planned?: { + id?: number; + recipient_id?: number | null; + recipient_name?: string | null; + amount?: number; + planned_date?: string; + currency?: string | null; + is_recurring?: boolean; + }; + candidates?: { + id?: number; + recipient_name?: string | null; + amount?: number; + transaction_date?: string; + currency?: string | null; + memo?: string | null; + }[]; }[]; - }[]; + }; }; }; }; @@ -6080,13 +6086,18 @@ export interface operations { }; requestBody?: never; responses: { - /** @description Bank list */ + /** @description Bank list (canonical `{items, total}` body) */ 200: { headers: { [name: string]: unknown; }; content: { - "application/json": components["schemas"]["Envelope"]; + "application/json": components["schemas"]["Envelope"] & { + data?: { + items: string[]; + total: number; + }; + }; }; }; }; @@ -6100,13 +6111,22 @@ export interface operations { }; requestBody?: never; responses: { - /** @description Adapter list */ + /** @description Adapter list (canonical `{items, total}` body) */ 200: { headers: { [name: string]: unknown; }; content: { - "application/json": components["schemas"]["Envelope"]; + "application/json": components["schemas"]["Envelope"] & { + data?: { + items: { + key: string; + name: string; + adapter_class?: string; + }[]; + total: number; + }; + }; }; }; }; @@ -7043,13 +7063,18 @@ export interface operations { }; requestBody?: never; responses: { - /** @description Quote data */ + /** @description Quote data (canonical `{items, total}` body) */ 200: { headers: { [name: string]: unknown; }; content: { - "application/json": components["schemas"]["Envelope"]; + "application/json": components["schemas"]["Envelope"] & { + data?: { + items: Record[]; + total: number; + }; + }; }; }; }; @@ -7067,13 +7092,26 @@ export interface operations { }; requestBody?: never; responses: { - /** @description OHLC chart data */ + /** @description OHLC chart data (canonical `{items, total}` series with `symbol`/`currency` alongside) */ 200: { headers: { [name: string]: unknown; }; content: { - "application/json": components["schemas"]["Envelope"]; + "application/json": components["schemas"]["Envelope"] & { + data?: { + symbol?: string; + currency?: string; + items: { + time?: number; + close?: number; + high?: number; + low?: number; + volume?: number; + }[]; + total: number; + }; + }; }; }; }; @@ -7089,13 +7127,18 @@ export interface operations { }; requestBody?: never; responses: { - /** @description News items */ + /** @description News items (canonical `{items, total}` body) */ 200: { headers: { [name: string]: unknown; }; content: { - "application/json": components["schemas"]["Envelope"]; + "application/json": components["schemas"]["Envelope"] & { + data?: { + items: Record[]; + total: number; + }; + }; }; }; }; @@ -7746,7 +7789,7 @@ export interface operations { }; requestBody?: never; responses: { - /** @description Model list */ + /** @description Model list (canonical `{items, total}` body) */ 200: { headers: { [name: string]: unknown; @@ -7754,7 +7797,8 @@ export interface operations { content: { "application/json": components["schemas"]["Envelope"] & { data?: { - models: components["schemas"]["OllamaModel"][]; + items: components["schemas"]["OllamaModel"][]; + total: number; }; }; }; @@ -9213,13 +9257,18 @@ export interface operations { }; requestBody?: never; responses: { - /** @description Request metrics */ + /** @description Request metrics (canonical `{items, total}` body) */ 200: { headers: { [name: string]: unknown; }; content: { - "application/json": components["schemas"]["Envelope"]; + "application/json": components["schemas"]["Envelope"] & { + data?: { + items: Record[]; + total: number; + }; + }; }; }; }; @@ -9589,7 +9638,7 @@ export interface operations { }; requestBody?: never; responses: { - /** @description Stored mappings ({ mappings: [...] }) */ + /** @description Stored mappings (canonical `{items, total}` body) */ 200: { headers: { [name: string]: unknown; @@ -9628,7 +9677,7 @@ export interface operations { }; }; responses: { - /** @description Updated mapping set */ + /** @description Updated mapping set (canonical `{items, total}` body) */ 200: { headers: { [name: string]: unknown; @@ -9762,7 +9811,7 @@ export interface operations { }; requestBody?: never; responses: { - /** @description { providers: [{ provider, label, envVar, configured, source, masked }] } */ + /** @description Canonical `{items, total}` body; each item is { provider, label, envVar, configured, source, masked } */ 200: { headers: { [name: string]: unknown; @@ -9790,7 +9839,7 @@ export interface operations { }; }; responses: { - /** @description Updated masked statuses */ + /** @description Updated masked statuses (canonical `{items, total}` body) */ 200: { headers: { [name: string]: unknown; diff --git a/apps/frontend/src/types/research.ts b/apps/frontend/src/types/research.ts index e6094d0a..606f1701 100644 --- a/apps/frontend/src/types/research.ts +++ b/apps/frontend/src/types/research.ts @@ -242,8 +242,10 @@ export interface MappingSaveInput { currency?: string; } +/** Canonical `{items, total}` collection body (GET and POST /api/research/mappings). */ export interface MappingsResponse { - mappings: InstrumentProviderMapping[]; + items: InstrumentProviderMapping[]; + total: number; } export type MappingDiscrepancyType = 'currency_mismatch' | 'price_outlier'; @@ -279,8 +281,10 @@ export interface ProviderKeyStatus { masked?: string; } +/** Canonical `{items, total}` collection body (GET and PUT /api/research/provider-keys). */ export interface ProviderKeysResponse { - providers: ProviderKeyStatus[]; + items: ProviderKeyStatus[]; + total: number; } // ── Fundamentals scorecard (ADR-081) ───────────────────────────────────────── diff --git a/apps/node-backend/scripts/checkjs-ratchet.js b/apps/node-backend/scripts/checkjs-ratchet.js index c00ba105..abacb7a7 100644 --- a/apps/node-backend/scripts/checkjs-ratchet.js +++ b/apps/node-backend/scripts/checkjs-ratchet.js @@ -42,38 +42,18 @@ const CONFIG_PATH = path.join(ROOT, 'tsconfig.check.strict.json'); * `apps/node-backend/`, POSIX separators. An entry ending in `/` is a directory * prefix; anything else is an exact file. * - * Seeded with the core data-layer row shapes (`src/types/rows.js`) and the - * repositories annotated against them. `src/repositories/` as a whole is the - * target β€” the info* aggregate repositories and the smaller side-table - * repositories are the remaining work. + * Seeded with the core data-layer row shapes (`src/types/rows.js`), then grown + * file-by-file until every repository was annotated. The whole data layer is + * now held as directory prefixes, so a NEW file under either directory is + * ratcheted from birth. Per the original plan, the next targets are further + * directories (services/, routes/, …) β€” grow them file-by-file, then collapse + * to the directory prefix once complete. * * @type {string[]} */ const RATCHETED = [ 'src/types/', - 'src/repositories/accountBalanceSql.js', - 'src/repositories/accountRepository.js', - 'src/repositories/cashflowForecastAccuracyRepository.js', - 'src/repositories/cashflowForecastMcRepository.js', - 'src/repositories/cashflowForecastMcRollingRepository.js', - 'src/repositories/categoryRepository.js', - 'src/repositories/importBatchRepository.js', - 'src/repositories/infoRepository.js', - 'src/repositories/infoRepositoryTags.js', - 'src/repositories/investmentRepository.js', - 'src/repositories/plannedTransactionRepository.js', - 'src/repositories/portfolioImportBatchRepository.js', - 'src/repositories/portfolioTransactionRepository.js', - 'src/repositories/portfolioTxRepo.common.js', - 'src/repositories/portfolioTxRepo.reads.js', - 'src/repositories/portfolioTxRepo.writes.js', - 'src/repositories/providerApiKeyRepository.js', - 'src/repositories/providerHealthRepository.js', - 'src/repositories/providerQuotaRepository.js', - 'src/repositories/recipientRepository.js', - 'src/repositories/splitRepository.js', - 'src/repositories/tagRepository.js', - 'src/repositories/transactionRepository.js', + 'src/repositories/', ]; /** Directory the "ready to ratchet" hint scans for already-clean files. */ diff --git a/apps/node-backend/src/controllers/investmentController.js b/apps/node-backend/src/controllers/investmentController.js index 7ff07c36..e7eeb385 100644 --- a/apps/node-backend/src/controllers/investmentController.js +++ b/apps/node-backend/src/controllers/investmentController.js @@ -104,6 +104,17 @@ const investmentBodySchema = z.looseObject({ location: maxLenField('location', 300), municipality: maxLenField('municipality', 200), currency: currencyField, + // Provider columns (migration 0001): URL shape is checked separately + // (validateProviderUrls), but an over-length yet valid URL/path still + // reached the VARCHAR column as a raw 22001 500. + price_provider_id: maxLenField('price_provider_id', 200), + price_provider_url: maxLenField('price_provider_url', 500), + price_provider_latest_url: maxLenField('price_provider_latest_url', 500), + price_provider_latest_path: maxLenField('price_provider_latest_path', 300), + price_provider_history_url: maxLenField('price_provider_history_url', 500), + price_provider_history_path: maxLenField('price_provider_history_path', 300), + price_provider_history_ts_path: maxLenField('price_provider_history_ts_path', 300), + price_provider_history_price_path: maxLenField('price_provider_history_price_path', 300), }); function parseInvestmentBody(body) { @@ -498,6 +509,17 @@ export async function updateTransaction(req, res) { const txnId = requireTxnId(req); const fields = { ...(req.body || {}) }; + // Validate a free-typed currency (ISO shape, uppercased) before it reaches + // the VARCHAR(10) column β€” create validates it, but PATCH forwarded the raw + // value (garbage stored; >10 chars 22001'd). The column is NOT NULL, so an + // explicit null/'' (clear) rejects instead of 500ing at the constraint. + if (fields.currency !== undefined) { + if (fields.currency === null || fields.currency === '') { + throw new ValidationError('currency cannot be cleared'); + } + fields.currency = assertCurrency(fields.currency); + } + // A date or currency change invalidates the stamped FX rate β€” recompute it // unless the client supplied one explicitly. if ( diff --git a/apps/node-backend/src/repositories/aiChatRepository.js b/apps/node-backend/src/repositories/aiChatRepository.js index c566ada7..05d14d3b 100644 --- a/apps/node-backend/src/repositories/aiChatRepository.js +++ b/apps/node-backend/src/repositories/aiChatRepository.js @@ -13,6 +13,9 @@ import { query } from '../database/connection.js'; +/** @typedef {import('../types/rows.js').AiConversationRow} AiConversationRow */ +/** @typedef {import('../types/rows.js').AiMessageRow} AiMessageRow */ + const CONVERSATION_COLUMNS = 'id, title, model, created_at AS "createdAt", updated_at AS "updatedAt"'; const MESSAGE_COLUMNS = @@ -23,6 +26,10 @@ const MESSAGE_COLUMNS = const PG_FK_VIOLATION = '23503'; export class ConversationDeletedError extends Error { + /** + * @param {string} conversationId UUID of the deleted conversation. + * @param {unknown} [cause] The underlying pg FK-violation error. + */ constructor(conversationId, cause) { super(`Conversation ${conversationId} was deleted while a message was being appended`); this.name = 'ConversationDeletedError'; @@ -32,12 +39,17 @@ export class ConversationDeletedError extends Error { } } +/** + * @param {any} value Arbitrary JSON-serialisable value. + * @returns {string|null} + */ function serializeJsonb(value) { if (value === undefined || value === null) return null; return JSON.stringify(value); } const aiChatRepository = { + /** @returns {Promise} */ async listConversations() { const result = await query( `SELECT ${CONVERSATION_COLUMNS} @@ -47,6 +59,10 @@ const aiChatRepository = { return result.rows; }, + /** + * @param {string} id UUID. + * @returns {Promise} + */ async getConversation(id) { const result = await query( `SELECT ${CONVERSATION_COLUMNS} FROM ai_conversations WHERE id = $1`, @@ -55,6 +71,10 @@ const aiChatRepository = { return result.rows[0] || null; }, + /** + * @param {{ title: string, model: string }} input + * @returns {Promise} + */ async createConversation({ title, model }) { const result = await query( `INSERT INTO ai_conversations (title, model) @@ -65,6 +85,11 @@ const aiChatRepository = { return result.rows[0]; }, + /** + * @param {string} id UUID. + * @param {string} title + * @returns {Promise} + */ async renameConversation(id, title) { const result = await query( `UPDATE ai_conversations @@ -76,6 +101,11 @@ const aiChatRepository = { return result.rows[0] || null; }, + /** + * @param {string} id UUID. + * @param {string} model + * @returns {Promise} + */ async updateConversationModel(id, model) { const result = await query( `UPDATE ai_conversations @@ -87,6 +117,10 @@ const aiChatRepository = { return result.rows[0] || null; }, + /** + * @param {string} id UUID. + * @returns {Promise} true if a row was removed + */ async deleteConversation(id) { const result = await query( `DELETE FROM ai_conversations WHERE id = $1 RETURNING id`, @@ -95,6 +129,10 @@ const aiChatRepository = { return result.rows.length > 0; }, + /** + * @param {string} conversationId UUID. + * @returns {Promise} + */ async getMessages(conversationId) { const result = await query( `SELECT ${MESSAGE_COLUMNS} @@ -106,6 +144,18 @@ const aiChatRepository = { return result.rows; }, + /** + * @param {{ + * conversationId: string, + * role: string, + * content?: string|null, + * toolName?: string|null, + * toolArgs?: any, + * toolResult?: any, + * status?: string, + * }} input + * @returns {Promise} + */ async appendMessage({ conversationId, role, @@ -132,7 +182,7 @@ const aiChatRepository = { ], ); return result.rows[0]; - } catch (err) { + } catch (/** @type {any} */ err) { if (err && err.code === PG_FK_VIOLATION) { throw new ConversationDeletedError(conversationId, err); } @@ -140,6 +190,11 @@ const aiChatRepository = { } }, + /** + * @param {string} id UUID. + * @param {string} status + * @returns {Promise} + */ async updateMessageStatus(id, status) { const result = await query( `UPDATE ai_messages diff --git a/apps/node-backend/src/repositories/attachmentRepository.js b/apps/node-backend/src/repositories/attachmentRepository.js index 6e76cf46..6c775e94 100644 --- a/apps/node-backend/src/repositories/attachmentRepository.js +++ b/apps/node-backend/src/repositories/attachmentRepository.js @@ -9,6 +9,13 @@ import { query } from '../database/connection.js'; import { buildLimitOffset } from '../lib/sqlClauses.js'; +/** @typedef {import('../types/rows.js').AttachmentRow} AttachmentRow */ +/** @typedef {import('../types/rows.js').FormattedAttachment} FormattedAttachment */ + +/** + * @param {AttachmentRow} row + * @returns {FormattedAttachment} + */ function formatRow(row) { return { id: row.id, @@ -25,6 +32,8 @@ export const attachmentRepository = { /** * Whether the parent transaction row exists. Upload guard β€” checked before * the file is written to disk so a bad id 404s instead of orphaning a file. + * @param {number} transactionId + * @returns {Promise} */ async transactionExists(transactionId) { const result = await query('SELECT id FROM transactions WHERE id = $1', [transactionId]); @@ -38,6 +47,7 @@ export const attachmentRepository = { * * @param {number} transactionId * @param {{ limit?: number|null, offset?: number }} [page] + * @returns {Promise} */ async listByTransaction(transactionId, { limit = null, offset = 0 } = {}) { const params = [transactionId]; @@ -50,7 +60,11 @@ export const attachmentRepository = { return result.rows.map(formatRow); }, - /** Attachment count for a transaction β€” the `total` for a paginated list. */ + /** + * Attachment count for a transaction β€” the `total` for a paginated list. + * @param {number} transactionId + * @returns {Promise} + */ async countByTransaction(transactionId) { const result = await query( 'SELECT COUNT(*) FROM attachments WHERE transaction_id = $1', @@ -63,6 +77,8 @@ export const attachmentRepository = { * List stored file paths for a set of transactions. Used before a hard * delete so the DB CASCADE (which only removes the rows) can be followed * by best-effort file removal. + * @param {number[]} transactionIds + * @returns {Promise} */ async listPathsByTransactionIds(transactionIds) { if (!Array.isArray(transactionIds) || transactionIds.length === 0) return []; @@ -70,11 +86,19 @@ export const attachmentRepository = { 'SELECT stored_path FROM attachments WHERE transaction_id = ANY($1::int[])', [transactionIds], ); - return result.rows.map((row) => row.stored_path); + return result.rows.map((/** @type {{ stored_path: string }} */ row) => row.stored_path); }, /** * Insert a new attachment row. Returns the created row. + * @param {{ + * transaction_id: number|string, + * filename: string, + * stored_path: string, + * mime_type: string, + * size_bytes: number, + * }} input + * @returns {Promise} */ async create({ transaction_id, filename, stored_path, mime_type, size_bytes }) { const sql = ` @@ -94,6 +118,8 @@ export const attachmentRepository = { /** * Fetch a single attachment by ID. Returns null if not found. + * @param {number|string} id + * @returns {Promise} */ async findById(id) { const result = await query('SELECT * FROM attachments WHERE id = $1', [id]); @@ -102,6 +128,8 @@ export const attachmentRepository = { /** * Delete an attachment row by ID. Returns true if a row was deleted. + * @param {number|string} id + * @returns {Promise} */ async deleteById(id) { const result = await query('DELETE FROM attachments WHERE id = $1', [id]); diff --git a/apps/node-backend/src/repositories/customParserConfigRepository.js b/apps/node-backend/src/repositories/customParserConfigRepository.js index 8d50e0a8..fa9206e0 100644 --- a/apps/node-backend/src/repositories/customParserConfigRepository.js +++ b/apps/node-backend/src/repositories/customParserConfigRepository.js @@ -1,8 +1,15 @@ import { query } from '../database/connection.js'; import { buildSetClauses } from '../lib/sqlClauses.js'; +/** @typedef {import('../types/rows.js').CustomParserConfigRow} CustomParserConfigRow */ +/** @typedef {import('../types/rows.js').FormattedCustomParserConfig} FormattedCustomParserConfig */ + const COLUMNS = 'id, name, kind, config_json, created_at, updated_at'; +/** + * @param {CustomParserConfigRow} r + * @returns {FormattedCustomParserConfig} + */ function mapRow(r) { return { id: r.id, @@ -16,6 +23,10 @@ function mapRow(r) { } const customParserConfigRepository = { + /** + * @param {string} [kind] + * @returns {Promise} + */ async getAll(kind = 'transaction') { const result = await query( `SELECT ${COLUMNS} FROM custom_parser_configs WHERE kind = $1 ORDER BY name ASC`, @@ -24,12 +35,21 @@ const customParserConfigRepository = { return result.rows.map(mapRow); }, + /** + * @param {number} id + * @returns {Promise} + */ async getById(id) { const result = await query(`SELECT ${COLUMNS} FROM custom_parser_configs WHERE id = $1`, [id]); const r = result.rows[0]; return r ? mapRow(r) : undefined; }, + /** + * @param {string} name + * @param {string} [kind] + * @returns {Promise} + */ async getByName(name, kind = 'transaction') { const result = await query( `SELECT ${COLUMNS} FROM custom_parser_configs WHERE name = $1 AND kind = $2`, @@ -39,6 +59,10 @@ const customParserConfigRepository = { return r ? mapRow(r) : undefined; }, + /** + * @param {{ name: string, config: any, kind?: string }} input + * @returns {Promise} + */ async create({ name, config, kind = 'transaction' }) { const result = await query( `INSERT INTO custom_parser_configs (name, kind, config_json) @@ -49,6 +73,11 @@ const customParserConfigRepository = { return mapRow(result.rows[0]); }, + /** + * @param {number} id + * @param {{ name?: string, config?: any }} patch + * @returns {Promise} + */ async update(id, { name, config }) { // Shared clause builder (lib/sqlClauses.js): undefined fields are skipped. const { clauses: fields, params: values, nextIdx: idx } = buildSetClauses({ @@ -68,6 +97,10 @@ const customParserConfigRepository = { return result.rows[0] ? mapRow(result.rows[0]) : undefined; }, + /** + * @param {number} id + * @returns {Promise} true if a row was removed + */ async delete(id) { const result = await query(`DELETE FROM custom_parser_configs WHERE id = $1 RETURNING id`, [id]); return result.rows.length > 0; diff --git a/apps/node-backend/src/repositories/infoRepositoryAverageVsCurrent.js b/apps/node-backend/src/repositories/infoRepositoryAverageVsCurrent.js index 13859762..10b738ed 100644 --- a/apps/node-backend/src/repositories/infoRepositoryAverageVsCurrent.js +++ b/apps/node-backend/src/repositories/infoRepositoryAverageVsCurrent.js @@ -57,6 +57,7 @@ export async function getAverageVsCurrentSpending(targetCurrency = 'EUR') { targetCurrency ); + /** @type {Record} */ const monthlySpending = {}; for (const row of past6Converted) { const dateStr = row.date instanceof Date ? formatDateToYmd(row.date) : row.date; @@ -91,6 +92,7 @@ export async function getAverageVsCurrentSpending(targetCurrency = 'EUR') { targetCurrency ); + /** @type {Record} */ const dailyMap = {}; for (const row of currentConverted) { const dateStr = row.date instanceof Date ? formatDateToYmd(row.date) : row.date; diff --git a/apps/node-backend/src/repositories/infoRepositoryBanks.js b/apps/node-backend/src/repositories/infoRepositoryBanks.js index 87d3de84..4f6ef917 100644 --- a/apps/node-backend/src/repositories/infoRepositoryBanks.js +++ b/apps/node-backend/src/repositories/infoRepositoryBanks.js @@ -134,14 +134,26 @@ export const banksRepository = { const [currentBalancesConverted, historyConverted] = await batchConvertGroupsWithHistoricalRateFallback( [ - latestBalanceResult.rows.map(r => ({ + latestBalanceResult.rows.map((/** @type {{ + bank_account: string, display_name: string, currency: string|null, + balance: string, drift: string|null, + anchor_date: string|null, post_anchor_count: string|null, + date: Date|null, transaction_count: string, + first_transaction: Date|null, last_transaction: Date|null, + }} */ r) => ({ ...r, amount: toNumber(toDecimal(r.balance)), currency: r.currency || 'EUR', })), historyResult.rows - .filter(r => r.bank_account) - .map(r => ({ + .filter((/** @type {{ + bank_account: string, day: string, currency: string, + balance: string, date: Date, + }} */ r) => r.bank_account) + .map((/** @type {{ + bank_account: string, day: string, currency: string, + balance: string, date: Date, + }} */ r) => ({ ...r, amount: toNumber(toDecimal(r.balance)), currency: r.currency || 'EUR', @@ -173,6 +185,7 @@ export const banksRepository = { totalNetPosition += balance; } + /** @type {Record>} */ const historyMap = {}; for (const row of historyConverted) { const key = row.bank_account; diff --git a/apps/node-backend/src/repositories/infoRepositoryForecast.js b/apps/node-backend/src/repositories/infoRepositoryForecast.js index b3a48e48..0af3f37e 100644 --- a/apps/node-backend/src/repositories/infoRepositoryForecast.js +++ b/apps/node-backend/src/repositories/infoRepositoryForecast.js @@ -17,7 +17,12 @@ import { ValidationError } from '../middleware/errorHandler.js'; import { buildExclusionClauses } from '../lib/filterBuilder.js'; // Sum converted rows into a sorted per-day net series (SIMP-50). +/** + * @param {Array>} rows Converted rows with `date` + `amount_eur`. + * @returns {Array<{ date: string, net: number }>} + */ function aggregateByDate(rows) { + /** @type {Map} */ const map = new Map(); for (const r of rows) { const iso = r.date instanceof Date ? formatDateToYmd(r.date) : String(r.date).slice(0, 10); @@ -28,9 +33,14 @@ function aggregateByDate(rows) { // Average, across months, of the running cumulative day-of-month net (SIMP-50). // `monthDayNet` is { monthKey: { dayOfMonth: net } }. +/** + * @param {Record>} monthDayNet + * @returns {Record} day-of-month β†’ average cumulative net + */ function computeAvgCumulativeByDay(monthDayNet) { const monthKeys = Object.keys(monthDayNet); const monthCount = monthKeys.length || 1; + /** @type {Record} */ const out = {}; for (const mk of monthKeys) { const dayNet = monthDayNet[mk]; @@ -46,6 +56,11 @@ function computeAvgCumulativeByDay(monthDayNet) { return out; } +/** + * @param {number[]} [excludedCategoryIds] + * @param {number[]} [excludedRecipientIds] + * @param {string} [targetCurrency] + */ export async function getCashflowComparison( excludedCategoryIds = [], excludedRecipientIds = [], @@ -142,6 +157,7 @@ export async function getCashflowComparison( 'date' ); + /** @type {Record>} */ const monthDayNet = {}; for (const row of pastConverted) { const eur = row.amount_eur; @@ -152,23 +168,27 @@ export async function getCashflowComparison( const avgCumulativeByDay = computeAvgCumulativeByDay(monthDayNet); + /** @type {Record} */ const currentDayNet = {}; for (const row of currentCashflowConverted) { currentDayNet[row.day_of_month] = (currentDayNet[row.day_of_month] || 0) + row.amount_eur; } let currentCum = 0; + /** @type {Record} */ const currentByDay = {}; for (let d = 1; d <= currentDay; d++) { currentCum += (currentDayNet[d] || 0); currentByDay[d] = currentCum; } + /** @type {Record} */ const plannedCurrentByDay = {}; for (const row of plannedCurrentConverted) { plannedCurrentByDay[row.day_of_month] = (plannedCurrentByDay[row.day_of_month] || 0) + row.amount_eur; } + /** @type {Record>} */ const plannedHistMonthDay = {}; for (const row of plannedHistConverted) { const mk = row.month_key; @@ -213,6 +233,12 @@ export async function getCashflowComparison( }; } +/** + * @param {number} historyMonths + * @param {number[]} [excludedCategoryIds] + * @param {number[]} [excludedRecipientIds] + * @param {string} [targetCurrency] + */ export async function getCashflowForecastData( historyMonths, excludedCategoryIds = [], @@ -299,6 +325,14 @@ export async function getCashflowForecastData( }; } +/** + * @param {number} historyMonths + * @param {number} daysBack + * @param {number} daysForward + * @param {number[]} [excludedCategoryIds] + * @param {number[]} [excludedRecipientIds] + * @param {string} [targetCurrency] + */ export async function getCashflowForecastDataRolling( historyMonths, daysBack, @@ -380,6 +414,12 @@ export async function getCashflowForecastDataRolling( }; } +/** + * @param {number} historyMonths + * @param {number[]} [excludedCategoryIds] + * @param {number[]} [excludedRecipientIds] + * @param {string} [targetCurrency] + */ export async function getCashflowForecastDataByCategory( historyMonths, excludedCategoryIds = [], @@ -452,7 +492,12 @@ export async function getCashflowForecastDataByCategory( 'date', ); + /** + * @param {Array>} rows + * @returns {Array<{ date: string, category_id: number|null, general: string, detail: string, net: number }>} + */ const aggregateByDateAndCategory = (rows) => { + /** @type {Map} */ const map = new Map(); for (const r of rows) { const date = r.date instanceof Date ? formatDateToYmd(r.date) : String(r.date).slice(0, 10); diff --git a/apps/node-backend/src/repositories/infoRepositoryHelpers.js b/apps/node-backend/src/repositories/infoRepositoryHelpers.js index f813c35f..cd5a3331 100644 --- a/apps/node-backend/src/repositories/infoRepositoryHelpers.js +++ b/apps/node-backend/src/repositories/infoRepositoryHelpers.js @@ -48,6 +48,8 @@ const ALLOWED_MV_NAMES = new Set([ * Positive results cached in-process indefinitely; negative results cached * for {@link MV_NEGATIVE_CACHE_TTL_MS} so we recover quickly after MV creation. * + * @param {string} viewName + * @returns {Promise} * @throws {Error} if {@code viewName} is not in the allowlist. */ export async function mvAvailable(viewName) { @@ -88,6 +90,10 @@ export function clearMvCache() { // ── Numeric helpers ──────────────────────────────────────────────────────── +/** + * @param {number|string|import('decimal.js').Decimal} value + * @returns {number} + */ export function roundToCents(value) { return roundMoney(value); } @@ -98,16 +104,30 @@ export function roundToCents(value) { // working while routes import the canonical helper from lib directly. export { formatDateToYmd, toWireDate }; +/** + * @param {number|string} year + * @param {number|string} month 1-based. + * @returns {string} 'YYYY-MM' + */ export function formatYearMonthKey(year, month) { return `${year}-${String(month).padStart(2, '0')}`; } +/** + * @param {Date} date + * @param {number} [days] + * @returns {Date} + */ export function addDaysUtc(date, days = 1) { const next = new Date(date); next.setUTCDate(next.getUTCDate() + days); return next; } +/** + * @param {Date} date + * @returns {string} 'YYYY-MM-DD' (UTC) + */ export function getDayKeyUtc(date) { const yyyy = date.getUTCFullYear(); const mm = String(date.getUTCMonth() + 1).padStart(2, '0'); @@ -115,6 +135,10 @@ export function getDayKeyUtc(date) { return `${yyyy}-${mm}-${dd}`; } +/** + * @param {string|Date} value + * @returns {string} 'YYYY-MM' + */ export function extractYearMonth(value) { return String(value).substring(0, 7); } @@ -127,8 +151,13 @@ export function extractYearMonth(value) { * summing absolute EUR per (period, entity), rounding totals to cents, and * sorting each period ascending by total. Shared by the recipient and tag * period-pivots (SIMP-49). + * + * @param {Array>} convertedRows + * @param {{ idField: string, labelField: string, idKey: string, labelKey: string }} shape + * @returns {Record>>} */ export function buildPeriodPivot(convertedRows, { idField, labelField, idKey, labelKey }) { + /** @type {Record>>} */ const periodMap = {}; for (const row of convertedRows) { const period = row.period; @@ -144,6 +173,7 @@ export function buildPeriodPivot(convertedRows, { idField, labelField, idKey, la periodMap[period][id].transactionCount += cnt; } + /** @type {Record>>} */ const pivot = {}; for (const [period, entities] of Object.entries(periodMap)) { pivot[period] = Object.values(entities) @@ -153,6 +183,12 @@ export function buildPeriodPivot(convertedRows, { idField, labelField, idKey, la return pivot; } +/** + * @param {Array<{ + * total_spending: number, total_income: number, net_amount: number, + * transaction_count: number, period_start?: string|null, period_end?: string|null, + * }>} months + */ export function buildMonthlySummary(months) { return { total_spending: toNumber(months.reduce((sum, m) => sum.plus(toDecimal(m.total_spending)), toDecimal(0))), @@ -165,6 +201,12 @@ export function buildMonthlySummary(months) { }; } +/** + * @param {Array>} rows + * @param {string} [amountField] + * @param {boolean} [fallbackToZero] + * @returns {Array>} rows with a numeric `amount` merged in + */ export function mapRowsForAmountConversion(rows, amountField = 'amount', fallbackToZero = true) { return rows.map(row => ({ ...row, @@ -176,15 +218,31 @@ export function mapRowsForAmountConversion(rows, amountField = 'amount', fallbac // ── Category helpers ─────────────────────────────────────────────────────── +/** + * @param {number|string} categoryId `-1` is the "uncategorised" sentinel. + * @returns {string} + */ export function getCategoryKey(categoryId) { return categoryId === -1 ? 'null' : String(categoryId); } +/** + * @param {any} categoryId A number or numeric string; `-1` is the + * "uncategorised" sentinel. (`any` because the value is fed to `parseInt`, + * whose declared parameter is `string` β€” the runtime coercion of a number is + * deliberate here.) + * @returns {number|null} + */ export function parseCategoryId(categoryId) { return categoryId === -1 ? null : parseInt(categoryId, 10); } +/** + * @param {Array>} convertedRows + * @returns {Array<{ id: number|null, name: string, count: number, total: number }>} + */ export function buildCategoryFromConvertedRows(convertedRows) { + /** @type {Map} */ const categoryMap = new Map(); for (const row of convertedRows) { @@ -212,6 +270,12 @@ export function buildCategoryFromConvertedRows(convertedRows) { // ── Currency conversion helpers ──────────────────────────────────────────── +/** + * @param {Array>} rows Rows with `amount` + `currency`. + * @param {string} targetCurrency + * @param {string} [dateField] Date field used for the historical rate lookup. + * @returns {Promise>>} rows with `amount_eur` merged in + */ export async function convertRowsWithHistoricalRateFallback(rows, targetCurrency, dateField = 'date') { try { return await convertRowsToEur(rows, targetCurrency, { useHistoricalRatesByDate: true, dateField }); @@ -228,10 +292,10 @@ export async function convertRowsWithHistoricalRateFallback(rows, targetCurrency * * The `_batchGroup` tag is stripped from all returned rows. * - * @param {Array>} groups - Each group has rows with `amount` + `currency` + * @param {Array>>} groups - Each group has rows with `amount` + `currency` * @param {string} targetCurrency * @param {string} [dateField='date'] - Date field used for historical rate lookup - * @returns {Promise>>} Converted groups in the same order as input + * @returns {Promise>>>} Converted groups in the same order as input */ export async function batchConvertGroupsWithHistoricalRateFallback(groups, targetCurrency, dateField = 'date') { const TAG = '_batchGroup'; @@ -255,6 +319,10 @@ export async function batchConvertGroupsWithHistoricalRateFallback(groups, targe // ── Investment spike sanitizer ───────────────────────────────────────────── +/** + * @param {Array<{ date: string, liquid: number, liabilities: number, investments: number, netWorth: number }>} snapshots + * @returns {Array<{ date: string, liquid: number, liabilities: number, investments: number, netWorth: number }>} + */ export function sanitizeIsolatedDailyInvestmentSpikes(snapshots) { if (!Array.isArray(snapshots) || snapshots.length < 3) { return Array.isArray(snapshots) ? snapshots : []; @@ -289,8 +357,12 @@ export function sanitizeIsolatedDailyInvestmentSpikes(snapshots) { if ((oppositeDirections && largeMove && bridgeLooksNormal) || localNeedlePeak || localNeedleTrough) { const correctedInvestments = Math.sqrt(prev * next); const liquid = Number(sanitized[i]?.liquid) || 0; + // Liabilities are stored as negative balances (ADR-092); the canonical + // formula everywhere else is netWorth = liquid + liabilities + investments, + // so the corrected point must include the liabilities term too. + const liabilities = Number(sanitized[i]?.liabilities) || 0; sanitized[i].investments = roundToCents(correctedInvestments); - sanitized[i].netWorth = roundToCents(liquid + correctedInvestments); + sanitized[i].netWorth = roundToCents(liquid + liabilities + correctedInvestments); } } diff --git a/apps/node-backend/src/repositories/infoRepositoryMonthly.js b/apps/node-backend/src/repositories/infoRepositoryMonthly.js index 42474754..df2bfa91 100644 --- a/apps/node-backend/src/repositories/infoRepositoryMonthly.js +++ b/apps/node-backend/src/repositories/infoRepositoryMonthly.js @@ -20,6 +20,12 @@ import { getIncludeTransfers, } from './infoRepositoryHelpers.js'; +/** + * @param {number[]} [excludedCategoryIds] + * @param {string} [targetCurrency] + * @param {number[]} [excludedRecipientIds] + * @param {boolean} [allTime] + */ export async function getMonthlyFinancialSummary( excludedCategoryIds = [], targetCurrency = 'EUR', @@ -75,6 +81,14 @@ export async function getMonthlyFinancialSummary( } const mergedConverted = await convertRowsToEur(mergedRows, targetCurrency, { useHistoricalRatesByDate: true, dateField: 'date' }); + /** + * @type {Record} + */ const monthMap = {}; for (const conv of mergedConverted) { const key = conv._key; @@ -204,7 +218,13 @@ export async function getMonthlyFinancialSummary( const result = await query(sql, params); logger.debug('Monthly summary query returned', { rowCount: result.rows.length }); - const dailyRows = result.rows.filter(r => r.date != null); + const dailyRows = result.rows.filter( + (/** @type {{ + month: number, year: number, period_start: Date, period_end: Date, + date: Date|null, currency: string|null, cnt: string|null, + income_amount: string|null, spending_amount: string|null, + }} */ r) => r.date != null, + ); // Convert each (date, currency) income/spending aggregate at that date's rate. const [incomeConverted, spendingConverted] = await Promise.all([ convertRowsToEur( @@ -219,6 +239,14 @@ export async function getMonthlyFinancialSummary( ), ]); + /** + * @type {Record} + */ const monthMap = {}; for (const row of result.rows) { const key = formatYearMonthKey(row.year, row.month); diff --git a/apps/node-backend/src/repositories/infoRepositoryNetWorth.js b/apps/node-backend/src/repositories/infoRepositoryNetWorth.js index 09b1a89e..8f37b738 100644 --- a/apps/node-backend/src/repositories/infoRepositoryNetWorth.js +++ b/apps/node-backend/src/repositories/infoRepositoryNetWorth.js @@ -76,6 +76,7 @@ export const netWorthRepository = { ORDER BY snapshot_date ASC `, [targetCurrency]); + /** @type {Record} */ const investmentsByDay = {}; for (const row of snapshotResult.rows) { investmentsByDay[row.day] = Number(row.investments) || 0; @@ -171,7 +172,10 @@ export const netWorthRepository = { ), convertRowsWithHistoricalRateFallback( mapRowsForAmountConversion( - currentBalancesResult.rows.map((r) => ({ ...r, day: todayYmd })), + currentBalancesResult.rows.map( + (/** @type {{ bank_account: string, is_liability: boolean, balance: string, currency: string }} */ r) => + ({ ...r, day: todayYmd }), + ), 'balance' ), targetCurrency, @@ -246,7 +250,9 @@ export const netWorthRepository = { // Split each in-net-worth account's daily balance into liquid assets vs // liabilities (ADR-092): debt balances are negative and must not drag the // "liquid assets" headline negative. netWorth = liquid + liabilities + investments. + /** @type {Record} */ const liquidByDay = {}; + /** @type {Record} */ const liabilitiesByDay = {}; for (const row of bankHistoryConverted) { const bucket = row.is_liability ? liabilitiesByDay : liquidByDay; @@ -322,7 +328,8 @@ export const netWorthRepository = { last.netWorth = roundToCents(last.liquid + (last.liabilities || 0) + investments); } - const latest = sanitizedSnapshots[sanitizedSnapshots.length - 1] || { liquid: 0, liabilities: 0, investments: 0, netWorth: 0 }; + const latest = sanitizedSnapshots[sanitizedSnapshots.length - 1] + || /** @type {{ date?: string, liquid: number, liabilities: number, investments: number, netWorth: number }} */ ({ liquid: 0, liabilities: 0, investments: 0, netWorth: 0 }); const currentMonthPrefix = latest.date ? extractYearMonth(latest.date) : null; const firstCurrentMonthIdx = currentMonthPrefix ? sanitizedSnapshots.findIndex(s => s.date.startsWith(currentMonthPrefix)) diff --git a/apps/node-backend/src/repositories/infoRepositoryPlanned.js b/apps/node-backend/src/repositories/infoRepositoryPlanned.js index f8387407..008a0ff9 100644 --- a/apps/node-backend/src/repositories/infoRepositoryPlanned.js +++ b/apps/node-backend/src/repositories/infoRepositoryPlanned.js @@ -20,6 +20,9 @@ const MS_PER_DAY = 24 * 60 * 60 * 1000; /** * Fixed day-step for the day-based recurrence patterns, or null for the * month-based ones (monthly/quarterly/yearly), whose step length varies. + * + * @param {string|null|undefined} pattern + * @returns {number|null} */ function dayStepForPattern(pattern) { const p = String(pattern || '').toLowerCase().trim(); @@ -48,8 +51,15 @@ function dayStepForPattern(pattern) { * Month-based patterns keep the plain walk (120 monthly hops = 10 years, far * beyond any realistic staleness, and bulk month jumps would change the * sequential month-end clamping semantics). + * + * @param {Date|string} plannedDate DATE column β€” a `Date` from pg. + * @param {string} pattern + * @param {string} startYmd 'YYYY-MM-DD' (inclusive) + * @param {string} endYmd 'YYYY-MM-DD' (exclusive) + * @returns {string[]} occurrence days as 'YYYY-MM-DD' */ function expandRecurringOccurrences(plannedDate, pattern, startYmd, endYmd) { + /** @type {string[]} */ const ymds = []; if (!pattern) return ymds; let current = plannedDate instanceof Date ? new Date(plannedDate.getTime()) : new Date(plannedDate); @@ -121,10 +131,35 @@ export const plannedRepository = { const startYmd = formatDateToYmd(nextMonth); const endYmd = formatDateToYmd(monthAfter); + /** + * `total_income` / `total_expenses` are deliberately `any`: they hold + * Decimals during the accumulation loop and are collapsed IN PLACE to + * numbers after it (see below) β€” a union type would reject one of the two + * phases without a runtime change. + * + * @type {Record, + * }>} + */ const dailyMap = {}; let occurrenceCount = 0; // Day totals accumulate as Decimals (monetary-arithmetic rule) and are // collapsed to numbers once, after the loop. + /** + * @param {string} dateStr 'YYYY-MM-DD' + * @param {any} row Converted planned row (`PlannedForecastRow`-ish + `amount_eur`). + * @param {number} eur + */ const pushOccurrence = (dateStr, row, eur) => { if (!dailyMap[dateStr]) { dailyMap[dateStr] = { date: dateStr, total_income: toDecimal(0), total_expenses: toDecimal(0), transactions: [] }; diff --git a/apps/node-backend/src/repositories/infoRepositoryRecipients.js b/apps/node-backend/src/repositories/infoRepositoryRecipients.js index 8fb7cd46..108ccbf3 100644 --- a/apps/node-backend/src/repositories/infoRepositoryRecipients.js +++ b/apps/node-backend/src/repositories/infoRepositoryRecipients.js @@ -27,6 +27,9 @@ export const recipientInsightsRepository = { * - top merchants by total spend (full list; frontend slices for chart/KPIs) * - spending frequency & average per recipient * - month-over-month comparison alerts ("You spent X% more at …") + * + * @param {string} [targetCurrency] + * @param {{ excludedCategoryIds?: number[], excludedRecipientIds?: number[] }} [opts] */ async getRecipientInsights(targetCurrency = 'EUR', { excludedCategoryIds = [], excludedRecipientIds = [] } = {}) { // Canonical exclusion semantics (lib/filterBuilder.buildExclusionClauses, @@ -63,6 +66,13 @@ export const recipientInsightsRepository = { targetCurrency ); + /** + * @type {Record} + */ const recipientAgg = {}; for (const row of topConverted) { const rid = row.recipient_id; @@ -141,6 +151,7 @@ export const recipientInsightsRepository = { const currentPeriod = periodResult.rows[0].current_period; const prevPeriod = periodResult.rows[0].prev_period; + /** @type {Record} */ const momAgg = {}; for (const row of momConverted) { const rid = row.recipient_id; @@ -165,6 +176,11 @@ export const recipientInsightsRepository = { return { topMerchants, monthOverMonth }; }, + /** + * @param {string} [targetCurrency] + * @param {number[]} [excludedRecipientIds] + * @param {number[]} [excludedCategoryIds] + */ async getRecipientByYear(targetCurrency = 'EUR', excludedRecipientIds = [], excludedCategoryIds = []) { // Canonical exclusion clauses (lib/filterBuilder.buildExclusionClauses). // Category exclusion (incl. hidden categories) must apply here too, or the @@ -204,6 +220,12 @@ export const recipientInsightsRepository = { { useHistoricalRatesByDate: true, dateField: 'date' } ); + /** + * @type {Record>} + */ const yearRecMap = {}; for (const row of converted) { const year = String(row.year); @@ -219,6 +241,12 @@ export const recipientInsightsRepository = { yearRecMap[year][rid].transactionCount += cnt; } + /** + * @type {Record>} + */ const recipientsByYear = {}; for (const [year, recs] of Object.entries(yearRecMap)) { recipientsByYear[year] = Object.values(recs) @@ -230,6 +258,16 @@ export const recipientInsightsRepository = { return { recipientsByYear }; }, + /** + * @param {number[]} [excludedRecipientIds] + * @param {string} [targetCurrency] + * @param {{ + * bucket?: string, + * startDate?: string|null, + * endDate?: string|null, + * recipientIds?: number[]|null, + * }} [opts] + */ async getRecipientPivot(excludedRecipientIds = [], targetCurrency = 'EUR', { bucket = 'monthly', startDate = null, endDate = null, recipientIds = null } = {}) { // Canonical recipient exclusion (lib/filterBuilder.buildExclusionClauses). const excl = buildExclusionClauses({ excludedRecipientIds }); @@ -256,7 +294,7 @@ export const recipientInsightsRepository = { `SELECT id FROM recipients WHERE id = ANY($1::int[]) OR primary_recipient_id = ANY($1::int[])`, [validIncludeIds], ); - const memberIds = memberRes.rows.map((row) => Number(row.id)); + const memberIds = memberRes.rows.map((/** @type {{ id: number }} */ row) => Number(row.id)); if (memberIds.length === 0) return { recipientPivot: {} }; params.push(memberIds); recipientInclude = `AND t.recipient_id = ANY($${params.length}::int[])`; diff --git a/apps/node-backend/src/repositories/infoRepositoryStatistics.js b/apps/node-backend/src/repositories/infoRepositoryStatistics.js index b68085fa..fa6aa319 100644 --- a/apps/node-backend/src/repositories/infoRepositoryStatistics.js +++ b/apps/node-backend/src/repositories/infoRepositoryStatistics.js @@ -38,6 +38,11 @@ export const statisticsRepository = { // (category, currency) converted once equals Ξ£ of the converted per-row // amounts (rate is linear and sign-preserving): numerically identical to // the old per-row loop, minus the row-cardinality transfer to Node. + // + // Effective category is the canonical 3-level resolution (own β†’ recipient + // default β†’ PRIMARY recipient's default), matching transactionRepository β€” + // an alias row categorised via its primary must not show as UNCATEGORISED + // here while the transactions list shows it categorised. const categoryAmountResult = await query(` SELECT COALESCE(c.id, -1) AS category_id, COALESCE(c.general || ':' || c.detail, 'UNCATEGORISED') AS name, @@ -46,7 +51,8 @@ export const statisticsRepository = { t.currency FROM transactions t LEFT JOIN recipients r ON t.recipient_id = r.id - LEFT JOIN categories c ON COALESCE(t.category_id, r.default_category_id) = c.id + LEFT JOIN recipients pr ON r.primary_recipient_id = pr.id + LEFT JOIN categories c ON COALESCE(t.category_id, r.default_category_id, pr.default_category_id) = c.id WHERE t.is_active = true ${includeTransfers ? '' : 'AND t.is_transfer = false'} GROUP BY COALESCE(c.id, -1), @@ -59,6 +65,7 @@ export const statisticsRepository = { targetCurrency ); + /** @type {Record} */ const catMap = {}; for (const row of catConverted) { const catId = row.category_id === -1 ? null : parseInt(row.category_id, 10); @@ -90,7 +97,7 @@ export const statisticsRepository = { ORDER BY a.name`, [] ); - return result.rows.map(r => r.bank_account); + return result.rows.map((/** @type {{ bank_account: string }} */ r) => r.bank_account); }, /** @@ -113,6 +120,11 @@ export const statisticsRepository = { return parseInt(result.rows[0].count, 10); }, + /** + * @param {number[]} [excludedCategoryIds] + * @param {string} [targetCurrency] + * @param {number[]} [excludedRecipientIds] + */ async getCategoryPivot(excludedCategoryIds = [], targetCurrency = 'EUR', excludedRecipientIds = []) { const includeTransfers = await getIncludeTransfers(); // Canonical exclusion clauses (lib/filterBuilder.buildExclusionClauses, @@ -132,9 +144,13 @@ export const statisticsRepository = { // the converted per-transaction amounts β€” numerically identical to the old // per-row loop. The sign-split also gives explicit income/expense per cell // so consumers no longer have to classify by the sign of the net total. + // Effective category is the canonical 3-level resolution (own β†’ + // recipient default β†’ PRIMARY recipient's default), matching + // transactionRepository β€” the same expression appears in the SELECT, the + // NOT NULL filter and the GROUP BY, and all three must stay identical. const sql = ` SELECT - COALESCE(t.category_id, r.default_category_id) AS category_id, + COALESCE(t.category_id, r.default_category_id, pr.default_category_id) AS category_id, CONCAT(c.general, ': ', c.detail) AS category_name, TO_CHAR(t.date, 'YYYY-MM') AS period, t.date, t.currency, @@ -144,12 +160,12 @@ export const statisticsRepository = { FROM transactions t LEFT JOIN recipients r ON t.recipient_id = r.id LEFT JOIN recipients pr ON r.primary_recipient_id = pr.id - LEFT JOIN categories c ON COALESCE(t.category_id, r.default_category_id) = c.id + LEFT JOIN categories c ON COALESCE(t.category_id, r.default_category_id, pr.default_category_id) = c.id WHERE t.is_active = true ${includeTransfers ? '' : 'AND t.is_transfer = false'} - AND COALESCE(t.category_id, r.default_category_id) IS NOT NULL + AND COALESCE(t.category_id, r.default_category_id, pr.default_category_id) IS NOT NULL ${exclusionWhere} - GROUP BY COALESCE(t.category_id, r.default_category_id), CONCAT(c.general, ': ', c.detail), TO_CHAR(t.date, 'YYYY-MM'), t.date, t.currency + GROUP BY COALESCE(t.category_id, r.default_category_id, pr.default_category_id), CONCAT(c.general, ': ', c.detail), TO_CHAR(t.date, 'YYYY-MM'), t.date, t.currency ORDER BY period `; @@ -170,6 +186,12 @@ export const statisticsRepository = { { useHistoricalRatesByDate: true, dateField: 'date' } ); + /** + * @type {Record>} + */ const periodCatMap = {}; for (const row of converted) { const period = row.period; @@ -192,6 +214,12 @@ export const statisticsRepository = { } } + /** + * @type {Record>} + */ const categoryPivot = {}; for (const [period, cats] of Object.entries(periodCatMap)) { categoryPivot[period] = Object.values(cats) diff --git a/apps/node-backend/src/repositories/instrumentProviderMapRepository.js b/apps/node-backend/src/repositories/instrumentProviderMapRepository.js index 72ae36b8..584b9ba5 100644 --- a/apps/node-backend/src/repositories/instrumentProviderMapRepository.js +++ b/apps/node-backend/src/repositories/instrumentProviderMapRepository.js @@ -9,6 +9,8 @@ import { query } from '../database/connection.js'; +/** @typedef {import('../types/rows.js').InstrumentProviderMapRow} InstrumentProviderMapRow */ + const COLUMNS = `id, instrument_key, key_type, provider, provider_symbol, resolved_name, exchange, currency, status, verified_at, created_at, updated_at`; @@ -17,7 +19,7 @@ const COLUMNS = `id, instrument_key, key_type, provider, provider_symbol, * All mappings for an instrument, ordered by provider. * @param {string} instrumentKey * @param {string} keyType 'isin' | 'internal' - * @returns {Promise} + * @returns {Promise} */ export async function listByInstrument(instrumentKey, keyType) { const result = await query( @@ -33,8 +35,17 @@ export async function listByInstrument(instrumentKey, keyType) { /** * Upsert one mapping. Conflict target is the (instrument_key, key_type, provider) * unique index; an existing row's symbol/name/exchange/currency/status are replaced. - * @param {object} m - * @returns {Promise} the upserted row + * @param {{ + * instrumentKey: string, + * keyType: string, + * provider: string, + * providerSymbol?: string|null, + * resolvedName?: string|null, + * exchange?: string|null, + * currency?: string|null, + * status?: string|null, + * }} m + * @returns {Promise} the upserted row */ export async function upsert(m) { const result = await query( diff --git a/apps/node-backend/src/repositories/investmentRepository.js b/apps/node-backend/src/repositories/investmentRepository.js index 1146d3d1..fbead255 100644 --- a/apps/node-backend/src/repositories/investmentRepository.js +++ b/apps/node-backend/src/repositories/investmentRepository.js @@ -568,6 +568,12 @@ export const investmentRepository = { } name = trimmedName; symbol = normalizeSymbol(symbol); + // Same uniqueness rule as update() (there is no DB unique index on symbol, + // so create was the one path that could still insert a duplicate β€” e.g. a + // duplicate-on-retry). excludeId 0 matches no row: ids start at 1. + if (typeof symbol === 'string' && symbol !== '') { + await ensureSymbolIsUnique(symbol, 0); + } const payload = { name, symbol, diff --git a/apps/node-backend/src/repositories/recipientBankAccountRepository.js b/apps/node-backend/src/repositories/recipientBankAccountRepository.js index dc6fd517..00c15199 100644 --- a/apps/node-backend/src/repositories/recipientBankAccountRepository.js +++ b/apps/node-backend/src/repositories/recipientBankAccountRepository.js @@ -8,7 +8,13 @@ import { query, withTransaction } from '../database/connection.js'; import { buildSetClauses } from '../lib/sqlClauses.js'; +/** @typedef {import('../types/rows.js').RecipientBankAccountRow} RecipientBankAccountRow */ + export const recipientBankAccountRepository = { + /** + * @param {number} id + * @returns {Promise} + */ async getById(id) { const result = await query( `SELECT * FROM recipient_bank_accounts WHERE id = $1`, @@ -17,6 +23,10 @@ export const recipientBankAccountRepository = { return result.rows[0] || null; }, + /** + * @param {string|null|undefined} accountNumber + * @returns {Promise} + */ async getByAccountNumber(accountNumber) { if (!accountNumber) return null; const result = await query( @@ -26,6 +36,11 @@ export const recipientBankAccountRepository = { return result.rows[0] || null; }, + /** + * @param {number} recipientId + * @param {boolean} [activeOnly] + * @returns {Promise} + */ async getByRecipientId(recipientId, activeOnly = true) { let sql = ` SELECT * FROM recipient_bank_accounts @@ -38,6 +53,10 @@ export const recipientBankAccountRepository = { return result.rows; }, + /** + * @param {number} recipientId + * @returns {Promise} + */ async getPrimaryAccount(recipientId) { const result = await query( `SELECT * FROM recipient_bank_accounts @@ -50,6 +69,15 @@ export const recipientBankAccountRepository = { /** * Create or get a bank account, enriching existing accounts with missing metadata. + * @param {{ + * recipientId: number, + * accountNumber: string, + * bankName?: string|null, + * address?: string|null, + * accountLabel?: string|null, + * setAsPrimary?: boolean, + * }} input + * @returns {Promise<{ bankAccount: RecipientBankAccountRow|null, created: boolean }>} */ async createOrGet({ recipientId, accountNumber, bankName = null, address = null, accountLabel = null, setAsPrimary = false }) { if (!accountNumber) throw new Error('Account number is required'); @@ -110,6 +138,11 @@ export const recipientBankAccountRepository = { return { bankAccount: created, created: true }; }, + /** + * @param {number} id + * @param {{ bankName?: string|null, address?: string|null, accountLabel?: string|null }} patch + * @returns {Promise} + */ async update(id, { bankName, address, accountLabel }) { // Shared clause builder (lib/sqlClauses.js): undefined fields are skipped. const { clauses: updates, params, nextIdx: paramIdx } = buildSetClauses({ @@ -125,6 +158,10 @@ export const recipientBankAccountRepository = { return result.rows[0] || null; }, + /** + * @param {number} id + * @returns {Promise} true if a row was deactivated + */ async softDelete(id) { const result = await query( `UPDATE recipient_bank_accounts SET is_active = false, updated_at = NOW() WHERE id = $1 RETURNING id`, @@ -138,6 +175,9 @@ export const recipientBankAccountRepository = { * primary already owns on uq_rba_account_number (migration 0029). The * primary's row is kept and the alias's deleted, rather than reassigned. * Must run BEFORE repointRecipient(). + * @param {number} primaryId + * @param {number[]} aliasIds + * @returns {Promise} rows deleted */ async deleteMergeDuplicates(primaryId, aliasIds) { const result = await query( @@ -151,7 +191,12 @@ export const recipientBankAccountRepository = { return result.rowCount ?? 0; }, - /** Repoint surviving alias bank accounts onto the merge primary. */ + /** + * Repoint surviving alias bank accounts onto the merge primary. + * @param {number} primaryId + * @param {number[]} aliasIds + * @returns {Promise} rows repointed + */ async repointRecipient(primaryId, aliasIds) { const result = await query( `UPDATE recipient_bank_accounts @@ -162,6 +207,11 @@ export const recipientBankAccountRepository = { return result.rowCount ?? 0; }, + /** + * @param {number} bankAccountId + * @param {number} recipientId + * @returns {Promise} true if the account was made primary + */ async setPrimary(bankAccountId, recipientId) { return withTransaction(async (client) => { await client.query( diff --git a/apps/node-backend/src/repositories/savedChartsRepository.js b/apps/node-backend/src/repositories/savedChartsRepository.js index e04d5924..6be96ff5 100644 --- a/apps/node-backend/src/repositories/savedChartsRepository.js +++ b/apps/node-backend/src/repositories/savedChartsRepository.js @@ -1,10 +1,35 @@ import { query } from '../database/connection.js'; import { buildSetClauses, buildLimitOffset } from '../lib/sqlClauses.js'; +/** @typedef {import('../types/rows.js').SavedChartRow} SavedChartRow */ + +/** + * The camelCase field bag the routes pass for create/update. All fields are + * optional on update; create requires the NOT NULL ones. + * + * @typedef {object} SavedChartInput + * @property {string} [name] + * @property {string} [chartType] + * @property {number[]} [categoryIds] + * @property {number[]} [recipientIds] + * @property {number[]} [tagIds] + * @property {boolean} [allCategories] + * @property {boolean} [allRecipients] + * @property {boolean} [allTags] + * @property {string} [chartVariant] + * @property {string} [timeBucket] + * @property {string|null} [dateRangeStart] 'YYYY-MM-DD' + * @property {string|null} [dateRangeEnd] 'YYYY-MM-DD' + */ + // date_range_* are DATE columns β€” emitted via to_char so the wire carries the // calendar day, not a pg Date that JSON-serializes to the previous day east of UTC. const COLUMNS = "id, name, chart_type, category_ids, recipient_ids, tag_ids, all_categories, all_recipients, all_tags, chart_variant, time_bucket, to_char(date_range_start, 'YYYY-MM-DD') AS date_range_start, to_char(date_range_end, 'YYYY-MM-DD') AS date_range_end, created_at, updated_at"; +/** + * @param {SavedChartRow} r + * @returns {SavedChartRow} + */ function mapRow(r) { return { ...r, @@ -24,8 +49,10 @@ const savedChartsRepository = { * narrows the result. * * @param {{ limit?: number|null, offset?: number }} [page] + * @returns {Promise} */ async getAll({ limit = null, offset = 0 } = {}) { + /** @type {any[]} */ const params = []; const sql = `SELECT ${COLUMNS} FROM saved_charts ORDER BY created_at ASC` + buildLimitOffset(params, { limit, offset }); @@ -33,18 +60,29 @@ const savedChartsRepository = { return result.rows.map(mapRow); }, - /** Row count β€” the `total` for a paginated list. */ + /** + * Row count β€” the `total` for a paginated list. + * @returns {Promise} + */ async getCount() { const result = await query('SELECT COUNT(*) FROM saved_charts'); return parseInt(result.rows[0].count, 10); }, + /** + * @param {number} id + * @returns {Promise} + */ async getById(id) { const result = await query(`SELECT ${COLUMNS} FROM saved_charts WHERE id = $1`, [id]); const r = result.rows[0]; return r ? mapRow(r) : null; }, + /** + * @param {SavedChartInput} input + * @returns {Promise} + */ async create({ name, chartType, categoryIds, recipientIds, tagIds, allCategories, allRecipients, allTags, chartVariant, timeBucket, dateRangeStart, dateRangeEnd }) { const result = await query( `INSERT INTO saved_charts (name, chart_type, category_ids, recipient_ids, tag_ids, all_categories, all_recipients, all_tags, chart_variant, time_bucket, date_range_start, date_range_end) @@ -55,6 +93,11 @@ const savedChartsRepository = { return mapRow(result.rows[0]); }, + /** + * @param {number} id + * @param {SavedChartInput} patch + * @returns {Promise} + */ async update(id, { name, chartType, categoryIds, recipientIds, tagIds, allCategories, allRecipients, allTags, chartVariant, timeBucket, dateRangeStart, dateRangeEnd }) { // Shared clause builder (lib/sqlClauses.js): undefined fields are skipped, // mapColumn translates the camelCase API bag to the snake_case columns. @@ -73,6 +116,10 @@ const savedChartsRepository = { return result.rows[0] ? mapRow(result.rows[0]) : null; }, + /** + * @param {number} id + * @returns {Promise} true if a row was removed + */ async delete(id) { const result = await query(`DELETE FROM saved_charts WHERE id = $1 RETURNING id`, [id]); return result.rows.length > 0; diff --git a/apps/node-backend/src/repositories/settingsRepository.js b/apps/node-backend/src/repositories/settingsRepository.js index d2122d76..0f2bf1ae 100644 --- a/apps/node-backend/src/repositories/settingsRepository.js +++ b/apps/node-backend/src/repositories/settingsRepository.js @@ -19,12 +19,20 @@ const STRING_VALUED_KEYS = new Set(['cost_basis_method']); * Legacy rows (and some restore paths) stored the JSON of the value inside a * jsonb string β€” e.g. jsonb `"true"` for the boolean true. Self-heal those on * read by parsing, but only for keys that are not string-valued by contract. + * + * @param {string} key + * @param {any} value Parsed JSONB value from pg. + * @returns {any} */ function reviveLegacyJsonString(key, value) { if (typeof value !== 'string' || STRING_VALUED_KEYS.has(key)) return value; try { return JSON.parse(value); } catch { return value; } } +/** + * @param {any} value Arbitrary JSON-serialisable setting value. + * @returns {any} + */ function normalizeSettingValue(value) { let normalizedValue = value; @@ -48,6 +56,8 @@ function normalizeSettingValue(value) { export const settingsRepository = { /** * Get a setting by key. Returns null if not found. + * @param {string} key + * @returns {Promise} Parsed JSONB value, or null. */ async get(key) { const result = await query('SELECT value FROM user_settings WHERE key = $1', [key]); @@ -57,9 +67,11 @@ export const settingsRepository = { /** * Get all settings as a keyβ†’value map. + * @returns {Promise>} */ async getAll() { const result = await query('SELECT key, value FROM user_settings ORDER BY key'); + /** @type {Record} */ const settings = {}; for (const row of result.rows) { settings[row.key] = reviveLegacyJsonString(row.key, row.value); @@ -69,6 +81,9 @@ export const settingsRepository = { /** * Upsert a setting (insert or update). + * @param {string} key + * @param {any} value Arbitrary JSON-serialisable value. + * @returns {Promise<{ key: string, value: any }>} */ async set(key, value) { const normalized = normalizeSettingValue(value); @@ -85,6 +100,8 @@ export const settingsRepository = { /** * Delete a setting by key. + * @param {string} key + * @returns {Promise} true if a row was removed */ async delete(key) { const result = await query('DELETE FROM user_settings WHERE key = $1 RETURNING key', [key]); @@ -93,6 +110,8 @@ export const settingsRepository = { /** * Bulk upsert multiple settings at once. + * @param {Record} settings keyβ†’value map. + * @returns {Promise} */ async setMany(settings) { const entries = Object.entries(settings); diff --git a/apps/node-backend/src/repositories/transactionRepository.js b/apps/node-backend/src/repositories/transactionRepository.js index 0a5565da..2c0aa099 100644 --- a/apps/node-backend/src/repositories/transactionRepository.js +++ b/apps/node-backend/src/repositories/transactionRepository.js @@ -611,17 +611,16 @@ export const transactionRepository = { mapColumn: (key) => (key === 'transaction_date' ? 'date' : key), }); + // Same 3-level enrichment as getById/getAll/create (shared fragments): the + // update response must not disagree with an immediately-following GET on + // alias-recipient rows categorised via their primary's default. const fetchSql = ` - SELECT t.*, r.name AS recipient_name, - CASE - WHEN c.id IS NOT NULL THEN c.general || ':' || c.detail - WHEN rc.id IS NOT NULL THEN rc.general || ':' || rc.detail - ELSE NULL - END AS category_name + SELECT t.*, + ${RECIPIENT_NAME_SQL} AS recipient_name, + ${EFFECTIVE_CATEGORY_ID_SQL} AS effective_category_id, + ${CATEGORY_NAME_SQL} AS category_name FROM transactions t - LEFT JOIN recipients r ON t.recipient_id = r.id - LEFT JOIN categories c ON t.category_id = c.id - LEFT JOIN categories rc ON r.default_category_id = rc.id + ${TRANSACTION_JOINS} WHERE t.id = $1 `; @@ -668,16 +667,11 @@ export const transactionRepository = { RETURNING * ) SELECT t.*, - r.name AS recipient_name, - CASE - WHEN c.id IS NOT NULL THEN c.general || ':' || c.detail - WHEN rc.id IS NOT NULL THEN rc.general || ':' || rc.detail - ELSE NULL - END AS category_name + ${RECIPIENT_NAME_SQL} AS recipient_name, + ${EFFECTIVE_CATEGORY_ID_SQL} AS effective_category_id, + ${CATEGORY_NAME_SQL} AS category_name FROM updated t - LEFT JOIN recipients r ON t.recipient_id = r.id - LEFT JOIN categories c ON t.category_id = c.id - LEFT JOIN categories rc ON r.default_category_id = rc.id + ${TRANSACTION_JOINS} `; const result = await query(sql, updateParams); diff --git a/apps/node-backend/src/repositories/watchlistRepository.js b/apps/node-backend/src/repositories/watchlistRepository.js index 97eaad98..aa69869f 100644 --- a/apps/node-backend/src/repositories/watchlistRepository.js +++ b/apps/node-backend/src/repositories/watchlistRepository.js @@ -10,9 +10,21 @@ import { buildSetClauses } from '../lib/sqlClauses.js'; // emit so it matches the `number` API/TS type. current_price/price_change are // added later by the route from the price provider, not read from this table. const WATCHLIST_NUMERIC_FIELDS = ['target_price', 'added_price']; + +/** @typedef {import('../types/rows.js').WatchlistRow} WatchlistRow */ +/** @typedef {import('../types/rows.js').FormattedWatchlistRow} FormattedWatchlistRow */ + +/** + * @param {any} row A raw {@link WatchlistRow}. + * @returns {FormattedWatchlistRow} + */ const mapWatchlistRow = (row) => coerceNumericFields(row, WATCHLIST_NUMERIC_FIELDS); export const watchlistRepository = { + /** + * @param {{ assetClass?: string|null }} [filters] + * @returns {{ where: string, params: any[], nextParam: number }} + */ buildWhereClause({ assetClass = null } = {}) { let where = 'WHERE 1=1'; const params = []; @@ -26,6 +38,10 @@ export const watchlistRepository = { return { where, params, nextParam: idx }; }, + /** + * @param {{ limit?: number, offset?: number, assetClass?: string|null }} [opts] + * @returns {Promise} + */ async getAll({ limit = 50, offset = 0, assetClass = null } = {}) { const { where, params, nextParam } = this.buildWhereClause({ assetClass }); let sql = `SELECT * FROM watchlist ${where}`; @@ -38,6 +54,10 @@ export const watchlistRepository = { return result.rows.map(mapWatchlistRow); }, + /** + * @param {{ limit?: number, offset?: number, assetClass?: string|null }} [opts] + * @returns {Promise<{ rows: FormattedWatchlistRow[], total: number }>} + */ async getAllWithCount({ limit = 50, offset = 0, assetClass = null } = {}) { const { where, params, nextParam } = this.buildWhereClause({ assetClass }); let idx = nextParam; @@ -51,10 +71,17 @@ export const watchlistRepository = { const queryParams = [...params, limit, offset]; const result = await query(sql, queryParams); const total = result.rows.length > 0 ? parseInt(result.rows[0].total_count, 10) : 0; - const rows = result.rows.map(({ total_count: _total_count, ...row }) => mapWatchlistRow(row)); + const rows = result.rows.map( + (/** @type {WatchlistRow & { total_count: string }} */ { total_count: _total_count, ...row }) => + mapWatchlistRow(row), + ); return { rows, total }; }, + /** + * @param {{ assetClass?: string|null }} [filters] + * @returns {Promise} + */ async getCount({ assetClass = null } = {}) { const { where, params } = this.buildWhereClause({ assetClass }); const sql = `SELECT count(*) FROM watchlist ${where}`; @@ -63,11 +90,28 @@ export const watchlistRepository = { return parseInt(result.rows[0].count, 10); }, + /** + * @param {number} id + * @returns {Promise} + */ async getById(id) { const result = await query('SELECT * FROM watchlist WHERE id = $1', [id]); return result.rows[0] ? mapWatchlistRow(result.rows[0]) : null; }, + /** + * @param {{ + * name: string, + * symbol?: string|null, + * asset_class: string, + * target_price: number|string, + * currency?: string, + * notes?: string|null, + * price_provider_id?: string|null, + * added_price?: number|string|null, + * }} input + * @returns {Promise} + */ async create({ name, symbol, asset_class, target_price, currency = 'EUR', notes, price_provider_id, added_price }) { const result = await query( `INSERT INTO watchlist (name, symbol, asset_class, target_price, currency, notes, price_provider_id, added_price) @@ -77,6 +121,11 @@ export const watchlistRepository = { return mapWatchlistRow(result.rows[0]); }, + /** + * @param {number} id + * @param {Record} fields Patch; only `allowed` keys are applied. + * @returns {Promise} + */ async update(id, fields) { const allowed = ['name', 'symbol', 'asset_class', 'target_price', 'currency', 'notes', 'price_provider_id']; const { clauses: setClauses, params, nextIdx: idx } = buildSetClauses(fields, { allowed }); @@ -89,6 +138,10 @@ export const watchlistRepository = { return result.rows[0] ? mapWatchlistRow(result.rows[0]) : null; }, + /** + * @param {number} id + * @returns {Promise} true if a row was removed + */ async delete(id) { const result = await query('DELETE FROM watchlist WHERE id = $1', [id]); return result.rowCount > 0; diff --git a/apps/node-backend/src/routes/admin.js b/apps/node-backend/src/routes/admin.js index 852c4943..97ab291a 100644 --- a/apps/node-backend/src/routes/admin.js +++ b/apps/node-backend/src/routes/admin.js @@ -320,8 +320,11 @@ router.post('/providers/:provider/probe', adminMutateLimiter, async (req, res) = // ── Request Metrics ─────────────────────────────────────────────────────────── +// Canonical collection shape `{items, total}` (unpaginated β€” `total` is the +// row count, present so pagination can land without breaking the shape). router.get('/metrics/requests', (_req, res) => { - res.ok(getMetrics()); + const items = getMetrics(); + res.ok({ items, total: items.length }); }); // ── Endpoint Manifest ───────────────────────────────────────────────────────── diff --git a/apps/node-backend/src/routes/ai.js b/apps/node-backend/src/routes/ai.js index b2ac6224..c65e3a5d 100644 --- a/apps/node-backend/src/routes/ai.js +++ b/apps/node-backend/src/routes/ai.js @@ -197,11 +197,14 @@ router.get('/status', async (req, res) => { }); // GET /api/ai/models +// +// Canonical collection shape `{items, total}`; unpaginated, so `total` is the +// row count (present so pagination can land without breaking the shape). router.get('/models', async (req, res) => { const client = getOllamaClient(); try { const models = await client.listModels(); - res.ok({ models }); + res.ok({ items: models, total: models.length }); } catch (err) { if (err instanceof OllamaError) { throw new AppError(`Ollama not reachable: ${err.message}`, { diff --git a/apps/node-backend/src/routes/info/statistics.js b/apps/node-backend/src/routes/info/statistics.js index 331f4774..de08f02c 100644 --- a/apps/node-backend/src/routes/info/statistics.js +++ b/apps/node-backend/src/routes/info/statistics.js @@ -28,9 +28,12 @@ const router = Router(); // cutover (ADR-010): the aggregations.js routes superseded them and they had // zero production callers. The category breakdown lives on via getCategoryBreakdown.) +// Both metadata lists use the canonical `{items, total}` collection shape +// (unpaginated β€” `total` is the row count, present so pagination could land +// without breaking the shape). router.get('/banks', async (req, res) => { const banks = await infoRepository.getBanks(); - res.ok({ banks }); + res.ok({ items: banks, total: banks.length }); }); router.get('/supported-adapters', async (req, res) => { @@ -39,7 +42,7 @@ router.get('/supported-adapters', async (req, res) => { // second hardcoded list to update. (The old list also referenced // *Adapter-class names that don't exist anywhere in the codebase.) const adapters = listAdapters(); - res.ok({ adapters, total_count: adapters.length }); + res.ok({ items: adapters, total: adapters.length }); }); router.get('/transaction-count', async (req, res) => { diff --git a/apps/node-backend/src/routes/plannedTransactions.js b/apps/node-backend/src/routes/plannedTransactions.js index 06a6e84a..48f05fc3 100644 --- a/apps/node-backend/src/routes/plannedTransactions.js +++ b/apps/node-backend/src/routes/plannedTransactions.js @@ -14,7 +14,7 @@ import { z } from 'zod'; import plannedTransactionRepository from '../services/plannedTransactionService.js'; import { resolveRecipientIdByName } from '../services/recipientService.js'; import { resolveCategoryIdByName } from '../services/categoryService.js'; -import { validateIdParam, assertYmd, validateId } from '../middleware/validation.js'; +import { validateIdParam, assertYmd, validateId, assertCurrency } from '../middleware/validation.js'; import { formatDateToYmd } from '../lib/dateFormat.js'; import { rateLimiter } from '../middleware/rateLimiter.js'; import { generateLoanRepaymentSchedule } from '../services/calculations/loanSchedule.js'; @@ -105,6 +105,25 @@ const maxOccurrencesField = z.unknown().transform((value, ctx) => { return n; }).optional(); +// Normalise/validate currency (ISO-4217) so free text never reaches the +// VARCHAR(3) column + 0046 CHECK as a raw 500 (create uppercases before +// insert, so "euro" became "EURO" β†’ CHECK violation; PATCH forwarded the raw +// value). Mirrors transactions.js: POST maps absent/''/null to undefined (the +// repository defaults to 'EUR'), PATCH rejects a cleared value (the column is +// NOT NULL). +const currencyField = ({ rejectEmpty = false } = {}) => z.unknown().transform((value, ctx) => { + if (rejectEmpty && (value == null || value === '')) { + ctx.addIssue({ code: 'custom', message: 'currency cannot be cleared' }); + return z.NEVER; + } + try { + return assertCurrency(value); + } catch (err) { + ctx.addIssue({ code: 'custom', message: /** @type {Error} */ (err).message }); + return z.NEVER; + } +}).optional(); + // POST body. The is_loan-conditional rules live in superRefine because the // loan branch in the handler overrides amount/planned_date/recurrence from the // generated schedule and DELETES truthy recurrence bounds β€” so only values @@ -113,6 +132,7 @@ const maxOccurrencesField = z.unknown().transform((value, ctx) => { const createPlannedSchema = z.looseObject({ tags: tagsField, reminder_days_before: reminderDaysBeforeField, + currency: currencyField(), }).superRefine((data, ctx) => { if (!data.bank_account) { ctx.addIssue({ code: 'custom', message: 'Missing required field: bank_account' }); @@ -185,6 +205,22 @@ const patchPlannedSchema = z.looseObject({ reminder_days_before: reminderDaysBeforeField, recurrence_end_date: recurrenceEndDateField, max_occurrences: maxOccurrencesField, + currency: currencyField({ rejectEmpty: true }), + // amount is a NOT NULL money column; POST already rejects zero/non-finite/ + // absurd values but PATCH forwarded the raw value to the SET builder, so + // `amount: 1e15` overflowed NUMERIC(15,2) β†’ 500, `"Infinity"`/null 500'd at + // the cast, and 0 stored a meaningless never-auto-matching row. Loan PATCHes + // that regenerate the schedule overwrite this value afterwards, exactly as + // before (their installment amount is always derived, never client-sent). + amount: z.unknown().transform((value, ctx) => { + const amountNum = Number(value); + if (value == null || value === '' || !Number.isFinite(amountNum) + || amountNum === 0 || Math.abs(amountNum) > MAX_PLANNED_AMOUNT) { + ctx.addIssue({ code: 'custom', message: 'amount must be a non-zero finite number within range' }); + return z.NEVER; + } + return amountNum; + }).optional(), recurrence_pattern: z.unknown().transform((value, ctx) => { if (value && !isValidPattern(/** @type {string} */ (value))) { ctx.addIssue({ code: 'custom', message: `Invalid recurrence_pattern: ${value}` }); @@ -235,9 +271,12 @@ function applyLoanPatchDefaults(fields, existing) { fields.loan_regular_payment_amount = generatedLoanSchedule.regular_payment_amount; fields.loan_first_payment_date = generatedLoanSchedule.first_due_date; - if (fields.amount === undefined) { - fields.amount = -Math.abs(generatedLoanSchedule.regular_payment_amount); - } + // The schedule was just regenerated, so the installment amount is ALWAYS + // re-derived from it β€” a defined client `amount` is ignored, exactly like + // POST. Keeping a stale client value here desynced `amount` from + // `loan_regular_payment_amount` whenever principal/rate/term changed (or + // on convert-to-loan via PATCH). + fields.amount = -Math.abs(generatedLoanSchedule.regular_payment_amount); if (fields.planned_date === undefined) { fields.planned_date = generatedLoanSchedule.first_due_date; } @@ -345,7 +384,9 @@ router.get('/due-soon', async (req, res) => { const days = Number.isFinite(raw) && raw > 0 ? Math.min(raw, 365) : 7; const rows = await plannedTransactionRepository.getDueSoon(days); const items = rows.map(formatPlannedTransaction); - res.ok(items, { days, total: items.length }); + // Canonical collection shape `{items, total}` in the data body (never counts + // in meta); `days` echoes the effective window alongside. + res.ok({ items, total: items.length, days }); }); /** @@ -359,7 +400,8 @@ router.get('/due-soon', async (req, res) => { */ router.get('/match-suggestions', async (req, res) => { const suggestions = await getMatchSuggestions(); - res.ok(suggestions, { total: suggestions.length }); + // Canonical collection shape `{items, total}` in the data body. + res.ok({ items: suggestions, total: suggestions.length }); }); router.get('/:id', validateIdParam, async (req, res) => { @@ -393,6 +435,22 @@ router.patch( const generatedLoanSchedule = applyLoanPatchDefaults(fields, existing); const loanScheduleDirective = resolveLoanScheduleDirective(generatedLoanSchedule, fields, existing); + // Same guard as POST, on the merged state: a recurring planned tx needs a + // pattern calculateNextDate can advance, or /execute leaves it perpetually + // due. Only enforced when this PATCH touches the recurrence fields, so an + // unrelated edit to a legacy broken row is not blocked. Loan PATCHes that + // regenerate the schedule have already defaulted pattern='monthly' above. + if (fields.is_recurring !== undefined || fields.recurrence_pattern !== undefined) { + const resultingIsLoan = fields.is_loan !== undefined ? !!fields.is_loan : !!existing.is_loan; + const resultingIsRecurring = fields.is_recurring !== undefined ? !!fields.is_recurring : !!existing.is_recurring; + const resultingPattern = fields.recurrence_pattern !== undefined + ? fields.recurrence_pattern + : existing.recurrence_pattern; + if (!resultingIsLoan && resultingIsRecurring && !isValidPattern(/** @type {string} */ (resultingPattern))) { + throw new ValidationError(`Invalid or missing recurrence_pattern: ${resultingPattern}`); + } + } + // When the loan schedule must change, the field update and the schedule // rewrite MUST happen in one transaction β€” otherwise a crash between them // leaves the planned row's loan params disagreeing with the installment rows. diff --git a/apps/node-backend/src/routes/portfolioImportRoutes.js b/apps/node-backend/src/routes/portfolioImportRoutes.js index f0125943..9693ee86 100644 --- a/apps/node-backend/src/routes/portfolioImportRoutes.js +++ b/apps/node-backend/src/routes/portfolioImportRoutes.js @@ -432,7 +432,18 @@ router.post('/batches/:id/rows/:rowId/investment-override', async (req, res) => const { batchId, rowId } = parseBatchRowIdParams(req); if (req.body.create_new === true) { - const investment = await createInvestmentForRow({ batchId, rowId }); + let investment; + try { + investment = await createInvestmentForRow({ batchId, rowId }); + } catch (err) { + // Repository/service VALIDATION_ERROR (missing default asset class, no + // name, duplicate symbol) β†’ typed 400, matching investmentController's + // translateRepoError β€” a raw coded Error would surface as a 500. + if (/** @type {any} */ (err)?.code === 'VALIDATION_ERROR') { + throw new ValidationError(/** @type {Error} */ (err).message); + } + throw err; + } if (!investment) throw new NotFoundError(`Row ${rowId} not found in batch ${batchId}`); res.ok({ row_id: rowId, investment_id: investment.id, created: true, investment }); return; diff --git a/apps/node-backend/src/routes/recipientBankAccounts.js b/apps/node-backend/src/routes/recipientBankAccounts.js index c3e7af3b..5b87001d 100644 --- a/apps/node-backend/src/routes/recipientBankAccounts.js +++ b/apps/node-backend/src/routes/recipientBankAccounts.js @@ -5,7 +5,7 @@ import { Router } from 'express'; import recipientBankAccountRepository from '../services/recipientBankAccountService.js'; import { NotFoundError, ValidationError } from '../middleware/errorHandler.js'; -import { validateIdParam } from '../middleware/validation.js'; +import { validateIdParam, assertMaxLength } from '../middleware/validation.js'; const router = Router(); @@ -27,6 +27,9 @@ router.post('/:id/bank-accounts', validateIdParam, async (req, res) => { const { account_number, bank_name, address, account_label, set_as_primary } = req.body; if (!account_number) throw new ValidationError('Missing required field: account_number'); + // account_number is VARCHAR(34) (IBAN max width, migration 0001) β€” an + // over-length value otherwise reached the column as a raw 22001 500. + assertMaxLength(account_number, 34, 'account_number'); const { bankAccount, created } = await recipientBankAccountRepository.createOrGet({ recipientId, diff --git a/apps/node-backend/src/routes/research.js b/apps/node-backend/src/routes/research.js index 1342231a..698502fd 100644 --- a/apps/node-backend/src/routes/research.js +++ b/apps/node-backend/src/routes/research.js @@ -215,10 +215,13 @@ function positiveInt(value) { } // GET /api/research/mappings?instrument_key=US0378331005&key_type=isin +// +// Canonical collection shape `{items, total}`; unpaginated, so `total` is the +// row count (present so pagination can land without breaking the shape). router.get('/mappings', async (req, res) => { const instrumentKey = requireInstrumentKey(req.query.instrument_key); const rows = await researchMappingService.list(instrumentKey, keyType(req.query.key_type)); - res.ok({ mappings: rows }); + res.ok({ items: rows, total: rows.length }); }); // POST /api/research/mappings/resolve { instrument_key, key_type, asset_class, query, investment_id } @@ -236,6 +239,8 @@ router.post('/mappings/resolve', async (req, res) => { }); // POST /api/research/mappings { instrument_key, key_type, mappings: [...] } +// Answers the updated mapping set in the same canonical `{items, total}` +// collection shape as GET /mappings (one response type for both). router.post('/mappings', async (req, res) => { const { instrument_key, key_type, mappings } = req.body ?? {}; const rows = await researchMappingService.save({ @@ -243,7 +248,7 @@ router.post('/mappings', async (req, res) => { keyType: keyType(key_type), mappings: parseResearchParam(mappingsArraySchema, mappings), }); - res.ok({ mappings: rows }); + res.ok({ items: rows, total: rows.length }); }); // DELETE /api/research/mappings/:id @@ -270,17 +275,21 @@ router.post('/mappings/audit', async (req, res) => { // Keys are masked in responses and never returned in full. // GET /api/research/provider-keys +// +// Canonical collection shape `{items, total}` (fixed provider roster, so +// `total` is simply the row count). router.get('/provider-keys', async (_req, res) => { - const providers = await researchProviderKeyService.listKeyStatuses(); - res.ok({ providers }); + const items = await researchProviderKeyService.listKeyStatuses(); + res.ok({ items, total: items.length }); }); // PUT /api/research/provider-keys/:provider { api_key } +// Answers the refreshed statuses in the same `{items, total}` shape as the GET. router.put('/provider-keys/:provider', async (req, res) => { const { api_key } = req.body ?? {}; await researchProviderKeyService.setKey(req.params.provider, api_key); - const providers = await researchProviderKeyService.listKeyStatuses(); - res.ok({ providers }); + const items = await researchProviderKeyService.listKeyStatuses(); + res.ok({ items, total: items.length }); }); // DELETE /api/research/provider-keys/:provider diff --git a/apps/node-backend/src/routes/settings.js b/apps/node-backend/src/routes/settings.js index 69cafd3f..70dbe1bf 100644 --- a/apps/node-backend/src/routes/settings.js +++ b/apps/node-backend/src/routes/settings.js @@ -161,6 +161,17 @@ const SETTING_SCHEMAS = { // Year-keyed maps: snapshots get the full profile validation per entry. belgian_tax_profile_snapshots_v1: z.record(z.string(), belgianTaxProfileSchema), belgian_tax_profile_snapshot_meta_v1: z.record(z.string(), jsonObjectSchema), + // Remaining first-party keys, same conservative top-level-shape guards. + // RecurringDetectionPanel stores an array of dismissed recipient ids (top- + // level shape only β€” entries stay unvalidated, matching the blob guards). + dismissed_recurring_patterns: z.array(z.unknown()), + // usePortfolioTaxAdjustments / usePortfolioTaxClassifications store + // ":" / ""-keyed entry maps. + portfolio_tax_adjustments_v1: z.record(z.string(), jsonObjectSchema), + portfolio_tax_classifications_v1: z.record(z.string(), jsonObjectSchema), + // Internal one-shot FX-repair flag (written repository-side; guarded here so + // an API write can't corrupt its type). + fx_full_history_repair_done: z.boolean(), }; const FORBIDDEN_SETTING_KEYS = new Set(['__proto__', 'constructor', 'prototype']); @@ -227,6 +238,27 @@ const SETTING_DEFAULTS = { includeTransfers: false, }; +/* ── Unknown-key policy ────────────────────────────────────────────────────── + * Every settings writer in the repo uses a fixed key (grep: frontend + * `saveSetting(` call sites, packaging/electron/main.js backup_settings, and + * the repository-side fx flag) β€” there is NO dynamic-key writer. So writes to + * a key outside the known set are typos/garbage and are rejected with a 400 + * naming the known keys, instead of storing arbitrary JSON forever. Reads and + * DELETE stay unrestricted so legacy keys from restored backups can still be + * listed and cleaned up. */ +const KNOWN_SETTING_KEYS = new Set([ + ...Object.keys(SETTING_SCHEMAS), + ...Object.keys(SETTING_DEFAULTS), +]); + +function assertKnownSettingKey(key) { + if (!KNOWN_SETTING_KEYS.has(key)) { + throw new ValidationError( + `Unknown setting key '${key}'. Known keys: ${[...KNOWN_SETTING_KEYS].sort().join(', ')}`, + ); + } +} + router.get('/:key', async (req, res) => { const { key } = req.params; const value = await settingsRepository.get(key); @@ -262,6 +294,7 @@ router.put('/:key', async (req, res) => { const { value } = req.body; assertSettingKeyLength(key); + assertKnownSettingKey(key); if (value === undefined) throw new ValidationError('Missing "value" in request body'); const result = await settingsRepository.set(key, validateSettingValue(key, value)); @@ -274,7 +307,10 @@ router.put('/', async (req, res) => { throw new ValidationError('Body must be a JSON object of keyβ†’value pairs'); } - for (const key of Object.keys(settings)) assertSettingKeyLength(key, true); + for (const key of Object.keys(settings)) { + assertSettingKeyLength(key, true); + assertKnownSettingKey(key); + } const validatedEntries = new Map(); for (const [key, value] of Object.entries(settings)) { diff --git a/apps/node-backend/src/services/aiChat/tools/insights.js b/apps/node-backend/src/services/aiChat/tools/insights.js index b4333b71..a6799c81 100644 --- a/apps/node-backend/src/services/aiChat/tools/insights.js +++ b/apps/node-backend/src/services/aiChat/tools/insights.js @@ -14,6 +14,7 @@ import settings from '../../../config/config.js'; import { toDecimal, roundToCents } from '../../../lib/money.js'; import { toYmd } from '../../../utils/portfolioMath.js'; import { parseEnum, parsePositiveInt } from './_validate.js'; +import { getQuotes } from '../../marketLookupService.js'; import { detectRecurringPatterns } from '../../recurringDetectionService.js'; import { getInsightsDigest } from '../../insightsDigestService.js'; @@ -211,17 +212,40 @@ export const getWatchlist = { assetClass, }); - const shaped = rows.map((item) => ({ - id: item.id, - name: item.name, - symbol: item.symbol || null, - assetClass: item.asset_class, - currentPrice: item.current_price != null - ? roundToCents(toDecimal(item.current_price)).toNumber() - : null, - currency: item.currency || 'EUR', - notes: item.notes || null, - })); + // current_price is NOT a watchlist column β€” the repository never emits it. + // The watchlist UI merges the live quote in client-side (GET + // /api/market/quote); mirror that here via the same service so the tool + // reports real prices instead of always-null. Quote failures (offline, no + // provider) degrade to null prices rather than failing the tool. + const symbols = [...new Set(rows.map((item) => item.symbol).filter(Boolean))]; + const priceBySymbol = new Map(); + if (symbols.length > 0) { + try { + const { items } = await getQuotes(symbols, true); + for (const quote of /** @type {Array<{ symbol?: string, price?: number|null }>} */ (items)) { + if (quote.symbol != null && quote.price != null) { + priceBySymbol.set(quote.symbol, quote.price); + } + } + } catch { + // Degrade gracefully: leave priceBySymbol empty β†’ currentPrice: null. + } + } + + const shaped = rows.map((item) => { + const livePrice = item.symbol ? priceBySymbol.get(item.symbol) : undefined; + return { + id: item.id, + name: item.name, + symbol: item.symbol || null, + assetClass: item.asset_class, + currentPrice: livePrice != null + ? roundToCents(toDecimal(livePrice)).toNumber() + : null, + currency: item.currency || 'EUR', + notes: item.notes || null, + }; + }); return { ok: true, diff --git a/apps/node-backend/src/services/importPipeline/adapters/_shared.js b/apps/node-backend/src/services/importPipeline/adapters/_shared.js index 19b2c81b..d8b40107 100644 --- a/apps/node-backend/src/services/importPipeline/adapters/_shared.js +++ b/apps/node-backend/src/services/importPipeline/adapters/_shared.js @@ -28,6 +28,21 @@ export function parseDecimalSafe(value) { } } +/** + * Normalize a CSV currency cell to an uppercase ISO-4217-shaped code, or null + * when the cell doesn't hold one. transactions.currency is VARCHAR(3) with an + * `^[A-Z]{3}$` CHECK (migration 0046), so a free-text cell ("euro", "US$") + * forwarded raw failed the whole commit as a raw DB 500 mid-import; a null + * falls back to the pipeline's EUR default at commit instead. + * + * @param {unknown} value + * @returns {string|null} + */ +export function normalizeIsoCurrency(value) { + const code = String(value ?? '').trim().toUpperCase(); + return /^[A-Z]{3}$/.test(code) ? code : null; +} + /** * Read a text file, decoding as UTF-8 but falling back to latin1 (ISO-8859-1) * when the bytes aren't valid UTF-8. Belgian bank exports are frequently diff --git a/apps/node-backend/src/services/importPipeline/adapters/generic.js b/apps/node-backend/src/services/importPipeline/adapters/generic.js index dde96288..a452c016 100644 --- a/apps/node-backend/src/services/importPipeline/adapters/generic.js +++ b/apps/node-backend/src/services/importPipeline/adapters/generic.js @@ -5,7 +5,7 @@ import { logger } from '../../../config/logger.js'; import { normalizeToUppercase } from '../../../lib/textNormalization.js'; -import { parseCsvFile, buildRawRowString, parseAmountField, SUPPORTED_DATE_FORMATS, parseDateWithFormat } from './_shared.js'; +import { parseCsvFile, buildRawRowString, parseAmountField, SUPPORTED_DATE_FORMATS, parseDateWithFormat, normalizeIsoCurrency } from './_shared.js'; const NAME = 'generic'; const BANK_LABEL = 'Generic'; @@ -34,8 +34,10 @@ function rowToTransaction(row, config) { const recipient = String(row[colMap.recipient] || '').trim(); const memo = colMap.memo ? String(row[colMap.memo] || '').trim() : ''; + // ISO-shape normalize (uppercase) or null β†’ commit's EUR default; a raw + // free-text cell failed the whole commit at the 0046 currency CHECK (500). let currency = null; - if (colMap.currency) currency = String(row[colMap.currency] || '').trim() || null; + if (colMap.currency) currency = normalizeIsoCurrency(row[colMap.currency]); let balance = null; if (colMap.balance) { diff --git a/apps/node-backend/src/services/importPipeline/adapters/vision.js b/apps/node-backend/src/services/importPipeline/adapters/vision.js index 1930d76a..176740bb 100644 --- a/apps/node-backend/src/services/importPipeline/adapters/vision.js +++ b/apps/node-backend/src/services/importPipeline/adapters/vision.js @@ -4,7 +4,7 @@ import { cleanRecipientName, normalizeToUppercase } from '../../../lib/textNormalization.js'; import { logger } from '../../../config/logger.js'; -import { parseCsvFile, buildOptionalComment, buildRawRowString, parseAmountField, parseDateFlexibleUtc } from './_shared.js'; +import { parseCsvFile, buildOptionalComment, buildRawRowString, parseAmountField, parseDateFlexibleUtc, normalizeIsoCurrency } from './_shared.js'; const NAME = 'vision'; const BANK_LABEL = 'Vision'; @@ -30,7 +30,9 @@ function rowToTransaction(row) { const recipientRaw = (row['Recipient'] || '').trim(); const recipient = recipientRaw ? normalizeToUppercase(cleanRecipientName(recipientRaw)) : 'UNKNOWN'; const memo = row['Memo'] ? normalizeToUppercase(row['Memo'].trim()) : ''; - const currency = (row['Currency'] || 'EUR').trim().toUpperCase(); + // ISO-shape normalize: a hand-edited "euro" cell became "EURO" and failed + // the whole commit at the VARCHAR(3) + 0046 CHECK as a raw 500. + const currency = normalizeIsoCurrency(row['Currency']) || 'EUR'; // Same guard-apostrophe cleanup as Amount, so a negative Balance survives the // round-trip instead of being silently nulled. const balanceStr = (row['Balance'] || '').replace(/'/g, '').trim(); diff --git a/apps/node-backend/src/services/marketLookupService.js b/apps/node-backend/src/services/marketLookupService.js index 1dd11f0f..e682a8f5 100644 --- a/apps/node-backend/src/services/marketLookupService.js +++ b/apps/node-backend/src/services/marketLookupService.js @@ -257,27 +257,28 @@ export async function searchSymbols(q) { } /** - * Batch quote lookup. Returns the `{ quotes }` payload for GET - * /api/market/quote. `basic` returns price fields only; the default (full) - * additionally fetches quoteSummary for fundamentals/analyst data. Results are - * per-symbol cached (QUOTE_CACHE_TTL_MS) and concurrent identical fetches are - * coalesced; a failed symbol is dropped rather than failing the batch. + * Batch quote lookup. Returns the canonical `{ items, total }` collection + * payload for GET /api/market/quote. `basic` returns price fields only; the + * default (full) additionally fetches quoteSummary for fundamentals/analyst + * data. Results are per-symbol cached (QUOTE_CACHE_TTL_MS) and concurrent + * identical fetches are coalesced; a failed symbol is dropped rather than + * failing the batch. * * @param {string[]} symbolList * @param {boolean} basic - * @returns {Promise<{ quotes: Array }>} + * @returns {Promise<{ items: Array, total: number }>} */ export async function getQuotes(symbolList, basic) { const quoteResults = await Promise.allSettled( symbolList.map((sym) => getCachedQuote(sym, basic)), ); - const quotes = quoteResults + const items = quoteResults .filter(/** @type {(r: PromiseSettledResult) => r is PromiseFulfilledResult} */ (r) => r.status === 'fulfilled' && r.value !== null && r.value !== undefined) .map((r) => r.value); - return { quotes }; + return { items, total: items.length }; } /** @@ -285,9 +286,12 @@ export async function getQuotes(symbolList, basic) { * `range`/`interval` are passed through to yahoo-finance2, which validates them * against its own literal-union types β€” leave them loosely typed. * + * The series travels in the canonical `items` key (with `total`); `symbol` and + * `currency` ride alongside in the body. + * * @param {string} symbol * @param {{ range?: any, interval?: any }} [options] - * @returns {Promise<{ symbol?: string, currency?: string, points: Array }>} + * @returns {Promise<{ symbol?: string, currency?: string, items: Array, total: number }>} */ export async function getChart(symbol, { range = '1mo', interval = '1d' } = {}) { /** @type {any} */ @@ -306,7 +310,7 @@ export async function getChart(symbol, { range = '1mo', interval = '1d' } = {}) throw upstreamError('Market chart unavailable', err); } - if (!result) return { points: [] }; + if (!result) return { items: [], total: 0 }; const points = (result.quotes || []) .filter((p) => p.close != null) @@ -321,17 +325,18 @@ export async function getChart(symbol, { range = '1mo', interval = '1d' } = {}) return { symbol: result.meta?.symbol, currency: result.meta?.currency, - points, + items: points, + total: points.length, }; } /** * Symbol news feed, deduplicated by title and sorted newest-first. Returns the - * `{ articles }` payload for GET /api/market/news. + * canonical `{ items, total }` collection payload for GET /api/market/news. * * @param {string} symbols Comma-separated symbols ('' falls back to SPY,QQQ,DIA). * @param {string} count Requested article count as a string; capped at 50. - * @returns {Promise<{ articles: Array }>} + * @returns {Promise<{ items: Array, total: number }>} */ export async function getNews(symbols, count) { const querySymbols = symbols || 'SPY,QQQ,DIA'; @@ -374,5 +379,5 @@ export async function getNews(symbols, count) { .sort((a, b) => (b.publishedAt || 0) - (a.publishedAt || 0)) .slice(0, newsCount); - return { articles }; + return { items: articles, total: articles.length }; } diff --git a/apps/node-backend/src/services/materializedViewService.js b/apps/node-backend/src/services/materializedViewService.js index 7f547a58..f79276aa 100644 --- a/apps/node-backend/src/services/materializedViewService.js +++ b/apps/node-backend/src/services/materializedViewService.js @@ -84,7 +84,12 @@ export async function createMaterializedViews() { // 2-3. Category totals and daily cashflow are fully independent of each other β€” // create them in parallel. await Promise.all([ - // 2. Category totals (all-time) + // 2. Category totals (all-time). Effective category resolves 3 levels + // (own β†’ recipient default β†’ PRIMARY recipient's default) so the MV + // fast path agrees with getCategoryBreakdown's live fallback and the + // transactions surfaces. Changing this definition requires a migration + // that DROPs the MV (see 0084): IF NOT EXISTS never redefines an + // existing view, so already-migrated installs keep the old SQL otherwise. query(` CREATE MATERIALIZED VIEW IF NOT EXISTS mv_category_totals AS SELECT @@ -95,7 +100,8 @@ export async function createMaterializedViews() { t.currency FROM transactions t LEFT JOIN recipients r ON t.recipient_id = r.id - LEFT JOIN categories c ON COALESCE(t.category_id, r.default_category_id) = c.id + LEFT JOIN recipients pr ON r.primary_recipient_id = pr.id + LEFT JOIN categories c ON COALESCE(t.category_id, r.default_category_id, pr.default_category_id) = c.id WHERE t.is_active = true AND t.is_transfer = false GROUP BY COALESCE(c.id, -1), diff --git a/apps/node-backend/src/services/portfolioImportBatchService.js b/apps/node-backend/src/services/portfolioImportBatchService.js index 40671ca7..1c3670e7 100644 --- a/apps/node-backend/src/services/portfolioImportBatchService.js +++ b/apps/node-backend/src/services/portfolioImportBatchService.js @@ -46,11 +46,16 @@ export async function createInvestmentForRow({ batchId, rowId }) { throw err; } + // CSV-derived currency: coalesce to EUR unless it is a clean ISO-shaped code + // (uppercased). investments.currency is VARCHAR(10) with no CHECK, so a + // malformed CSV cell either stored garbage or (>10 chars) 500'd the insert β€” + // fallback (not reject) matches the pipeline's existing `|| 'EUR'` coalescing. + const rawCurrency = String(row.currency || '').trim().toUpperCase(); const investment = await investmentRepository.create(/** @type {any} */ ({ name, symbol: (row.symbol_raw || '').trim() || undefined, asset_class: row.default_asset_class, - currency: (row.currency || 'EUR').trim() || 'EUR', + currency: /^[A-Z]{3}$/.test(rawCurrency) ? rawCurrency : 'EUR', price_provider: 'manual', })); diff --git a/apps/node-backend/src/types/rows.js b/apps/node-backend/src/types/rows.js index 7350aea8..3a301839 100644 --- a/apps/node-backend/src/types/rows.js +++ b/apps/node-backend/src/types/rows.js @@ -265,6 +265,23 @@ * @property {Date|null} [updated_at] */ +/** + * A row of `recipient_bank_accounts` (`SELECT *` / `RETURNING *`; baseline + * schema). + * + * @typedef {object} RecipientBankAccountRow + * @property {number} id + * @property {number|null} recipient_id + * @property {string} account_number VARCHAR(34), stored trimmed + uppercased. + * @property {string|null} bank_name + * @property {string|null} account_label + * @property {string|null} address + * @property {boolean} is_primary + * @property {boolean} is_active + * @property {Date|null} created_at + * @property {Date|null} updated_at + */ + /** * `RecipientRow` plus the derived columns the list / detail / update reads project. * @@ -514,6 +531,79 @@ * @property {number} count */ +// --------------------------------------------------------------------------- +// Saved charts +// --------------------------------------------------------------------------- + +/** + * A row of `saved_charts` as projected by `savedChartsRepository`'s shared + * `COLUMNS` list (baseline + migrations 0017/0063/0064). The two DATE columns + * are `to_char`-formatted in SQL, so they are calendar-day strings, not `Date`s. + * INTEGER[] columns come back from pg as `number[]` already; the repository's + * `mapRow` re-normalises them (and the three booleans) defensively without + * changing the type, so raw and emitted shapes coincide. + * + * @typedef {object} SavedChartRow + * @property {number} id + * @property {string} name + * @property {string} chart_type + * @property {number[]} category_ids INTEGER[]. + * @property {number[]} recipient_ids INTEGER[]. + * @property {number[]} tag_ids INTEGER[] (migration 0063). + * @property {boolean} all_categories + * @property {boolean} all_recipients + * @property {boolean} all_tags + * @property {string} chart_variant + * @property {string} time_bucket + * @property {string|null} date_range_start 'YYYY-MM-DD' β€” `to_char`-formatted in the projection. + * @property {string|null} date_range_end 'YYYY-MM-DD' β€” `to_char`-formatted in the projection. + * @property {Date} created_at + * @property {Date} updated_at + */ + +// --------------------------------------------------------------------------- +// Watchlist +// --------------------------------------------------------------------------- + +/** + * A raw row of `watchlist` (`SELECT *` / `RETURNING *`; baseline schema + + * migration 0058). NOT what the repository returns β€” every read funnels + * through `mapWatchlistRow`. + * + * @typedef {object} WatchlistRow + * @property {number} id + * @property {string} name + * @property {string|null} symbol + * @property {string} asset_class `asset_class` enum: stock|etf|crypto|metals|real_estate|savings|bond. + * @property {string} target_price NUMERIC(18,6) β€” pg emits NUMERIC as a string. + * @property {string} currency + * @property {string|null} notes + * @property {string|null} price_provider_id + * @property {string|null} added_price NUMERIC(18,6); NULL on rows predating migration 0058. + * @property {Date} created_at + * @property {Date} updated_at + */ + +/** + * A `watchlist` row after `mapWatchlistRow` coerced the two NUMERIC columns + * (`WATCHLIST_NUMERIC_FIELDS`) to numbers. + * + * @typedef {object} FormattedWatchlistRow + * @property {number} id + * @property {string} name + * @property {string|null} symbol + * @property {string} asset_class + * @property {number} target_price + * @property {string} currency + * @property {string|null} notes + * @property {string|null} price_provider_id + * @property {number|null} added_price + * @property {Date} created_at + * @property {Date} updated_at + * @property {number|null} [current_price] Not a column β€” only present when a caller (the watchlist route) merged the live quote in; bare repository reads never emit it. + * @property {number|null} [price_change] Not a column β€” merged in by the watchlist route alongside `current_price`. + */ + // --------------------------------------------------------------------------- // Tags // --------------------------------------------------------------------------- @@ -530,6 +620,123 @@ * @property {Date} updated_at */ +// --------------------------------------------------------------------------- +// AI chat (ai_conversations + ai_messages) +// --------------------------------------------------------------------------- + +/** + * An `ai_conversations` row as projected by `aiChatRepository`'s + * `CONVERSATION_COLUMNS` β€” the timestamps are aliased to camelCase in SQL. + * + * @typedef {object} AiConversationRow + * @property {string} id UUID. + * @property {string} title + * @property {string} model + * @property {Date} createdAt Aliased from `created_at` (TIMESTAMPTZ). + * @property {Date} updatedAt Aliased from `updated_at` (TIMESTAMPTZ). + */ + +/** + * An `ai_messages` row as projected by `aiChatRepository`'s `MESSAGE_COLUMNS` + * β€” the snake_case columns are aliased to camelCase in SQL. + * + * @typedef {object} AiMessageRow + * @property {string} id UUID. + * @property {string} conversationId UUID FK β†’ ai_conversations. + * @property {'user'|'assistant'|'tool'|'system'} role + * @property {string|null} content + * @property {string|null} toolName + * @property {any} toolArgs JSONB β€” parsed value or null. + * @property {any} toolResult JSONB β€” parsed value or null. + * @property {'complete'|'streaming'|'aborted'|'error'} status + * @property {Date} createdAt + */ + +// --------------------------------------------------------------------------- +// Attachments +// --------------------------------------------------------------------------- + +/** + * A raw row of `attachments` (`SELECT *` / `RETURNING *`, migration 0004). All + * three BIGINT columns come back from pg as strings. + * + * @typedef {object} AttachmentRow + * @property {string} id BIGSERIAL β€” string, not number. + * @property {string} transaction_id BIGINT FK β†’ transactions β€” string. + * @property {string} filename + * @property {string} stored_path + * @property {string} mime_type + * @property {string} size_bytes BIGINT β€” string. + * @property {Date} created_at + */ + +/** + * What `attachmentRepository`'s `formatRow` emits: `size_bytes` coerced to a + * number; `id` / `transaction_id` stay BIGINT strings. + * + * @typedef {object} FormattedAttachment + * @property {string} id + * @property {string} transaction_id + * @property {string} filename + * @property {string} stored_path + * @property {string} mime_type + * @property {number} size_bytes + * @property {Date} created_at + */ + +// --------------------------------------------------------------------------- +// Custom parser configs +// --------------------------------------------------------------------------- + +/** + * A raw row of `custom_parser_configs` (migrations 0037 + 0041). NOT what the + * repository returns β€” every path funnels through its `mapRow`. + * + * @typedef {object} CustomParserConfigRow + * @property {number} id + * @property {string} name + * @property {'transaction'|'portfolio'} kind + * @property {any} config_json JSONB β€” pg hands it back already parsed. + * @property {Date} created_at + * @property {Date} updated_at + */ + +/** + * What `customParserConfigRepository`'s `mapRow` emits: `config_json` re-keyed + * to `config` (parsed if it somehow arrived as a string). + * + * @typedef {object} FormattedCustomParserConfig + * @property {number} id + * @property {string} name + * @property {'transaction'|'portfolio'} kind + * @property {any} config Parsed JSONB parser definition. + * @property {Date} created_at + * @property {Date} updated_at + */ + +// --------------------------------------------------------------------------- +// Research provider mappings (ADR-079) +// --------------------------------------------------------------------------- + +/** + * A row of `instrument_provider_map` (migration 0042) as projected by + * `instrumentProviderMapRepository`'s shared `COLUMNS` list β€” the full table. + * + * @typedef {object} InstrumentProviderMapRow + * @property {number} id + * @property {string} instrument_key ISIN (`key_type='isin'`) or internal id. + * @property {'isin'|'internal'} key_type + * @property {string} provider + * @property {string|null} provider_symbol + * @property {string|null} resolved_name + * @property {string|null} exchange + * @property {string|null} currency + * @property {'confirmed'|'auto'|'failed'} status + * @property {Date|null} verified_at TIMESTAMPTZ + * @property {Date} created_at + * @property {Date} updated_at + */ + // --------------------------------------------------------------------------- // Import batches // --------------------------------------------------------------------------- diff --git a/apps/node-backend/tests/aiChatInsights.test.js b/apps/node-backend/tests/aiChatInsights.test.js index e3a98596..190a0cce 100644 --- a/apps/node-backend/tests/aiChatInsights.test.js +++ b/apps/node-backend/tests/aiChatInsights.test.js @@ -29,11 +29,16 @@ vi.mock('../src/services/recurringDetectionService.js', () => ({ detectRecurringPatterns: vi.fn(), })); +vi.mock('../src/services/marketLookupService.js', () => ({ + getQuotes: vi.fn(), +})); + import { infoRepository } from '../src/repositories/infoRepository.js'; import { watchlistRepository } from '../src/repositories/watchlistRepository.js'; import { categoryRepository } from '../src/repositories/categoryRepository.js'; import { transactionRepository } from '../src/repositories/transactionRepository.js'; import { detectRecurringPatterns } from '../src/services/recurringDetectionService.js'; +import { getQuotes } from '../src/services/marketLookupService.js'; import { getBankBalances, getSpendingPace, @@ -203,14 +208,19 @@ describe('getRecipientInsights', () => { }); describe('getWatchlist', () => { - it('reshapes raw rows into the tool envelope', async () => { + it('reshapes raw rows and merges live quote prices by symbol', async () => { + // current_price is not a watchlist column β€” the tool must fetch the live + // quote (like the watchlist page does) instead of reading it off the row. watchlistRepository.getAllWithCount.mockResolvedValueOnce({ rows: [ - { id: 1, name: 'Apple', symbol: 'AAPL', asset_class: 'stock', current_price: '174.99', currency: 'USD', notes: null }, + { id: 1, name: 'Apple', symbol: 'AAPL', asset_class: 'stock', currency: 'USD', notes: null }, ], total: 1, }); + getQuotes.mockResolvedValueOnce({ items: [{ symbol: 'AAPL', price: 174.99 }], total: 1 }); + const r = await getWatchlist.run({}); + expect(getQuotes).toHaveBeenCalledWith(['AAPL'], true); expect(r.data).toEqual([ { id: 1, name: 'Apple', symbol: 'AAPL', assetClass: 'stock', currentPrice: 174.99, currency: 'USD', notes: null }, ]); @@ -223,17 +233,47 @@ describe('getWatchlist', () => { expect(watchlistRepository.getAllWithCount).toHaveBeenCalledWith(expect.objectContaining({ assetClass: 'crypto' })); }); - it('handles null current_price', async () => { + it('skips the quote fetch and reports null price for symbol-less items', async () => { watchlistRepository.getAllWithCount.mockResolvedValueOnce({ - rows: [{ id: 1, name: 'X', symbol: null, asset_class: 'bond', current_price: null, currency: null, notes: null }], + rows: [{ id: 1, name: 'X', symbol: null, asset_class: 'bond', currency: null, notes: null }], total: 1, }); const r = await getWatchlist.run({}); + expect(getQuotes).not.toHaveBeenCalled(); expect(r.data[0].currentPrice).toBeNull(); expect(r.data[0].currency).toBe('EUR'); // fallback expect(r.data[0].symbol).toBeNull(); }); + it('degrades to null prices when the quote service fails', async () => { + watchlistRepository.getAllWithCount.mockResolvedValueOnce({ + rows: [{ id: 1, name: 'Apple', symbol: 'AAPL', asset_class: 'stock', currency: 'USD', notes: null }], + total: 1, + }); + getQuotes.mockRejectedValueOnce(new Error('offline')); + + const r = await getWatchlist.run({}); + expect(r.ok).toBe(true); + expect(r.data[0].currentPrice).toBeNull(); + }); + + it('reports null price for a symbol the quote batch dropped', async () => { + // getQuotes drops failed symbols from items rather than failing the batch. + watchlistRepository.getAllWithCount.mockResolvedValueOnce({ + rows: [ + { id: 1, name: 'Apple', symbol: 'AAPL', asset_class: 'stock', currency: 'USD', notes: null }, + { id: 2, name: 'Unknown', symbol: 'NOPE', asset_class: 'stock', currency: 'EUR', notes: null }, + ], + total: 2, + }); + getQuotes.mockResolvedValueOnce({ items: [{ symbol: 'AAPL', price: 200.5 }], total: 1 }); + + const r = await getWatchlist.run({}); + expect(getQuotes).toHaveBeenCalledWith(['AAPL', 'NOPE'], true); + expect(r.data[0].currentPrice).toBe(200.5); + expect(r.data[1].currentPrice).toBeNull(); + }); + it('rejects unknown asset class', async () => { await expect(getWatchlist.run({ assetClass: 'magic-beans' })).rejects.toThrow(/assetClass/); }); diff --git a/apps/node-backend/tests/importPipelineAdapters.test.js b/apps/node-backend/tests/importPipelineAdapters.test.js index a099c4a2..988a2caa 100644 --- a/apps/node-backend/tests/importPipelineAdapters.test.js +++ b/apps/node-backend/tests/importPipelineAdapters.test.js @@ -206,6 +206,18 @@ describe('generic adapter β€” configurable mapping', () => { parseGeneric(writeTempCSV('generic', 'Date,Amount,Payee,Note,Cur\n31.12.2024,"-10,00",S,x,EUR'), cfg), ).rejects.toThrow(/Unsupported date_format/); }); + + it('normalizes ISO-shaped currency to uppercase and nulls free text (was a commit-time CHECK 500)', async () => { + const csv = [ + 'Date,Amount,Payee,Note,Cur', + '24/11/2025,"-10,00",SHOP,a,usd', // lowercase ISO β†’ USD + '23/11/2025,"-11,00",SHOP,b,euro', // free text β†’ null (commit defaults EUR) + '22/11/2025,"-12,00",SHOP,c,', // empty β†’ null + ].join('\n'); + const txns = await parseGeneric(writeTempCSV('generic', csv), config); + + expect(txns.map((t) => t.currency)).toEqual(['USD', null, null]); + }); }); describe('wise adapter β€” cross-currency direction', () => { diff --git a/apps/node-backend/tests/infoRepoStatistics.db.test.js b/apps/node-backend/tests/infoRepoStatistics.db.test.js new file mode 100644 index 00000000..12806c8b --- /dev/null +++ b/apps/node-backend/tests/infoRepoStatistics.db.test.js @@ -0,0 +1,328 @@ +/** + * Real-Postgres tests for infoRepositoryStatistics. + * + * DB-backed complement to infoRepoStatistics.test.js (which stays: it runs + * without a DB). The mock suite feeds pre-shaped rows into the JS aggregation + * and asserts SQL substrings; here the live queries run against a migrated + * schema with realistic fixtures β€” NUMERIC amounts as strings with cents, DATE + * columns spanning the Febβ†’Mar 2024 month boundary, multiple currencies with + * seeded `exchange_rates` rows (so conversion resolves from the DB, never the + * network), transfers and inactive rows mixed in, and alias recipients. + * + * The materialized-view fast path is NOT exercised: the MVs are dropped by + * migration 0045 and recreated only at runtime by materializedViewService, so + * a freshly-migrated test DB always takes the live-query path (mvAvailable β†’ + * false). That matches a fresh production boot. + * + * Process-level caches (currency memoryCache/historical index, MV + * availability) are cleared around every test so no test sees another's rates. + */ + +import { afterAll, afterEach, beforeAll, describe, expect, it } from 'vitest'; +import { + acquireDbSuiteLock, + closeTestPool, + getTestPool, + hasTestDatabase, + releaseDbSuiteLock, +} from './setup/db.js'; +import { statisticsRepository } from '../src/repositories/infoRepositoryStatistics.js'; +import transactionRepository from '../src/repositories/transactionRepository.js'; +import { clearMvCache } from '../src/repositories/infoRepositoryHelpers.js'; +import { clearMemoryCache } from '../src/services/currency/currencyConversionService.js'; +import { closePool } from '../src/database/connection.js'; + +const cat = {}; +const rec = {}; + +async function seedBase() { + const pool = getTestPool(); + for (const [key, [general, detail]] of Object.entries({ + Food: ['Food', 'Groceries'], + Bills: ['Bills', 'Utilities'], + })) { + const { rows } = await pool.query( + 'INSERT INTO categories (general, detail) VALUES ($1, $2) RETURNING id', + [general, detail], + ); + cat[key] = rows[0].id; + } + const addRecipient = async (name, { defaultCategoryId = null, primaryId = null } = {}) => { + const { rows } = await pool.query( + `INSERT INTO recipients (name, normalized_name, default_category_id, primary_recipient_id) + VALUES ($1, $2, $3, $4) RETURNING id`, + [name, name.toLowerCase(), defaultCategoryId, primaryId], + ); + return rows[0].id; + }; + rec.aldi = await addRecipient('Aldi', { defaultCategoryId: cat.Food }); + rec.aldiAlias = await addRecipient('Aldi Anderlecht', { primaryId: rec.aldi }); + rec.misc = await addRecipient('Misc Payee'); +} + +/** + * Ensure an accounts row exists for a label. Pre-created because the sync + * trigger's own onboarding INSERT is broken at schema head (0076 regressed its + * ON CONFLICT arbiter to the raw name after 0066 dropped that constraint β€” + * pinned in transactionRepository.db.test.js); only the resolve path works. + */ +async function ensureAccount(name) { + await getTestPool().query( + `INSERT INTO accounts (name, display_name) VALUES ($1, $1) + ON CONFLICT (lower(btrim(name))) DO NOTHING`, + [name], + ); +} + +async function insertTxn({ + date, + amount, + currency = 'EUR', + recipientId, + categoryId = null, + bank = 'MAIN BANK', + isActive = true, + isTransfer = false, +}) { + if (bank) await ensureAccount(bank); + const { rows } = await getTestPool().query( + `INSERT INTO transactions (date, amount, currency, recipient_id, category_id, bank_account, is_active, is_transfer) + VALUES ($1, $2, $3, $4, $5, $6, $7, $8) RETURNING id`, + [date, amount, currency, recipientId, categoryId, bank, isActive, isTransfer], + ); + return rows[0].id; +} + +/** Seed one exchange_rates row; conversion then resolves from the DB. */ +async function insertRate(code, date, rate, isLatest = false) { + await getTestPool().query( + `INSERT INTO exchange_rates (currency_code, rate_date, rate_to_eur, is_latest) + VALUES ($1, $2, $3, $4)`, + [code, date, rate, isLatest], + ); +} + +describe.skipIf(!hasTestDatabase())('repositories/infoRepositoryStatistics (real DB)', () => { + beforeAll(async () => { + expect( + process.env.DATABASE_URL, + 'DATABASE_URL must equal TEST_DATABASE_URL for this suite (see scripts/with-test-db.sh)', + ).toBe(process.env.TEST_DATABASE_URL); + // DB suites share one database across parallel vitest workers β€” serialize. + await acquireDbSuiteLock(); + }, 180_000); + + afterEach(async () => { + const pool = getTestPool(); + await pool.query('DELETE FROM transactions'); + await pool.query('DELETE FROM accounts'); + await pool.query('DELETE FROM recipients'); + await pool.query('DELETE FROM categories'); + await pool.query('DELETE FROM exchange_rates'); + await pool.query(`DELETE FROM user_settings WHERE key = 'includeTransfers'`); + for (const bag of [cat, rec]) for (const k of Object.keys(bag)) delete bag[k]; + // Process-level caches: rates loaded from this test's exchange_rates rows + // (and the negative MV probe) must not leak into the next test. + clearMemoryCache(); + clearMvCache(); + }); + + afterAll(async () => { + await releaseDbSuiteLock(); + await closeTestPool(); + await closePool(); + }); + + // ─────────────────────────────────────────────────────────────────────────── + // getCategoryBreakdown (live path) + // ─────────────────────────────────────────────────────────────────────────── + describe('getCategoryBreakdown', () => { + it('aggregates per category across currencies, excluding transfers and inactive rows', async () => { + await seedBase(); + // USD converts at the stored latest rate (0.45 EUR per USD) β€” no network. + await insertRate('USD', '2024-06-01', '0.45', true); + + await insertTxn({ date: '2024-02-10', amount: '-10.00', recipientId: rec.aldi }); // Food via recipient default + await insertTxn({ date: '2024-02-15', amount: '-20.50', recipientId: rec.misc, categoryId: cat.Food }); + await insertTxn({ date: '2024-02-15', amount: '-30.00', currency: 'USD', recipientId: rec.misc, categoryId: cat.Food }); + await insertTxn({ date: '2024-03-05', amount: '-40.00', recipientId: rec.misc, categoryId: cat.Bills }); + await insertTxn({ date: '2024-03-06', amount: '-5.00', recipientId: rec.misc }); // no category anywhere + await insertTxn({ date: '2024-03-07', amount: '-2.00', recipientId: rec.misc }); // no category anywhere + await insertTxn({ date: '2024-03-08', amount: '-100.00', recipientId: rec.misc, categoryId: cat.Bills, isTransfer: true }); + await insertTxn({ date: '2024-03-09', amount: '-999.00', recipientId: rec.misc, categoryId: cat.Bills, isActive: false }); + + const r = await statisticsRepository.getCategoryBreakdown(); + + // Sorted by count DESC: Food (3), Uncategorised (2), Bills (1). + expect(r).toEqual([ + { id: cat.Food, name: 'Food:Groceries', count: 3, total: -44 }, // βˆ’10 βˆ’ 20.50 βˆ’ (30 Γ— 0.45) + { id: null, name: 'UNCATEGORISED', count: 2, total: -7 }, + { id: cat.Bills, name: 'Bills:Utilities', count: 1, total: -40 }, // transfer/inactive stay out + ]); + }); + + it('includes transfers when the includeTransfers setting is on', async () => { + await seedBase(); + await getTestPool().query( + `INSERT INTO user_settings (key, value) VALUES ('includeTransfers', 'true'::jsonb)`, + ); + await insertTxn({ date: '2024-03-05', amount: '-40.00', recipientId: rec.misc, categoryId: cat.Bills }); + await insertTxn({ date: '2024-03-08', amount: '-100.00', recipientId: rec.misc, categoryId: cat.Bills, isTransfer: true }); + + const r = await statisticsRepository.getCategoryBreakdown(); + expect(r).toEqual([ + { id: cat.Bills, name: 'Bills:Utilities', count: 2, total: -140 }, + ]); + }); + }); + + // ─────────────────────────────────────────────────────────────────────────── + // getBanks / getTransactionCount + // ─────────────────────────────────────────────────────────────────────────── + describe('getBanks', () => { + it('lists only accounts with ACTIVE transactions, ordered by name', async () => { + await seedBase(); + await insertTxn({ date: '2024-02-10', amount: '-1.00', recipientId: rec.misc, bank: 'ZZZ BANK' }); + await insertTxn({ date: '2024-02-11', amount: '-2.00', recipientId: rec.misc, bank: 'AAA BANK' }); + // The dual-write trigger creates the GHOST BANK account row, but its only + // transaction is inactive β€” so it must not surface. + await insertTxn({ date: '2024-02-12', amount: '-3.00', recipientId: rec.misc, bank: 'GHOST BANK', isActive: false }); + + expect(await statisticsRepository.getBanks()).toEqual(['AAA BANK', 'ZZZ BANK']); + }); + }); + + describe('getTransactionCount', () => { + it('counts active rows, optionally scoped to one account', async () => { + await seedBase(); + await insertTxn({ date: '2024-02-10', amount: '-1.00', recipientId: rec.misc, bank: 'MAIN BANK' }); + await insertTxn({ date: '2024-02-11', amount: '-2.00', recipientId: rec.misc, bank: 'MAIN BANK' }); + await insertTxn({ date: '2024-02-12', amount: '-3.00', recipientId: rec.misc, bank: 'OTHER BANK' }); + await insertTxn({ date: '2024-02-13', amount: '-4.00', recipientId: rec.misc, bank: 'MAIN BANK', isActive: false }); + const { rows } = await getTestPool().query( + `SELECT id FROM accounts WHERE name = 'MAIN BANK'`, + ); + + expect(await statisticsRepository.getTransactionCount()).toBe(3); + expect(await statisticsRepository.getTransactionCount({ accountId: rows[0].id })).toBe(2); + }); + }); + + // ─────────────────────────────────────────────────────────────────────────── + // getCategoryPivot + // ─────────────────────────────────────────────────────────────────────────── + describe('getCategoryPivot', () => { + it('splits income/expense within a mixed-sign category-month', async () => { + await seedBase(); + await insertTxn({ date: '2024-02-10', amount: '500.00', recipientId: rec.misc, categoryId: cat.Food }); + await insertTxn({ date: '2024-02-10', amount: '-300.00', recipientId: rec.misc, categoryId: cat.Food }); + + const r = await statisticsRepository.getCategoryPivot(); + expect(Object.keys(r.categoryPivot)).toEqual(['2024-02']); + expect(r.categoryPivot['2024-02']).toEqual([ + { + categoryId: cat.Food, + categoryName: 'Food: Groceries', // pivot format is 'general: detail' (with space) + total: 200, + income: 500, + expense: -300, + transactionCount: 2, + }, + ]); + }); + + it('converts each month at its own historical rate (per-date FX from exchange_rates)', async () => { + await seedBase(); + await insertRate('USD', '2024-02-15', '0.5'); + await insertRate('USD', '2024-03-15', '0.4'); + await insertTxn({ date: '2024-02-15', amount: '-30.00', currency: 'USD', recipientId: rec.misc, categoryId: cat.Food }); + await insertTxn({ date: '2024-03-15', amount: '-30.00', currency: 'USD', recipientId: rec.misc, categoryId: cat.Food }); + + const r = await statisticsRepository.getCategoryPivot(); + // Same nominal amount, different months, different rates: βˆ’15 vs βˆ’12. + expect(r.categoryPivot['2024-02']).toEqual([ + { categoryId: cat.Food, categoryName: 'Food: Groceries', total: -15, income: 0, expense: -15, transactionCount: 1 }, + ]); + expect(r.categoryPivot['2024-03']).toEqual([ + { categoryId: cat.Food, categoryName: 'Food: Groceries', total: -12, income: 0, expense: -12, transactionCount: 1 }, + ]); + }); + + it('sorts cells ascending by total and applies alias-aware exclusions', async () => { + await seedBase(); + // Recorded under the ALIAS with its OWN category. (Categories reached + // via the primary's default also enter the pivot β€” 3-level resolution, + // see the last test in this file.) + await insertTxn({ date: '2024-02-10', amount: '-10.00', recipientId: rec.aldiAlias, categoryId: cat.Food }); + await insertTxn({ date: '2024-02-12', amount: '-20.00', recipientId: rec.misc, categoryId: cat.Bills }); + + const all = await statisticsRepository.getCategoryPivot(); + expect(all.categoryPivot['2024-02'].map((c) => [c.categoryId, c.total])).toEqual([ + [cat.Bills, -20], // ascending by total + [cat.Food, -10], + ]); + + // Excluding the PRIMARY recipient also removes rows recorded under its alias. + const exclRecipient = await statisticsRepository.getCategoryPivot([], 'EUR', [rec.aldi]); + expect(exclRecipient.categoryPivot['2024-02'].map((c) => c.categoryId)).toEqual([cat.Bills]); + + const exclCategory = await statisticsRepository.getCategoryPivot([cat.Bills]); + expect(exclCategory.categoryPivot['2024-02'].map((c) => c.categoryId)).toEqual([cat.Food]); + }); + + // Regression coverage (formerly a pinned cross-surface inconsistency): + // both statistics queries used to resolve the effective category over TWO + // levels only β€” COALESCE(t.category_id, r.default_category_id) β€” while + // the transactions surfaces (list, GET, uncategorised queue) resolve + // THREE (…, pr.default_category_id). A row recorded under an alias whose + // PRIMARY carries the default category was therefore categorised in the + // transactions list but UNCATEGORISED in the breakdown and silently + // ABSENT from the pivot (whose WHERE requires the COALESCE non-NULL). + // Both queries now use the canonical 3-level pattern, so the breakdown + // and pivot must AGREE with the transactions list. (Rows uncategorised at + // ALL three levels are still excluded from the pivot by design, so the + // mock suite's "missing category_id β†’ Uncategorised" JS branch remains a + // defensive path never fed by the real query.) + it('alias row categorised only via its primary: Bills in breakdown and pivot, agreeing with the transactions list', async () => { + const pool = getTestPool(); + await seedBase(); + const { rows } = await pool.query( + `INSERT INTO recipients (name, normalized_name, default_category_id, primary_recipient_id) + VALUES ('Electrabel', 'electrabel', $1, NULL) RETURNING id`, + [cat.Bills], + ); + const aliasRes = await pool.query( + `INSERT INTO recipients (name, normalized_name, primary_recipient_id) + VALUES ('Electrabel Invoicing', 'electrabel invoicing', $1) RETURNING id`, + [rows[0].id], + ); + const txnId = await insertTxn({ date: '2024-02-10', amount: '-120.00', recipientId: aliasRes.rows[0].id }); + + // The transactions list categorises the row via the primary's default… + const listed = (await transactionRepository.getAll({})).find((t) => t.id === txnId); + expect(listed.effective_category_id).toBe(cat.Bills); + expect(listed.category_name).toBe('Bills:Utilities'); + + // …and the breakdown now agrees (formerly UNCATEGORISED here). + const breakdown = await statisticsRepository.getCategoryBreakdown(); + expect(breakdown).toEqual([ + { id: cat.Bills, name: 'Bills:Utilities', count: 1, total: -120 }, + ]); + + // …as does the pivot (the row formerly vanished from it entirely). + const pivot = await statisticsRepository.getCategoryPivot(); + expect(pivot.categoryPivot).toEqual({ + '2024-02': [ + { + categoryId: cat.Bills, + categoryName: 'Bills: Utilities', // pivot format is 'general: detail' + total: -120, + income: 0, + expense: -120, + transactionCount: 1, + }, + ], + }); + }); + }); +}); diff --git a/apps/node-backend/tests/infoRepositoryHelpers.test.js b/apps/node-backend/tests/infoRepositoryHelpers.test.js index 2d8c7cd0..2514ebfd 100644 --- a/apps/node-backend/tests/infoRepositoryHelpers.test.js +++ b/apps/node-backend/tests/infoRepositoryHelpers.test.js @@ -5,7 +5,11 @@ vi.mock('../src/database/connection.js', () => ({ })); import { query } from '../src/database/connection.js'; -import { mvAvailable, clearMvCache } from '../src/repositories/infoRepositoryHelpers.js'; +import { + mvAvailable, + clearMvCache, + sanitizeIsolatedDailyInvestmentSpikes, +} from '../src/repositories/infoRepositoryHelpers.js'; import { sanitizeIsolatedValueSpikes } from '../src/utils/portfolioMath.js'; describe('sanitizeIsolatedValueSpikes', () => { @@ -24,6 +28,44 @@ describe('sanitizeIsolatedValueSpikes', () => { }); }); +describe('sanitizeIsolatedDailyInvestmentSpikes', () => { + it('recomputes the corrected day netWorth including liabilities (liquid + liabilities + investments)', () => { + // Liabilities are stored as negative balances (ADR-092), exactly as the + // net-worth builder emits them: netWorth = liquid + liabilities + investments. + const snapshots = [ + { date: '2025-01-01', liquid: 500, liabilities: -200, investments: 1000, netWorth: 1300 }, + { date: '2025-01-02', liquid: 500, liabilities: -200, investments: 2000, netWorth: 2300 }, // isolated needle + { date: '2025-01-03', liquid: 500, liabilities: -200, investments: 1010, netWorth: 1310 }, + ]; + + const out = sanitizeIsolatedDailyInvestmentSpikes(snapshots); + + // Corrected investments: geometric mean sqrt(1000 * 1010) β‰ˆ 1004.99. + expect(out[1].investments).toBe(1004.99); + // Regression pin: netWorth must include the -200 liabilities term. + // The pre-fix recomputation (liquid + investments only) produced 1504.99. + expect(out[1].netWorth).toBe(1304.99); + + // Control: non-spike neighbor days are untouched, liabilities included. + expect(out[0]).toEqual({ date: '2025-01-01', liquid: 500, liabilities: -200, investments: 1000, netWorth: 1300 }); + expect(out[2]).toEqual({ date: '2025-01-03', liquid: 500, liabilities: -200, investments: 1010, netWorth: 1310 }); + + // Input is not mutated. + expect(snapshots[1].investments).toBe(2000); + expect(snapshots[1].netWorth).toBe(2300); + }); + + it('leaves a sustained investments move untouched', () => { + const snapshots = [ + { date: '2025-01-01', liquid: 500, liabilities: -200, investments: 1000, netWorth: 1300 }, + { date: '2025-01-02', liquid: 500, liabilities: -200, investments: 2000, netWorth: 2300 }, + { date: '2025-01-03', liquid: 500, liabilities: -200, investments: 2010, netWorth: 2310 }, + ]; + const out = sanitizeIsolatedDailyInvestmentSpikes(snapshots); + expect(out).toEqual(snapshots); // next does not revert β†’ not a needle + }); +}); + describe('mvAvailable', () => { beforeEach(() => { vi.clearAllMocks(); diff --git a/apps/node-backend/tests/investmentControllerValidation.test.js b/apps/node-backend/tests/investmentControllerValidation.test.js index 77ab44e4..c0ac1eaa 100644 --- a/apps/node-backend/tests/investmentControllerValidation.test.js +++ b/apps/node-backend/tests/investmentControllerValidation.test.js @@ -137,7 +137,16 @@ describe('createInvestment β€” numeric boundary pins', () => { describe('createInvestment β€” string width and currency pins', () => { it('accepts strings exactly at the column width and rejects one char over', async () => { - const widths = [['name', 200], ['symbol', 20], ['location', 300], ['municipality', 200]]; + const widths = [ + ['name', 200], ['symbol', 20], ['location', 300], ['municipality', 200], + // Provider columns: URL shape is validated separately; the width guard + // stops an over-length-but-valid value 22001-ing at the VARCHAR column. + ['price_provider_id', 200], + ['price_provider_url', 500], ['price_provider_latest_url', 500], + ['price_provider_latest_path', 300], ['price_provider_history_url', 500], + ['price_provider_history_path', 300], ['price_provider_history_ts_path', 300], + ['price_provider_history_price_path', 300], + ]; for (const [field, max] of widths) { await createInvestment(createReq({ [field]: 'x'.repeat(max) }), mockRes()); await expect(createInvestment(createReq({ [field]: 'x'.repeat(max + 1) }), mockRes())) diff --git a/apps/node-backend/tests/investmentRepository.test.js b/apps/node-backend/tests/investmentRepository.test.js index d651276e..37b867be 100644 --- a/apps/node-backend/tests/investmentRepository.test.js +++ b/apps/node-backend/tests/investmentRepository.test.js @@ -14,6 +14,7 @@ describe('investmentRepository.create', () => { it('creates through legacy investments table when inheritance schema is absent', async () => { query + .mockResolvedValueOnce({ rows: [] }) // symbol-uniqueness check (no duplicate) .mockResolvedValueOnce({ rows: [{ investments_base: null }] }) .mockResolvedValueOnce({ rows: [{ id: 1, name: 'BTC', asset_class: 'crypto' }] }); @@ -27,10 +28,15 @@ describe('investmentRepository.create', () => { expect(query).toHaveBeenNthCalledWith( 1, - "SELECT to_regclass('public.investments_base') AS investments_base" + 'SELECT id FROM investments WHERE LOWER(symbol) = LOWER($1) AND id <> $2 LIMIT 1', + ['BTC', 0] ); expect(query).toHaveBeenNthCalledWith( 2, + "SELECT to_regclass('public.investments_base') AS investments_base" + ); + expect(query).toHaveBeenNthCalledWith( + 3, `INSERT INTO investments (name, symbol, asset_class, currency, current_price, interest_rate, maturity_date, location, municipality, cadastral_income, municipality_tax_rate, notes, price_provider, price_provider_id, price_provider_url, price_provider_latest_url, price_provider_latest_path, price_provider_history_url, price_provider_history_path, price_provider_history_ts_path, price_provider_history_price_path) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14, $15, $16, $17, $18, $19, $20, $21) RETURNING *`, ['BTC', 'BTC', 'crypto', 'EUR', 50000, null, null, null, null, null, null, null, 'manual', null, null, null, null, null, null, null, null] @@ -40,6 +46,7 @@ describe('investmentRepository.create', () => { it('falls back to inheritance tables when insert into investments view is not updatable', async () => { query + .mockResolvedValueOnce({ rows: [] }) // symbol-uniqueness check (no duplicate) .mockResolvedValueOnce({ rows: [{ investments_base: null }] }) .mockRejectedValueOnce({ message: 'cannot insert into view "investments"', code: '55000' }) .mockResolvedValueOnce({ rows: [{ id: 17 }] }) @@ -57,22 +64,23 @@ describe('investmentRepository.create', () => { }); expect(query).toHaveBeenNthCalledWith( - 2, + 3, `INSERT INTO investments (name, symbol, asset_class, currency, current_price, interest_rate, maturity_date, location, municipality, cadastral_income, municipality_tax_rate, notes, price_provider, price_provider_id, price_provider_url, price_provider_latest_url, price_provider_latest_path, price_provider_history_url, price_provider_history_path, price_provider_history_ts_path, price_provider_history_price_path) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14, $15, $16, $17, $18, $19, $20, $21) RETURNING *`, ['AAPL', 'AAPL', 'stock', 'USD', 180.5, null, null, null, null, null, null, 'Tech stock', 'yahoo', 'AAPL', null, null, null, null, null, null, null] ); expect(query).toHaveBeenNthCalledWith( - 3, + 4, 'INSERT INTO stock_investments (name, currency, notes, price_provider, price_provider_id, price_provider_url, price_provider_latest_url, price_provider_latest_path, price_provider_history_url, price_provider_history_path, price_provider_history_ts_path, price_provider_history_price_path, symbol, current_price) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14) RETURNING id', ['AAPL', 'USD', 'Tech stock', 'yahoo', 'AAPL', null, null, null, null, null, null, null, 'AAPL', 180.5] ); - expect(query).toHaveBeenNthCalledWith(4, 'SELECT i.*, COALESCE(tp.show_in_ticker, true) AS show_in_ticker FROM investments i LEFT JOIN investment_ticker_prefs tp ON tp.investment_id = i.id WHERE i.id = $1', [17]); + expect(query).toHaveBeenNthCalledWith(5, 'SELECT i.*, COALESCE(tp.show_in_ticker, true) AS show_in_ticker FROM investments i LEFT JOIN investment_ticker_prefs tp ON tp.investment_id = i.id WHERE i.id = $1', [17]); expect(result).toEqual({ id: 17, asset_class: 'stock', name: 'AAPL' }); }); it('falls back to legacy insert columns when modern provider columns are missing', async () => { query + .mockResolvedValueOnce({ rows: [] }) // symbol-uniqueness check (no duplicate) .mockResolvedValueOnce({ rows: [{ investments_base: null }] }) .mockRejectedValueOnce({ code: '42703', @@ -92,7 +100,7 @@ describe('investmentRepository.create', () => { }); expect(query).toHaveBeenNthCalledWith( - 3, + 4, `INSERT INTO investments (name, symbol, asset_class, currency, current_price, interest_rate, maturity_date, location, municipality, cadastral_income, municipality_tax_rate, notes, price_provider, price_provider_id, price_provider_url) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14, $15) RETURNING *`, ['Napoleon', 'NAPOLEON', 'metals', 'EUR', 706.5, null, null, null, null, null, null, null, 'custom', 'napoleon.price', 'https://example.com/latest'] @@ -102,6 +110,7 @@ describe('investmentRepository.create', () => { it('falls back to inheritance create when legacy insert also hits non-updatable investments view', async () => { query + .mockResolvedValueOnce({ rows: [] }) // symbol-uniqueness check (no duplicate) .mockResolvedValueOnce({ rows: [{ investments_base: null }] }) .mockRejectedValueOnce({ code: '42703', @@ -131,11 +140,11 @@ describe('investmentRepository.create', () => { }); expect(query).toHaveBeenNthCalledWith( - 5, + 6, 'INSERT INTO stock_investments (name, currency, notes, price_provider, price_provider_id, price_provider_url, price_provider_latest_url, price_provider_latest_path, price_provider_history_url, price_provider_history_path, price_provider_history_ts_path, price_provider_history_price_path, symbol, current_price) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14) RETURNING id', ['Napoleon 20F', 'EUR', null, 'custom', null, null, 'https://example.com/latest', 'napoleon.price', 'https://example.com/history', 'points', 'timestamp_ms', 'price', 'NAPOLEON', 706.5] ); - expect(query).toHaveBeenNthCalledWith(6, 'SELECT i.*, COALESCE(tp.show_in_ticker, true) AS show_in_ticker FROM investments i LEFT JOIN investment_ticker_prefs tp ON tp.investment_id = i.id WHERE i.id = $1', [99]); + expect(query).toHaveBeenNthCalledWith(7, 'SELECT i.*, COALESCE(tp.show_in_ticker, true) AS show_in_ticker FROM investments i LEFT JOIN investment_ticker_prefs tp ON tp.investment_id = i.id WHERE i.id = $1', [99]); expect(result).toEqual({ id: 99, asset_class: 'metals', name: 'Napoleon 20F' }); }); @@ -167,6 +176,7 @@ describe('investmentRepository.create', () => { it('falls back to legacy inheritance columns when child table misses modern provider columns', async () => { query + .mockResolvedValueOnce({ rows: [] }) // symbol-uniqueness check (no duplicate) .mockResolvedValueOnce({ rows: [{ investments_base: 'investments_base' }] }) .mockResolvedValueOnce({ rows: [{ metals_investments: null }] }) .mockRejectedValueOnce({ @@ -188,16 +198,17 @@ describe('investmentRepository.create', () => { }); expect(query).toHaveBeenNthCalledWith( - 4, + 5, 'INSERT INTO stock_investments (name, currency, notes, price_provider, price_provider_id, price_provider_url, symbol, current_price) VALUES ($1, $2, $3, $4, $5, $6, $7, $8) RETURNING id', ['Napoleon 20F', 'EUR', null, 'custom', 'napoleon.price', 'https://example.com/latest', 'NAPOLEON', 705.2] ); - expect(query).toHaveBeenNthCalledWith(5, 'SELECT i.*, COALESCE(tp.show_in_ticker, true) AS show_in_ticker FROM investments i LEFT JOIN investment_ticker_prefs tp ON tp.investment_id = i.id WHERE i.id = $1', [44]); + expect(query).toHaveBeenNthCalledWith(6, 'SELECT i.*, COALESCE(tp.show_in_ticker, true) AS show_in_ticker FROM investments i LEFT JOIN investment_ticker_prefs tp ON tp.investment_id = i.id WHERE i.id = $1', [44]); expect(result).toEqual({ id: 44, asset_class: 'metals', name: 'Napoleon 20F' }); }); it('resyncs investments_base sequence and retries when inherited insert hits duplicate id', async () => { query + .mockResolvedValueOnce({ rows: [] }) // symbol-uniqueness check (no duplicate) .mockResolvedValueOnce({ rows: [{ investments_base: 'investments_base' }] }) .mockRejectedValueOnce({ code: '23505', @@ -218,24 +229,63 @@ describe('investmentRepository.create', () => { }); expect(query).toHaveBeenNthCalledWith( - 2, + 3, 'INSERT INTO stock_investments (name, currency, notes, price_provider, price_provider_id, price_provider_url, price_provider_latest_url, price_provider_latest_path, price_provider_history_url, price_provider_history_path, price_provider_history_ts_path, price_provider_history_price_path, symbol, current_price) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14) RETURNING id', ['AAPL', 'USD', null, 'manual', null, null, null, null, null, null, null, null, 'AAPL', 180.5] ); expect(query).toHaveBeenNthCalledWith( - 3, + 4, "SELECT setval(pg_get_serial_sequence('investments_base', 'id'), COALESCE((SELECT MAX(id) FROM investments_base), 0) + 1, false)" ); expect(query).toHaveBeenNthCalledWith( - 4, + 5, 'INSERT INTO stock_investments (name, currency, notes, price_provider, price_provider_id, price_provider_url, price_provider_latest_url, price_provider_latest_path, price_provider_history_url, price_provider_history_path, price_provider_history_ts_path, price_provider_history_price_path, symbol, current_price) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14) RETURNING id', ['AAPL', 'USD', null, 'manual', null, null, null, null, null, null, null, null, 'AAPL', 180.5] ); - expect(query).toHaveBeenNthCalledWith(5, 'SELECT i.*, COALESCE(tp.show_in_ticker, true) AS show_in_ticker FROM investments i LEFT JOIN investment_ticker_prefs tp ON tp.investment_id = i.id WHERE i.id = $1', [42]); + expect(query).toHaveBeenNthCalledWith(6, 'SELECT i.*, COALESCE(tp.show_in_ticker, true) AS show_in_ticker FROM investments i LEFT JOIN investment_ticker_prefs tp ON tp.investment_id = i.id WHERE i.id = $1', [42]); expect(result).toEqual({ id: 42, asset_class: 'stock', name: 'AAPL' }); }); }); +describe('investmentRepository.create symbol uniqueness', () => { + beforeEach(() => { + vi.resetAllMocks(); + __resetInvestmentSchemaCache(); + }); + + it('rejects a duplicate symbol on create with the same error shape as update', async () => { + query.mockResolvedValueOnce({ rows: [{ id: 7 }] }); // uniqueness check finds a match + + await expect( + investmentRepository.create({ name: 'Apple', symbol: ' aapl ', asset_class: 'stock', currency: 'USD' }) + ).rejects.toMatchObject({ code: 'VALIDATION_ERROR', message: 'symbol must be unique' }); + + // Normalized (trim/uppercase) before the check; excludeId 0 matches no row. + expect(query).toHaveBeenCalledWith( + 'SELECT id FROM investments WHERE LOWER(symbol) = LOWER($1) AND id <> $2 LIMIT 1', + ['AAPL', 0] + ); + expect(query).toHaveBeenCalledTimes(1); // nothing written + }); + + it('skips the uniqueness check when no symbol is provided', async () => { + query + .mockResolvedValueOnce({ rows: [{ investments_base: 'investments_base' }] }) + .mockResolvedValueOnce({ rows: [{ id: 12 }] }) + .mockResolvedValueOnce({ rows: [{ id: 12, asset_class: 'savings', name: 'Book' }] }); + + const result = await investmentRepository.create({ + name: 'Book', asset_class: 'savings', currency: 'EUR', interest_rate: 2, + }); + + expect(result).toEqual({ id: 12, asset_class: 'savings', name: 'Book' }); + expect(query).not.toHaveBeenCalledWith( + 'SELECT id FROM investments WHERE LOWER(symbol) = LOWER($1) AND id <> $2 LIMIT 1', + expect.anything() + ); + }); +}); + describe('investmentRepository.update', () => { beforeEach(() => { vi.resetAllMocks(); diff --git a/apps/node-backend/tests/routes/info.test.js b/apps/node-backend/tests/routes/info.test.js index 49a2e124..5f916391 100644 --- a/apps/node-backend/tests/routes/info.test.js +++ b/apps/node-backend/tests/routes/info.test.js @@ -107,13 +107,13 @@ describe('Info Routes', () => { // Registry-derived: one entry per non-generic adapter, keyed by name with // the adapter's bankName label. Adding an adapter exposes it automatically. - const keys = data.adapters.map((a) => a.key); + const keys = data.items.map((a) => a.key); expect(keys).toContain('bnp'); expect(keys).toContain('wise'); expect(keys).not.toContain('generic'); - expect(data.total_count).toBe(data.adapters.length); + expect(data.total).toBe(data.items.length); // bankName label, not a hardcoded display string / nonexistent class name. - const bnp = data.adapters.find((a) => a.key === 'bnp'); + const bnp = data.items.find((a) => a.key === 'bnp'); expect(bnp.name).toBe('BNP Paribas Fortis'); expect(bnp.adapter_class).toBeUndefined(); }); @@ -127,7 +127,7 @@ describe('Info Routes', () => { const res = mockResponse(); await routeHandlers['get:/banks'](req, res); - expect(res.json.mock.calls[0][0].data.banks).toHaveLength(2); + expect(res.json.mock.calls[0][0].data.items).toHaveLength(2); }); it('should return empty for no banks', async () => { @@ -137,7 +137,7 @@ describe('Info Routes', () => { const res = mockResponse(); await routeHandlers['get:/banks'](req, res); - expect(res.json.mock.calls[0][0].data.banks).toEqual([]); + expect(res.json.mock.calls[0][0].data).toEqual({ items: [], total: 0 }); }); it('should handle database errors', async () => { @@ -190,8 +190,8 @@ describe('Info Routes', () => { await routeHandlers['get:/supported-adapters'](req, res); const result = res.json.mock.calls[0][0].data; - expect(result.adapters).toBeDefined(); - expect(result.total_count).toBeGreaterThan(0); + expect(result.items).toBeDefined(); + expect(result.total).toBeGreaterThan(0); }); }); diff --git a/apps/node-backend/tests/routes/investments.test.js b/apps/node-backend/tests/routes/investments.test.js index 4f0ef0c3..92b1e120 100644 --- a/apps/node-backend/tests/routes/investments.test.js +++ b/apps/node-backend/tests/routes/investments.test.js @@ -670,6 +670,26 @@ describe('Investment Routes', () => { const res = mockResponse(); await expect(routeHandlers['patch:/transactions/:txnId'](req, res)).rejects.toBeInstanceOf(ValidationError); }); + + it('rejects a free-text or cleared currency and uppercases a valid one', async () => { + // PATCH forwarded the raw value to the VARCHAR(10) column (create + // validates): garbage stored, >10 chars 22001'd, null hit NOT NULL. + for (const currency of ['euro', null, '']) { + const req = { params: { txnId: '1' }, body: { currency } }; + await expect(routeHandlers['patch:/transactions/:txnId'](req, mockResponse())) + .rejects.toBeInstanceOf(ValidationError); + } + expect(portfolioTransactionRepository.update).not.toHaveBeenCalled(); + + portfolioTransactionRepository.update.mockResolvedValue({ id: 1, investment_id: 10, currency: 'USD' }); + // fx_rate_to_eur supplied explicitly β†’ the fx recompute path is skipped. + const req = { params: { txnId: '1' }, body: { currency: 'usd', fx_rate_to_eur: 0.9 } }; + await routeHandlers['patch:/transactions/:txnId'](req, mockResponse()); + expect(portfolioTransactionRepository.update).toHaveBeenCalledWith( + 1, + expect.objectContaining({ currency: 'USD' }), + ); + }); }); // ── GET /api/investments/:id/summary ─────────────────────── diff --git a/apps/node-backend/tests/routes/marketLookup.test.js b/apps/node-backend/tests/routes/marketLookup.test.js index 437270a0..5939dcd0 100644 --- a/apps/node-backend/tests/routes/marketLookup.test.js +++ b/apps/node-backend/tests/routes/marketLookup.test.js @@ -98,8 +98,8 @@ describe('Market Lookup Routes', () => { await routeHandlers['get:/quote'](req, res); const body = res.json.mock.calls[0][0]; - expect(body.data.quotes).toHaveLength(1); - expect(body.data.quotes[0]).toMatchObject({ + expect(body.data.items).toHaveLength(1); + expect(body.data.items[0]).toMatchObject({ symbol: 'AAPL', name: 'Apple Inc.', marketCap: 300, @@ -110,14 +110,14 @@ describe('Market Lookup Routes', () => { beta: 1.1, priceToBook: 4.4, }); - expect(body.data.quotes[0].analystConsensus).toEqual({ + expect(body.data.items[0].analystConsensus).toEqual({ strongBuy: 4, buy: 10, hold: 3, sell: 1, strongSell: 0, }); - expect(body.data.quotes[0].recentAnalystActions).toHaveLength(1); + expect(body.data.items[0].recentAnalystActions).toHaveLength(1); }); it('detail=basic returns price-only fields and skips the quoteSummary call', async () => { @@ -137,7 +137,7 @@ describe('Market Lookup Routes', () => { await routeHandlers['get:/quote'](req, res); expect(mockYahooQuoteSummary).not.toHaveBeenCalled(); - const quote = res.json.mock.calls[0][0].data.quotes[0]; + const quote = res.json.mock.calls[0][0].data.items[0]; expect(quote).toMatchObject({ symbol: 'AAPL', name: 'Apple Inc.', @@ -160,7 +160,7 @@ describe('Market Lookup Routes', () => { await routeHandlers['get:/quote'](req, res); - expect(res.json).toHaveBeenCalledWith({ ok: true, data: { quotes: [] } }); + expect(res.json).toHaveBeenCalledWith({ ok: true, data: { items: [], total: 0 } }); }); }); @@ -279,7 +279,7 @@ describe('Market Lookup Routes', () => { await routeHandlers['get:/chart'](req, res); - expect(res.json).toHaveBeenCalledWith({ ok: true, data: { points: [] } }); + expect(res.json).toHaveBeenCalledWith({ ok: true, data: { items: [], total: 0 } }); }); it('maps chart points and filters null closes', async () => { @@ -299,8 +299,8 @@ describe('Market Lookup Routes', () => { const payload = res.json.mock.calls[0][0].data; expect(payload.symbol).toBe('AAPL'); expect(payload.currency).toBe('USD'); - expect(payload.points).toHaveLength(1); - expect(payload.points[0]).toMatchObject({ close: 200, high: 201, low: 199, volume: 10 }); + expect(payload.items).toHaveLength(1); + expect(payload.items[0]).toMatchObject({ close: 200, high: 201, low: 199, volume: 10 }); }); it('throws AppError (502) when chart request crashes', async () => { @@ -330,7 +330,7 @@ describe('Market Lookup Routes', () => { { validateResult: false }, ); const payload = res.json.mock.calls[0][0].data; - expect(payload.points).toHaveLength(1); + expect(payload.items).toHaveLength(1); }); }); @@ -373,11 +373,11 @@ describe('Market Lookup Routes', () => { await routeHandlers['get:/news'](req, res); const body = res.json.mock.calls[0][0].data; - expect(body.articles).toHaveLength(2); - expect(body.articles[0].title).toBe('Unique headline'); - expect(body.articles[0].thumbnail).toBe('https://img.example.com/c.jpg'); - expect(body.articles[1].title).toBe('Shared headline'); - expect(body.articles[1].thumbnail).toBe('https://img.example.com/a.jpg'); + expect(body.items).toHaveLength(2); + expect(body.items[0].title).toBe('Unique headline'); + expect(body.items[0].thumbnail).toBe('https://img.example.com/c.jpg'); + expect(body.items[1].title).toBe('Shared headline'); + expect(body.items[1].thumbnail).toBe('https://img.example.com/a.jpg'); }); it('should tolerate yahoo search failures and return empty results', async () => { @@ -389,7 +389,7 @@ describe('Market Lookup Routes', () => { await routeHandlers['get:/news'](req, res); expect(res.status).not.toHaveBeenCalled(); - expect(res.json).toHaveBeenCalledWith({ ok: true, data: { articles: [] } }); + expect(res.json).toHaveBeenCalledWith({ ok: true, data: { items: [], total: 0 } }); }); it('uses default symbols and caps count at 50', async () => { @@ -450,7 +450,7 @@ describe('Market Lookup Routes', () => { await routeHandlers['get:/quote'](req, res); - const quote = res.json.mock.calls[0][0].data.quotes[0]; + const quote = res.json.mock.calls[0][0].data.items[0]; expect(quote.name).toBe('MSFT'); expect(quote.currency).toBe('USD'); expect(quote.analystConsensus).toEqual({ strongBuy: 1, buy: 2, hold: 3, sell: 4, strongSell: 5 }); @@ -471,7 +471,7 @@ describe('Market Lookup Routes', () => { await routeHandlers['get:/quote'](req, res); - const quote = res.json.mock.calls[0][0].data.quotes[0]; + const quote = res.json.mock.calls[0][0].data.items[0]; expect(quote.symbol).toBe('NVDA'); expect(quote.price).toBe(900); expect(quote.analystConsensus).toBeNull(); @@ -514,7 +514,7 @@ describe('Market Lookup Routes', () => { await routeHandlers['get:/news'](req, res); - const article = res.json.mock.calls[0][0].data.articles[0]; + const article = res.json.mock.calls[0][0].data.items[0]; expect(article.thumbnail).toBeNull(); }); }); diff --git a/apps/node-backend/tests/routes/plannedTransactions.test.js b/apps/node-backend/tests/routes/plannedTransactions.test.js index fbfa06f0..acdd6882 100644 --- a/apps/node-backend/tests/routes/plannedTransactions.test.js +++ b/apps/node-backend/tests/routes/plannedTransactions.test.js @@ -386,6 +386,28 @@ describe('Planned Transaction Routes', () => { .rejects.toBeInstanceOf(ValidationError); }); + it('normalises currency to uppercase ISO and rejects free text', async () => { + // Free-typed "euro" used to be uppercased to "EURO" by the repository and + // then violate the 0046 ISO CHECK as a raw 500. + const badReq = { body: { ...validBody, currency: 'euro' } }; + await expect(routeHandlers['post:/'](badReq, mockResponse())).rejects.toBeInstanceOf(ValidationError); + expect(plannedTransactionRepository.create).not.toHaveBeenCalled(); + + const okReq = { body: { ...validBody, currency: 'usd' } }; + await routeHandlers['post:/'](okReq, mockResponse()); + expect(plannedTransactionRepository.create).toHaveBeenCalledWith( + expect.objectContaining({ currency: 'USD' }), + ); + }); + + it('maps an absent/empty currency to undefined so the repository default (EUR) applies', async () => { + const req = { body: { ...validBody, currency: '' } }; + await routeHandlers['post:/'](req, mockResponse()); + expect(plannedTransactionRepository.create).toHaveBeenCalledWith( + expect.objectContaining({ currency: undefined }), + ); + }); + it('drops truthy recurrence bounds on a loan instead of validating them', async () => { const req = { body: { @@ -449,6 +471,71 @@ describe('Planned Transaction Routes', () => { expect(plannedTransactionRepository.update).not.toHaveBeenCalled(); }); + it('normalises a valid currency and rejects free-text or cleared currency on PATCH', async () => { + // PATCH forwarded the raw value to the SET builder, so "euro" hit the + // 0046 ISO CHECK as a raw 500 and null hit the NOT NULL constraint. + for (const body of [{ currency: 'euro' }, { currency: null }, { currency: '' }]) { + await expect(routeHandlers['patch:/:id']({ params: { id: '1' }, body }, mockResponse())) + .rejects.toBeInstanceOf(ValidationError); + } + expect(plannedTransactionRepository.update).not.toHaveBeenCalled(); + + await routeHandlers['patch:/:id']({ params: { id: '1' }, body: { currency: 'usd' } }, mockResponse()); + expect(plannedTransactionRepository.update).toHaveBeenCalledWith( + 1, + expect.objectContaining({ currency: 'USD' }), + ); + }); + + it('coerces a valid amount and rejects zero/absurd/non-finite/cleared amounts on PATCH', async () => { + for (const body of [ + { amount: 0 }, + { amount: 1e15 }, + { amount: 'Infinity' }, + { amount: null }, + { amount: '' }, + ]) { + await expect(routeHandlers['patch:/:id']({ params: { id: '1' }, body }, mockResponse())) + .rejects.toBeInstanceOf(ValidationError); + } + expect(plannedTransactionRepository.update).not.toHaveBeenCalled(); + + await routeHandlers['patch:/:id']({ params: { id: '1' }, body: { amount: '-42.50' } }, mockResponse()); + expect(plannedTransactionRepository.update).toHaveBeenCalledWith( + 1, + expect.objectContaining({ amount: -42.5 }), + ); + }); + + it('rejects turning recurrence on (or clearing the pattern) when the merged state has no valid pattern', async () => { + // is_recurring:true on a row without a pattern used to store and leave + // the row perpetually due after /execute (the POST guard has an exact + // sibling); clearing the pattern on a recurring row recreated it. + await expect(routeHandlers['patch:/:id']({ params: { id: '1' }, body: { is_recurring: true } }, mockResponse())) + .rejects.toBeInstanceOf(ValidationError); + + plannedTransactionRepository.getById.mockResolvedValue({ id: 1, is_loan: false, is_recurring: true, recurrence_pattern: 'monthly' }); + await expect(routeHandlers['patch:/:id']({ params: { id: '1' }, body: { recurrence_pattern: null } }, mockResponse())) + .rejects.toBeInstanceOf(ValidationError); + expect(plannedTransactionRepository.update).not.toHaveBeenCalled(); + + // Turning recurrence on WITH a valid pattern still passes. + plannedTransactionRepository.getById.mockResolvedValue({ id: 1, is_loan: false }); + await routeHandlers['patch:/:id']({ params: { id: '1' }, body: { is_recurring: true, recurrence_pattern: 'monthly' } }, mockResponse()); + expect(plannedTransactionRepository.update).toHaveBeenCalledWith( + 1, + expect.objectContaining({ is_recurring: true, recurrence_pattern: 'monthly' }), + ); + + // An unrelated edit to a legacy broken row (recurring, no pattern) is NOT blocked. + plannedTransactionRepository.getById.mockResolvedValue({ id: 1, is_loan: false, is_recurring: true, recurrence_pattern: null }); + await routeHandlers['patch:/:id']({ params: { id: '1' }, body: { memo: 'still editable' } }, mockResponse()); + expect(plannedTransactionRepository.update).toHaveBeenCalledWith( + 1, + expect.objectContaining({ memo: 'still editable' }), + ); + }); + it('passes explicit nulls through to clear recurrence bounds and pattern', async () => { const req = { params: { id: '1' }, @@ -539,6 +626,92 @@ describe('Planned Transaction Routes', () => { expect(res.json).toHaveBeenCalled(); }); + // Sign/amount derivation pins: whenever the repayment schedule is + // (re)generated by a PATCH, `amount` MUST be re-derived server-side as + // -|regular_payment_amount| β€” a defined client amount is ignored, exactly + // like POST. Keeping the client value desynced amount from + // loan_regular_payment_amount. (10000 @ 6% / 12 months β†’ 860.66, same + // real-schedule figure the POST test pins.) + const existingLoan = { + id: 1, + is_loan: true, + loan_type: 'amortizing', + loan_principal: 5000, + loan_annual_interest_rate: 3, + loan_term_months: 24, + loan_start_date: '2026-04-01', + loan_payment_day: 1, + }; + + it('re-derives amount from the regenerated schedule even when the client sends a stale amount', async () => { + plannedTransactionRepository.getById.mockResolvedValueOnce(existingLoan); + plannedTransactionRepository.updateWithLoanSchedule.mockResolvedValue({ id: 1, is_loan: true }); + + const req = { + params: { id: '1' }, + // Client edits the principal but sends its stale (pre-regeneration) amount. + body: { + loan_principal: 10000, + loan_annual_interest_rate: 6, + loan_term_months: 12, + amount: -214.03, + }, + }; + await routeHandlers['patch:/:id'](req, mockResponse()); + + expect(plannedTransactionRepository.updateWithLoanSchedule).toHaveBeenCalledWith( + 1, + expect.objectContaining({ + amount: -860.66, + loan_regular_payment_amount: 860.66, + }), + expect.any(Array), + ); + }); + + it('re-derives (and force-negates) amount on convert-to-loan even when the client sends a positive amount', async () => { + plannedTransactionRepository.getById.mockResolvedValueOnce({ id: 1, is_loan: false }); + plannedTransactionRepository.updateWithLoanSchedule.mockResolvedValue({ id: 1, is_loan: true }); + + const req = { + params: { id: '1' }, + body: { + is_loan: true, + loan_type: 'amortizing', + loan_principal: 10000, + loan_annual_interest_rate: 6, + loan_term_months: 12, + loan_start_date: '2026-04-01', + loan_payment_day: 1, + amount: 500, // stale client value, wrong sign β€” must be ignored + }, + }; + await routeHandlers['patch:/:id'](req, mockResponse()); + + expect(plannedTransactionRepository.updateWithLoanSchedule).toHaveBeenCalledWith( + 1, + expect.objectContaining({ amount: -860.66 }), + expect.any(Array), + ); + }); + + it('keeps a client amount on a loan PATCH that touches no schedule input', async () => { + // No loan field changed and is_loan not re-asserted β†’ the schedule is not + // regenerated, so the client's amount passes through untouched (boundary + // of the re-derivation rule). + plannedTransactionRepository.getById.mockResolvedValueOnce(existingLoan); + plannedTransactionRepository.update.mockResolvedValue({ id: 1, is_loan: true }); + + const req = { params: { id: '1' }, body: { memo: 'note', amount: -123.45 } }; + await routeHandlers['patch:/:id'](req, mockResponse()); + + expect(plannedTransactionRepository.updateWithLoanSchedule).not.toHaveBeenCalled(); + expect(plannedTransactionRepository.update).toHaveBeenCalledWith( + 1, + expect.objectContaining({ amount: -123.45 }), + ); + }); + it('should clear loan fields and loan schedule atomically when toggled off', async () => { plannedTransactionRepository.getById.mockResolvedValueOnce({ id: 1, diff --git a/apps/node-backend/tests/routes/recipientBankAccounts.test.js b/apps/node-backend/tests/routes/recipientBankAccounts.test.js index 1fcfa6eb..71e61160 100644 --- a/apps/node-backend/tests/routes/recipientBankAccounts.test.js +++ b/apps/node-backend/tests/routes/recipientBankAccounts.test.js @@ -25,7 +25,9 @@ vi.mock('../../src/repositories/recipientBankAccountRepository.js', () => ({ }, })); -vi.mock('../../src/middleware/validation.js', () => ({ +vi.mock('../../src/middleware/validation.js', async (importOriginal) => ({ + // Keep the real helpers (assertMaxLength, …); only stub the middleware. + ...(await importOriginal()), validateIdParam: (req, res, next) => next(), })); @@ -108,6 +110,30 @@ describe('Recipient Bank Account Routes', () => { const res = mockResponse(); await expect(routeHandlers['post:/:id/bank-accounts'](req, res)).rejects.toBeInstanceOf(ValidationError); }); + + it('rejects an account_number longer than the VARCHAR(34) column (was a raw 22001 500)', async () => { + const req = { params: { id: '1' }, body: { account_number: 'X'.repeat(35) } }; + const res = mockResponse(); + await expect(routeHandlers['post:/:id/bank-accounts'](req, res)).rejects.toBeInstanceOf(ValidationError); + expect(bankAccountRepo.createOrGet).not.toHaveBeenCalled(); + }); + + it('accepts an account_number at the 34-char boundary (IBAN max)', async () => { + const acct = 'X'.repeat(34); + bankAccountRepo.createOrGet.mockResolvedValue({ + bankAccount: { id: 2, account_number: acct }, + created: true, + }); + + const req = { params: { id: '1' }, body: { account_number: acct } }; + const res = mockResponse(); + await routeHandlers['post:/:id/bank-accounts'](req, res); + + expect(res.status).toHaveBeenCalledWith(201); + expect(bankAccountRepo.createOrGet).toHaveBeenCalledWith( + expect.objectContaining({ accountNumber: acct }), + ); + }); }); describe('PATCH /:recipientId/bank-accounts/:accountId', () => { diff --git a/apps/node-backend/tests/routes/settings.test.js b/apps/node-backend/tests/routes/settings.test.js index 83647734..0d83c48f 100644 --- a/apps/node-backend/tests/routes/settings.test.js +++ b/apps/node-backend/tests/routes/settings.test.js @@ -248,6 +248,43 @@ describe('Settings Routes', () => { }); }); + it('rejects an unknown setting key with a 400 naming the known keys', async () => { + const req = { params: { key: 'totally_unknown_key' }, body: { value: { any: 'json' } } }; + const res = mockResponse(); + + await expect(routeHandlers['put:/:key'](req, res)).rejects.toBeInstanceOf(ValidationError); + await expect(routeHandlers['put:/:key'](req, res)).rejects.toThrow(/Unknown setting key 'totally_unknown_key'.*Known keys:/); + expect(settingsRepository.set).not.toHaveBeenCalled(); + }); + + it('accepts dismissed_recurring_patterns as an array (RecurringDetectionPanel payload)', async () => { + settingsRepository.set.mockResolvedValue({ key: 'dismissed_recurring_patterns', value: [3, 7] }); + + const req = { params: { key: 'dismissed_recurring_patterns' }, body: { value: [3, 7] } }; + const res = mockResponse(); + await routeHandlers['put:/:key'](req, res); + + expect(settingsRepository.set).toHaveBeenCalledWith('dismissed_recurring_patterns', [3, 7]); + }); + + it('rejects a non-array dismissed_recurring_patterns', async () => { + const req = { params: { key: 'dismissed_recurring_patterns' }, body: { value: 'weekly' } }; + const res = mockResponse(); + + await expect(routeHandlers['put:/:key'](req, res)).rejects.toBeInstanceOf(ValidationError); + }); + + it('accepts a portfolio_tax_adjustments_v1 entry map (usePortfolioTaxAdjustments payload)', async () => { + const value = { '2026:4': { taxes: 12.5, fees: 3 } }; + settingsRepository.set.mockResolvedValue({ key: 'portfolio_tax_adjustments_v1', value }); + + const req = { params: { key: 'portfolio_tax_adjustments_v1' }, body: { value } }; + const res = mockResponse(); + await routeHandlers['put:/:key'](req, res); + + expect(settingsRepository.set).toHaveBeenCalledWith('portfolio_tax_adjustments_v1', value); + }); + it('propagates error when single setting save fails', async () => { settingsRepository.set.mockRejectedValue(new Error('boom')); @@ -315,6 +352,13 @@ describe('Settings Routes', () => { await expect(routeHandlers['put:/'](req, res)).rejects.toThrow('boom'); }); + it('rejects an unknown key via bulk (no unknown-key bypass)', async () => { + const req = { body: { onboarding_complete: true, mystery_key: { any: 'json' } } }; + const res = mockResponse(); + await expect(routeHandlers['put:/'](req, res)).rejects.toBeInstanceOf(ValidationError); + expect(settingsRepository.setMany).not.toHaveBeenCalled(); + }); + it('rejects an invalid cost_basis_method via bulk (no validation bypass)', async () => { const req = { body: { cost_basis_method: 'bogus' } }; const res = mockResponse(); diff --git a/apps/node-backend/tests/services/transferReconciliation.db.test.js b/apps/node-backend/tests/services/transferReconciliation.db.test.js index 3bbea51a..71f594bc 100644 --- a/apps/node-backend/tests/services/transferReconciliation.db.test.js +++ b/apps/node-backend/tests/services/transferReconciliation.db.test.js @@ -25,7 +25,13 @@ */ import { afterAll, afterEach, beforeAll, describe, expect, it } from 'vitest'; -import { closeTestPool, getTestPool, hasTestDatabase } from '../setup/db.js'; +import { + acquireDbSuiteLock, + closeTestPool, + getTestPool, + hasTestDatabase, + releaseDbSuiteLock, +} from '../setup/db.js'; import { reconcileTransfers, getTransferSuggestions, @@ -133,7 +139,10 @@ describe.skipIf(!hasTestDatabase())('services/transferReconciliationService (rea process.env.DATABASE_URL, 'DATABASE_URL must equal TEST_DATABASE_URL for this suite (see scripts/with-test-db.sh)', ).toBe(process.env.TEST_DATABASE_URL); - }); + // DB suites share one database across parallel vitest workers β€” serialize + // them behind the shared advisory lock (see tests/setup/db.js). + await acquireDbSuiteLock(); + }, 180_000); afterEach(async () => { const pool = getTestPool(); @@ -152,6 +161,7 @@ describe.skipIf(!hasTestDatabase())('services/transferReconciliationService (rea }); afterAll(async () => { + await releaseDbSuiteLock(); await closeTestPool(); await closePool(); }); diff --git a/apps/node-backend/tests/setup/db.js b/apps/node-backend/tests/setup/db.js index cc5d8513..63e5227a 100644 --- a/apps/node-backend/tests/setup/db.js +++ b/apps/node-backend/tests/setup/db.js @@ -51,3 +51,46 @@ export async function closeTestPool() { export function hasTestDatabase() { return Boolean(process.env.TEST_DATABASE_URL); } + +// ── Cross-suite serialization ─────────────────────────────────────────────── +// Vitest runs test FILES in parallel workers, but every DB-backed suite shares +// the one TEST_DATABASE_URL database and wipes whole tables between tests β€” +// two suites running concurrently delete each other's fixtures mid-test. A +// session-scoped Postgres advisory lock serializes them at the database level +// without slowing the (much larger) non-DB portion of the run: each suite +// takes the lock in beforeAll and releases it in afterAll, so DB suites queue +// behind one another while everything else stays parallel. +// +// Suites must pass a generous timeout to beforeAll (the wait is the sum of the +// suites ahead in the queue): `beforeAll(acquireDbSuiteLock, 180_000)`. + +const DB_SUITE_LOCK_KEY = 715_001; + +/** @type {import('pg').PoolClient | null} */ +let lockClient = null; + +/** Block until this process holds the shared DB-suite advisory lock. */ +export async function acquireDbSuiteLock() { + const pool = getTestPool(); + if (!pool || lockClient) return; + const client = await pool.connect(); + try { + await client.query('SELECT pg_advisory_lock($1)', [DB_SUITE_LOCK_KEY]); + } catch (err) { + client.release(); + throw err; + } + lockClient = client; +} + +/** Release the advisory lock (safe to call when not held). */ +export async function releaseDbSuiteLock() { + if (!lockClient) return; + const client = lockClient; + lockClient = null; + try { + await client.query('SELECT pg_advisory_unlock($1)', [DB_SUITE_LOCK_KEY]); + } finally { + client.release(); + } +} diff --git a/apps/node-backend/tests/transactionRepository.db.test.js b/apps/node-backend/tests/transactionRepository.db.test.js new file mode 100644 index 00000000..197e9e67 --- /dev/null +++ b/apps/node-backend/tests/transactionRepository.db.test.js @@ -0,0 +1,646 @@ +/** + * Real-Postgres tests for transactionRepository. + * + * DB-backed complement to transactionRepositoryBehavior.test.js and + * transactionRepositoryOrdering.test.js (which stay: they run without a DB). + * Those suites choreograph SQL result ORDER on a mocked pool, so they assert + * the query strings we wrote rather than the rows Postgres actually returns. + * Everything here runs the same behaviours against a migrated schema with + * realistic fixtures: NUMERIC amounts as strings with cents, real DATE columns + * spanning a (leap-year) month boundary, multiple accounts and currencies, + * alias recipients, inactive rows mixed in, and the dual-write account trigger + * (migration 0051) live. + * + * Isolation strategy (per the setup/db.js contract): per-test targeted DELETEs + * rather than a wrapping transaction β€” create()/update() open their own + * withTransaction, which would nest, and the account-sync trigger's + * INSERT ... ON CONFLICT into accounts is easier to reason about against a + * corpus each test fully owns. + */ + +import { afterAll, afterEach, beforeAll, beforeEach, describe, expect, it } from 'vitest'; +import { + acquireDbSuiteLock, + closeTestPool, + getTestPool, + hasTestDatabase, + releaseDbSuiteLock, +} from './setup/db.js'; +import transactionRepository from '../src/repositories/transactionRepository.js'; +import { closePool } from '../src/database/connection.js'; + +/** 'YYYY-MM-DD' of a row's `date` column (pg returns DATE as a JS Date). */ +const ymd = (d) => new Date(d).toISOString().slice(0, 10); + +// Fixture ids, repopulated by seedCorpus() before each test. +const cat = {}; // Food, Bills, Salary +const rec = {}; // delhaize, delhaizeAlias, electrabel, electrabelAlias, employer +const acc = {}; // KBC CURRENT, WISE USD +const tag = {}; // groceries (active), archived (inactive) +const T = {}; // t1..t7 + +/** + * Ensure an accounts row exists for a label, returning its id. Uses the + * 0066 normalized-identity arbiter (lower(btrim(name))) directly. + * + * The fixtures PRE-CREATE accounts rather than letting the sync trigger mint + * them, so corpus setup stays independent of the trigger under test. (The + * trigger's own onboarding path β€” once broken at head by the 0076 ON CONFLICT + * regression, fixed by migration 0083 β€” has its own dedicated test below.) + */ +async function ensureAccount(name) { + const pool = getTestPool(); + await pool.query( + `INSERT INTO accounts (name, display_name) VALUES ($1, $1) + ON CONFLICT (lower(btrim(name))) DO NOTHING`, + [name], + ); + const { rows } = await pool.query( + 'SELECT id FROM accounts WHERE lower(btrim(name)) = lower(btrim($1))', + [name], + ); + return rows[0].id; +} + +/** + * Insert one transaction through plain SQL (NOT the repository, so repository + * behaviour is never asserted against itself). `bank_account` is written as + * the raw string and account_id left NULL: the trg_transactions_account_sync + * trigger resolves the account exactly as production inserts do (the account + * row itself is pre-created β€” see ensureAccount). + */ +async function insertTxn({ + date, + amount, + currency = 'EUR', + recipientId, + categoryId = null, + bank = 'KBC CURRENT', + memo = null, + isActive = true, + isTransfer = false, +}) { + if (bank) await ensureAccount(bank); + const { rows } = await getTestPool().query( + `INSERT INTO transactions (date, amount, currency, recipient_id, category_id, bank_account, memo, is_active, is_transfer) + VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9) + RETURNING id`, + [date, amount, currency, recipientId, categoryId, bank, memo, isActive, isTransfer], + ); + return rows[0].id; +} + +/** + * Shared corpus. Dates straddle the Febβ†’Mar 2024 boundary (2024 is a leap + * year, so 2024-02-29 is a real day); amounts are NUMERIC strings with cents; + * expenses are negative; two accounts and two currencies; one inactive row. + * + * t1 2024-02-27 -52.30 EUR delhaizeAlias (no category anywhere β†’ uncategorised) + * t7 2024-02-28 -30.00 EUR electrabel (category via recipient default) + * t2 2024-02-29 -18.75 EUR delhaize (own category Food) + * t3 2024-03-01 -120.00 EUR electrabelAlias (category via PRIMARY's default β€” 3rd level) + * t4 2024-03-01 2500.00 EUR employer (income, category via recipient default) + * t5 2024-03-02 -45.10 USD delhaize (own category Food, WISE USD account) + * t6 2024-03-02 -9.99 EUR delhaize (INACTIVE β€” must be invisible everywhere) + */ +async function seedCorpus() { + const pool = getTestPool(); + + for (const [key, [general, detail]] of Object.entries({ + Food: ['Food', 'Groceries'], + Bills: ['Bills', 'Utilities'], + Salary: ['Income', 'Salary'], + })) { + const { rows } = await pool.query( + 'INSERT INTO categories (general, detail) VALUES ($1, $2) RETURNING id', + [general, detail], + ); + cat[key] = rows[0].id; + } + + const addRecipient = async (name, { defaultCategoryId = null, primaryId = null } = {}) => { + const { rows } = await pool.query( + `INSERT INTO recipients (name, normalized_name, default_category_id, primary_recipient_id) + VALUES ($1, $2, $3, $4) RETURNING id`, + [name, name.toLowerCase(), defaultCategoryId, primaryId], + ); + return rows[0].id; + }; + rec.delhaize = await addRecipient('Delhaize'); + rec.delhaizeAlias = await addRecipient('Delhaize BXL', { primaryId: rec.delhaize }); + rec.electrabel = await addRecipient('Electrabel', { defaultCategoryId: cat.Bills }); + rec.electrabelAlias = await addRecipient('Electrabel Invoicing', { primaryId: rec.electrabel }); + rec.employer = await addRecipient('Acme Payroll', { defaultCategoryId: cat.Salary }); + + for (const [key, [slug, color, isActive]] of Object.entries({ + groceries: ['groceries', '#0a0', true], + archived: ['archived', '#999', false], + })) { + const { rows } = await pool.query( + 'INSERT INTO tags (slug, color, is_active) VALUES ($1, $2, $3) RETURNING id', + [slug, color, isActive], + ); + tag[key] = rows[0].id; + } + + T.t1 = await insertTxn({ date: '2024-02-27', amount: '-52.30', recipientId: rec.delhaizeAlias, memo: 'CARD PAYMENT DELHAIZE' }); + T.t2 = await insertTxn({ date: '2024-02-29', amount: '-18.75', recipientId: rec.delhaize, categoryId: cat.Food, memo: 'DELHAIZE BXL' }); + T.t3 = await insertTxn({ date: '2024-03-01', amount: '-120.00', recipientId: rec.electrabelAlias, memo: 'ELECTRABEL INVOICE 2024' }); + T.t4 = await insertTxn({ date: '2024-03-01', amount: '2500.00', recipientId: rec.employer, memo: 'SALARY FEBRUARY' }); + T.t5 = await insertTxn({ date: '2024-03-02', amount: '-45.10', currency: 'USD', recipientId: rec.delhaize, categoryId: cat.Food, bank: 'WISE USD', memo: 'DELHAIZE US' }); + T.t6 = await insertTxn({ date: '2024-03-02', amount: '-9.99', recipientId: rec.delhaize, memo: 'DELHAIZE GHOST', isActive: false }); + T.t7 = await insertTxn({ date: '2024-02-28', amount: '-30.00', recipientId: rec.electrabel, memo: 'ELECTRABEL DOMICILIERING' }); + + // Account ids as resolved by the dual-write trigger. + const accounts = await pool.query('SELECT id, name FROM accounts'); + for (const row of accounts.rows) acc[row.name] = row.id; + + await pool.query( + 'INSERT INTO transaction_tags (transaction_id, tag_id) VALUES ($1, $2), ($1, $3), ($4, $2)', + [T.t1, tag.groceries, tag.archived, T.t2], + ); +} + +describe.skipIf(!hasTestDatabase())('repositories/transactionRepository (real DB)', () => { + beforeAll(async () => { + expect( + process.env.DATABASE_URL, + 'DATABASE_URL must equal TEST_DATABASE_URL for this suite (see scripts/with-test-db.sh)', + ).toBe(process.env.TEST_DATABASE_URL); + // DB suites share one database across parallel vitest workers β€” serialize. + await acquireDbSuiteLock(); + }, 180_000); + + beforeEach(seedCorpus); + + afterEach(async () => { + const pool = getTestPool(); + // FK order: executions β†’ planned β†’ junction β†’ transactions β†’ the rest. + // (transactions.account_id is ON DELETE RESTRICT, so accounts go after.) + await pool.query('DELETE FROM planned_transaction_executions'); + await pool.query('DELETE FROM planned_transactions'); + await pool.query('DELETE FROM transaction_tags'); + await pool.query('DELETE FROM transactions'); + await pool.query('DELETE FROM tags'); + await pool.query('DELETE FROM accounts'); + await pool.query('DELETE FROM recipients'); + await pool.query('DELETE FROM categories'); + for (const bag of [cat, rec, acc, tag, T]) for (const k of Object.keys(bag)) delete bag[k]; + }); + + afterAll(async () => { + await releaseDbSuiteLock(); + await closeTestPool(); + await closePool(); + }); + + // ─────────────────────────────────────────────────────────────────────────── + // getAll β€” enrichment, ordering, filters + // ─────────────────────────────────────────────────────────────────────────── + describe('getAll', () => { + it('returns active rows ordered date DESC, id DESC with real NUMERIC/DATE types', async () => { + const rows = await transactionRepository.getAll({}); + expect(rows.map((r) => r.id)).toEqual([T.t5, T.t4, T.t3, T.t2, T.t7, T.t1]); + const t2 = rows.find((r) => r.id === T.t2); + // pg NUMERIC arrives as a string, at the column's scale: NUMERIC(18,4). + expect(t2.amount).toBe('-18.7500'); + expect(ymd(t2.date)).toBe('2024-02-29'); // leap day survives the DATE round-trip + expect(t2.currency).toBe('EUR'); + }); + + it('resolves recipient_name and the 3-level effective category for alias rows', async () => { + const rows = await transactionRepository.getAll({}); + const byId = Object.fromEntries(rows.map((r) => [r.id, r])); + // Alias rows display the PRIMARY's name. + expect(byId[T.t1].recipient_name).toBe('Delhaize'); + expect(byId[T.t3].recipient_name).toBe('Electrabel'); + // Effective category: own β†’ recipient default β†’ primary default β†’ null. + expect(byId[T.t2].effective_category_id).toBe(cat.Food); + expect(byId[T.t7].effective_category_id).toBe(cat.Bills); + expect(byId[T.t3].effective_category_id).toBe(cat.Bills); // via the PRIMARY's default + expect(byId[T.t3].category_name).toBe('Bills:Utilities'); + expect(byId[T.t1].effective_category_id).toBeNull(); + expect(byId[T.t1].category_name).toBeNull(); + }); + + it('attaches tags per row (junction truth, inactive tags included) and [] otherwise', async () => { + const rows = await transactionRepository.getAll({}); + const byId = Object.fromEntries(rows.map((r) => [r.id, r])); + expect(byId[T.t1].tags.map((t) => t.slug).sort()).toEqual(['archived', 'groceries']); + expect(byId[T.t1].tags.find((t) => t.slug === 'archived').is_active).toBe(false); + expect(byId[T.t2].tags).toEqual([ + { id: tag.groceries, slug: 'groceries', color: '#0a0', is_active: true }, + ]); + expect(byId[T.t4].tags).toEqual([]); + }); + + it('filters by accountId (FK, trigger-resolved) and bankAccount (ILIKE substring)', async () => { + const wise = await transactionRepository.getAll({ accountId: acc['WISE USD'] }); + expect(wise.map((r) => r.id)).toEqual([T.t5]); + const kbc = await transactionRepository.getAll({ bankAccount: 'kbc' }); + expect(kbc.map((r) => r.id)).toEqual([T.t4, T.t3, T.t2, T.t7, T.t1]); + }); + + it('applies inclusive date bounds across the month boundary', async () => { + const march = await transactionRepository.getAll({ startDate: '2024-03-01' }); + expect(march.map((r) => r.id)).toEqual([T.t5, T.t4, T.t3]); + const feb = await transactionRepository.getAll({ endDate: '2024-02-29' }); + expect(feb.map((r) => r.id)).toEqual([T.t2, T.t7, T.t1]); + }); + + it('recipientId matches the recipient AND aliases under it; alias id matches only itself', async () => { + const group = await transactionRepository.getAll({ recipientId: rec.delhaize }); + expect(group.map((r) => r.id)).toEqual([T.t5, T.t2, T.t1]); // t1 recorded under the alias + const aliasOnly = await transactionRepository.getAll({ recipientId: rec.delhaizeAlias }); + expect(aliasOnly.map((r) => r.id)).toEqual([T.t1]); + // recipientGroupId from the ALIAS resolves the whole primary group. + const fullGroup = await transactionRepository.getAll({ recipientGroupId: rec.delhaizeAlias }); + expect(fullGroup.map((r) => r.id)).toEqual([T.t5, T.t2, T.t1]); + }); + + it('sorts by amount numerically, not lexicographically', async () => { + const rows = await transactionRepository.getAll({ sortBy: 'amount', sortDir: 'asc' }); + // A string sort would put '-120.0000' between '-18.7500' and '-30.0000'. + expect(rows.map((r) => r.amount)).toEqual([ + '-120.0000', '-52.3000', '-45.1000', '-30.0000', '-18.7500', '2500.0000', + ]); + }); + + it('computes running_balance as a per-account ledger (partitioned, date ASC over the filtered set)', async () => { + const rows = await transactionRepository.getAll({ includeBalance: true }); + const rb = Object.fromEntries(rows.map((r) => [r.id, Number(r.running_balance)])); + // KBC CURRENT: -52.30 β†’ -82.30 β†’ -101.05 β†’ -221.05 β†’ +2278.95 (date ASC, id ASC) + expect(rb[T.t1]).toBe(-52.3); + expect(rb[T.t7]).toBe(-82.3); + expect(rb[T.t2]).toBe(-101.05); + expect(rb[T.t3]).toBe(-221.05); + expect(rb[T.t4]).toBe(2278.95); + // WISE USD is its own partition β€” NOT a continuation of KBC's sum. + expect(rb[T.t5]).toBe(-45.1); + }); + }); + + // ─────────────────────────────────────────────────────────────────────────── + // Pagination β€” the tiebreaker behaviour the ordering suite pins as SQL text + // ─────────────────────────────────────────────────────────────────────────── + describe('pagination tiebreaker', () => { + it('never duplicates or skips same-date rows across pages', async () => { + // Five more rows all on one date: only the id tiebreaker orders them. + const sameDay = []; + for (let i = 0; i < 5; i++) { + sameDay.push(await insertTxn({ + date: '2024-03-05', + amount: `-${i + 1}.00`, + recipientId: rec.delhaize, + })); + } + const seen = []; + for (let offset = 0; offset < 11; offset += 2) { + const page = await transactionRepository.getAll({ limit: 2, offset }); + seen.push(...page.map((r) => r.id)); + } + expect(seen).toHaveLength(11); + expect(new Set(seen).size).toBe(11); // no duplicates + expect(seen.slice(0, 5)).toEqual([...sameDay].reverse()); // id DESC within the date + }); + }); + + // ─────────────────────────────────────────────────────────────────────────── + // getCount / getAllWithCount + // ─────────────────────────────────────────────────────────────────────────── + describe('getCount', () => { + it('counts active rows by default and everything with active:false', async () => { + expect(await transactionRepository.getCount({})).toBe(6); + expect(await transactionRepository.getCount({ active: false })).toBe(7); + }); + + it('counts by effective category across all three levels', async () => { + // Bills: t7 (recipient default) + t3 (primary's default via alias). + expect(await transactionRepository.getCount({ categoryId: cat.Bills })).toBe(2); + expect(await transactionRepository.getCount({ categoryId: cat.Food })).toBe(2); + }); + }); + + describe('getAllWithCount', () => { + it('returns a page of rows with the total over the whole filtered set', async () => { + const { rows, total } = await transactionRepository.getAllWithCount({ limit: 2, offset: 0 }); + expect(rows.map((r) => r.id)).toEqual([T.t5, T.t4]); + expect(total).toBe(6); + }); + + it('splits income from expenses by amount sign', async () => { + const income = await transactionRepository.getAllWithCount({ transactionType: 'income' }); + expect(income.rows.map((r) => r.id)).toEqual([T.t4]); + expect(income.total).toBe(1); + const expense = await transactionRepository.getAllWithCount({ transactionType: 'expense' }); + expect(expense.total).toBe(5); + }); + + it('amount bounds compare magnitude by default and the signed amount when amountSigned', async () => { + const magnitude = await transactionRepository.getAllWithCount({ amountMin: 50 }); + expect(magnitude.rows.map((r) => r.id).sort((a, b) => a - b)).toEqual( + [T.t1, T.t3, T.t4].sort((a, b) => a - b), // |βˆ’52.30|, |βˆ’120|, |2500| + ); + const signed = await transactionRepository.getAllWithCount({ amountMax: -100, amountSigned: true }); + expect(signed.rows.map((r) => r.id)).toEqual([T.t3]); // only βˆ’120.00 ≀ βˆ’100 + }); + + it('categoryIds matches the effective category (own / recipient default / primary default)', async () => { + const { rows, total } = await transactionRepository.getAllWithCount({ + categoryIds: [cat.Food, cat.Salary], + }); + expect(total).toBe(3); + expect(rows.map((r) => r.id)).toEqual([T.t5, T.t4, T.t2]); + }); + + it('tagSlugs matches rows via ACTIVE tags only', async () => { + const groceries = await transactionRepository.getAllWithCount({ tagSlugs: ['groceries'] }); + expect(groceries.rows.map((r) => r.id)).toEqual([T.t2, T.t1]); + // The inactive tag is attached to t1's row payload (see getAll test) but + // deliberately does NOT match as a filter. + const archived = await transactionRepository.getAllWithCount({ tagSlugs: ['archived'] }); + expect(archived.total).toBe(0); + }); + + it('free-text search spans memo, recipient (incl. primary), category and amount text', async () => { + const byName = await transactionRepository.getAllWithCount({ search: 'delhaize' }); + expect(byName.rows.map((r) => r.id)).toEqual([T.t5, T.t2, T.t1]); // t6 inactive stays out + const byCategory = await transactionRepository.getAllWithCount({ search: 'utilities' }); + expect(byCategory.rows.map((r) => r.id)).toEqual([T.t3, T.t7]); + const byAmount = await transactionRepository.getAllWithCount({ search: '2500' }); + expect(byAmount.rows.map((r) => r.id)).toEqual([T.t4]); + }); + }); + + // ─────────────────────────────────────────────────────────────────────────── + // Uncategorised queue β€” the alias-regression the mock suite pins as SQL text + // ─────────────────────────────────────────────────────────────────────────── + describe('getUncategorised / getUncategorisedWithCount', () => { + it('lists only rows whose FULL 3-level effective category is NULL', async () => { + const rows = await transactionRepository.getUncategorised({}); + // t3 (alias whose primary carries a default) must NOT leak into the + // queue β€” the regression that motivated the 3-level predicate. t6 is + // uncategorised too but inactive. + expect(rows.map((r) => r.id)).toEqual([T.t1]); + }); + + it('supports recipientName / bankAccount narrowing', async () => { + const hit = await transactionRepository.getUncategorised({ recipientName: 'delh' }); + expect(hit.map((r) => r.id)).toEqual([T.t1]); + const miss = await transactionRepository.getUncategorised({ bankAccount: 'wise' }); + expect(miss).toEqual([]); + }); + + it('getUncategorisedWithCount: rows are uncategorised, total keeps full getCount semantics', async () => { + const { rows, total } = await transactionRepository.getUncategorisedWithCount({}); + expect(rows.map((r) => r.id)).toEqual([T.t1]); + expect(total).toBe(6); // documented asymmetry: total counts ALL active rows + }); + + it('returns total 0 and no rows when nothing matches', async () => { + const res = await transactionRepository.getUncategorisedWithCount({ recipientName: 'nobody-here' }); + expect(res).toEqual({ rows: [], total: 0 }); + }); + }); + + // ─────────────────────────────────────────────────────────────────────────── + // getById / create / update / hardDelete + // ─────────────────────────────────────────────────────────────────────────── + describe('getById', () => { + it('returns the enriched row and null for a missing id', async () => { + const row = await transactionRepository.getById(T.t3); + expect(row).toMatchObject({ + id: T.t3, + recipient_name: 'Electrabel', + effective_category_id: cat.Bills, + category_name: 'Bills:Utilities', + tags: [], + }); + expect(await transactionRepository.getById(T.t7 + 100_000)).toBeNull(); + }); + }); + + describe('create', () => { + it('uppercases bank/memo/currency, and the trigger resolves the account case-insensitively', async () => { + // Pre-existing account stored with different casing than the write. + const revolutId = await ensureAccount('Revolut Main'); + const row = await transactionRepository.create({ + transaction_date: '2024-03-10', + bank_account: 'revolut main', + recipient_id: rec.delhaize, + amount: '-3.20', + memo: 'coffee', + currency: 'usd', + category_id: null, + comment: 'espresso', + }); + expect(row).toMatchObject({ + bank_account: 'REVOLUT MAIN', + memo: 'COFFEE', + currency: 'USD', + amount: '-3.2000', + comment: 'espresso', + is_active: true, + tags: [], + }); + expect(row.balance).toBeNull(); // manual rows never carry a bank stamp + // 'REVOLUT MAIN' reuses 'Revolut Main' instead of minting a twin (0076 fix #2). + expect(row.account_id).toBe(revolutId); + const { rows } = await getTestPool().query( + `SELECT count(*)::int AS n FROM accounts WHERE lower(name) = 'revolut main'`, + ); + expect(rows[0].n).toBe(1); + }); + + // Regression coverage for the (fixed) 0076 ON CONFLICT arbiter finding: + // migration 0066 dropped uq_accounts_name for the expression index + // uq_accounts_name_norm on lower(btrim(name)), and the trigger rewrite in + // 0076 regressed the onboarding INSERT's arbiter back to + // `ON CONFLICT (name)` β€” matching NO unique index, so every first-seen + // label raised 42P10 at schema head ("there is no unique or exclusion + // constraint matching the ON CONFLICT specification"). Fixed by migration + // 0083 (CREATE OR REPLACE for already-migrated installs) plus an in-place + // edit of 0076 (fresh installs). This test formerly PINNED the failure; + // it now asserts the repaired behaviour: a brand-new label onboards + // exactly one account (trimmed name, 0066 normalized-identity dedup + // across casings). No mock suite could see this: the arbiter is only + // validated when the trigger actually executes against the real schema. + it('onboards a NEW account for a first-seen bank label (0076 ON CONFLICT regression, fixed by 0083)', async () => { + const row = await transactionRepository.create({ + transaction_date: '2024-03-10', + bank_account: 'BRAND NEW BANK', + recipient_id: rec.delhaize, + amount: '-1.00', + category_id: null, + comment: null, + }); + // The trigger onboarded the account and stamped the FK on the row. + const { rows: accounts } = await getTestPool().query( + `SELECT id, name FROM accounts WHERE lower(btrim(name)) = 'brand new bank'`, + ); + expect(accounts).toHaveLength(1); + expect(accounts[0].name).toBe('BRAND NEW BANK'); // trimmed by the trigger + expect(row.account_id).toBe(accounts[0].id); + + // A second casing/spacing of the SAME new label (raw SQL, bypassing the + // repository's toUpperCase) resolves to the existing account instead of + // minting a twin β€” the 0066 dedup semantics survive the 0083 fix. + const { rows: second } = await getTestPool().query( + `INSERT INTO transactions (date, amount, currency, recipient_id, bank_account) + VALUES ('2024-03-11', -2.00, 'EUR', $1, ' Brand New Bank ') + RETURNING account_id`, + [rec.delhaize], + ); + expect(second[0].account_id).toBe(accounts[0].id); + const { rows: count } = await getTestPool().query( + `SELECT count(*)::int AS n FROM accounts WHERE lower(btrim(name)) = 'brand new bank'`, + ); + expect(count[0].n).toBe(1); // ONE account across both casings + }); + + it('defaults currency to EUR and nulls bank/memo when absent', async () => { + const row = await transactionRepository.create({ + transaction_date: '2024-03-10', + recipient_id: rec.delhaize, + amount: '5.00', + category_id: null, + comment: null, + }); + expect(row.currency).toBe('EUR'); + expect(row.bank_account).toBeNull(); + expect(row.memo).toBeNull(); + expect(row.account_id).toBeNull(); // no bank string β†’ trigger leaves the FK alone + }); + + it('resolves the effective category through the recipient on the returned row', async () => { + const row = await transactionRepository.create({ + transaction_date: '2024-03-10', + recipient_id: rec.electrabelAlias, + amount: '-77.77', + category_id: null, + comment: null, + }); + expect(row.recipient_name).toBe('Electrabel'); + expect(row.effective_category_id).toBe(cat.Bills); + expect(row.category_name).toBe('Bills:Utilities'); + }); + + it('sets tags atomically, silently dropping inactive/unknown slugs', async () => { + const row = await transactionRepository.create({ + transaction_date: '2024-03-10', + recipient_id: rec.delhaize, + amount: '-8.00', + category_id: null, + comment: null, + tags: ['groceries', 'archived', 'does-not-exist'], + }); + expect(row.tags.map((t) => t.slug)).toEqual(['groceries']); + const { rows } = await getTestPool().query( + 'SELECT tag_id FROM transaction_tags WHERE transaction_id = $1', [row.id], + ); + expect(rows).toEqual([{ tag_id: tag.groceries }]); + }); + }); + + describe('update', () => { + it('maps transaction_date β†’ date and persists NUMERIC edits', async () => { + const row = await transactionRepository.update(T.t1, { + transaction_date: '2024-03-15', + amount: '-60.00', + }); + expect(ymd(row.date)).toBe('2024-03-15'); + expect(row.amount).toBe('-60.0000'); + expect(row.updated_at).not.toBeNull(); + const { rows } = await getTestPool().query( + 'SELECT date, amount FROM transactions WHERE id = $1', [T.t1], + ); + expect(ymd(rows[0].date)).toBe('2024-03-15'); + expect(rows[0].amount).toBe('-60.0000'); + }); + + it('returns null for a missing id, with fields and tags-only alike', async () => { + expect(await transactionRepository.update(T.t7 + 100_000, { amount: '1.00' })).toBeNull(); + expect(await transactionRepository.update(T.t7 + 100_000, { tags: ['groceries'] })).toBeNull(); + // The tags-only 404 probe must not have leaked a junction row (FK 23503 β†’ 500 bug). + const { rows } = await getTestPool().query( + 'SELECT 1 FROM transaction_tags WHERE transaction_id = $1', [T.t7 + 100_000], + ); + expect(rows).toEqual([]); + }); + + it('tags-only PATCH replaces the junction set', async () => { + const row = await transactionRepository.update(T.t1, { tags: ['groceries'] }); + expect(row.tags.map((t) => t.slug)).toEqual(['groceries']); // 'archived' junction removed + const cleared = await transactionRepository.update(T.t1, { tags: [] }); + expect(cleared.tags).toEqual([]); + }); + + it('no writable fields and no tags falls back to the current row', async () => { + const row = await transactionRepository.update(T.t2, {}); + expect(row.id).toBe(T.t2); + expect(row.amount).toBe('-18.7500'); + }); + + // Regression coverage (formerly a pinned bug): update()'s RETURNING + // enrichment used to join only 2 category levels (own + recipient default) + // β€” unlike getById/getAll/create, which resolve 3. For a row categorised + // via its PRIMARY recipient's default, the update response reported + // category_name NULL (and the alias's own name) even though a follow-up + // GET showed 'Bills:Utilities' / the primary's name. update() now shares + // the same 3-level fragments as the read paths, in both the RETURNING CTE + // and the tags-path fetch, so the update response and an immediate GET + // must be identical. + it('update() response resolves the full 3-level category β€” identical to getById on alias rows', async () => { + const updated = await transactionRepository.update(T.t3, { amount: '-121.00' }); + const fetched = await transactionRepository.getById(T.t3); + expect(updated.category_name).toBe('Bills:Utilities'); // via the PRIMARY's default + expect(updated.effective_category_id).toBe(cat.Bills); + expect(updated.recipient_name).toBe('Electrabel'); // COALESCE(pr.name, r.name) + expect(fetched).toEqual(updated); // response ≑ immediate GET, field for field + + // The tags-path fetch (fields + tags in one PATCH) resolves identically. + const withTags = await transactionRepository.update(T.t3, { amount: '-122.00', tags: [] }); + expect(withTags.category_name).toBe('Bills:Utilities'); + expect(withTags.effective_category_id).toBe(cat.Bills); + expect(withTags.recipient_name).toBe('Electrabel'); + }); + }); + + describe('hardDelete', () => { + it('deletes exactly once (junction rows cascade)', async () => { + expect(await transactionRepository.hardDelete(T.t1)).toBe(true); + expect(await transactionRepository.hardDelete(T.t1)).toBe(false); + const { rows } = await getTestPool().query( + 'SELECT 1 FROM transaction_tags WHERE transaction_id = $1', [T.t1], + ); + expect(rows).toEqual([]); + }); + }); + + // ─────────────────────────────────────────────────────────────────────────── + // listRecentUnlinked β€” NOT EXISTS against planned_transaction_executions + // ─────────────────────────────────────────────────────────────────────────── + describe('listRecentUnlinked', () => { + it('excludes already-linked and inactive rows, resolves the recipient cluster root', async () => { + const pool = getTestPool(); + const planned = await pool.query( + `INSERT INTO planned_transactions (planned_date, amount, recipient_id) + VALUES ('2024-02-29', '-18.75', $1) RETURNING id`, + [rec.delhaize], + ); + await pool.query( + `INSERT INTO planned_transaction_executions (planned_transaction_id, executed_transaction_id, execution_date) + VALUES ($1, $2, '2024-02-29')`, + [planned.rows[0].id, T.t2], + ); + + const rows = await transactionRepository.listRecentUnlinked({ sinceDate: '2024-02-28' }); + // t1 predates sinceDate, t2 is linked, t6 is inactive. + expect(rows.map((r) => r.id)).toEqual([T.t5, T.t4, T.t3, T.t7]); + const t3 = rows.find((r) => r.id === T.t3); + expect(t3.recipient_cluster_id).toBe(rec.electrabel); // alias resolves to its primary + expect(t3.recipient_name).toBe('Electrabel Invoicing'); // but the row keeps its own name + expect(t3.amount).toBe('-120.0000'); + }); + }); +}); diff --git a/apps/node-backend/tests/visionAdapter.test.js b/apps/node-backend/tests/visionAdapter.test.js index fa981c7d..d5041d83 100644 --- a/apps/node-backend/tests/visionAdapter.test.js +++ b/apps/node-backend/tests/visionAdapter.test.js @@ -83,6 +83,18 @@ INVALID_DATE,Main Account,Skip Date,Note,-10.00,EUR,944.80,OTHER,invalid date expect(txns[0].currency).toBe('EUR'); }); + it('uppercases an ISO currency and falls back to EUR for free text (was a commit-time CHECK 500)', async () => { + const csv = `Date,Bank Account,Recipient,Memo,Amount,Currency,Balance,Category,Comment +2026-03-04,Main,Acme,one,-10.00,usd,100.00,OTHER, +2026-03-05,Main,Acme,two,-11.00,euro,89.00,OTHER, +`; + + tmpPath = writeTempCSV(csv); + const txns = await parse(tmpPath); + + expect(txns.map((t) => t.currency)).toEqual(['USD', 'EUR']); + }); + it('builds comment from imported category and existing comment', async () => { const csv = `Date,Bank Account,Recipient,Memo,Amount,Currency,Balance,Category,Comment 2026-03-05,Personal,Electric Company,bill,-120.00,EUR,1859.80,UTILITIES,Paid by direct debit diff --git a/docs/api/admin.md b/docs/api/admin.md index 9b97f96a..ed48a7f3 100644 --- a/docs/api/admin.md +++ b/docs/api/admin.md @@ -636,24 +636,27 @@ The probe updates the `provider_health` row (calls `recordSuccess` or `recordErr ### GET /api/admin/metrics/requests -Retrieve rolling request metrics per route from the in-memory window (last 15 minutes, 1-minute buckets). +Retrieve rolling request metrics per route from the in-memory window (last 15 minutes, 1-minute buckets). Canonical collection body `{ items, total }` (unpaginated β€” `total` is the row count). **Response:** `200 OK` ```json -[ - { - "route": "GET:/api/transactions", - "method": "GET", - "path": "/api/transactions", - "count": 142, - "errors": 3, - "error_rate": 0.021, - "p50_ms": 18, - "p95_ms": 67, - "window_minutes": 15 - } -] +{ + "items": [ + { + "route": "GET:/api/transactions", + "method": "GET", + "path": "/api/transactions", + "count": 142, + "errors": 3, + "error_rate": 0.021, + "p50_ms": 18, + "p95_ms": 67, + "window_minutes": 15 + } + ], + "total": 1 +} ``` **Notes:** diff --git a/docs/api/ai.md b/docs/api/ai.md index 3394f672..680a3dee 100644 --- a/docs/api/ai.md +++ b/docs/api/ai.md @@ -49,14 +49,15 @@ When Ollama is unreachable the route still returns 200 with `{"ok": false, ...}` ## GET /api/ai/models -Pass-through of `GET /api/tags` from Ollama. +Pass-through of `GET /api/tags` from Ollama, served in the canonical collection body `{ items, total }` (unpaginated β€” `total` is the row count). **Response 200:** ```json { - "models": [ + "items": [ { "name": "llama3.2:3b", "size": 2100000000, "modified": "2026-04-10T08:12:00Z" } - ] + ], + "total": 1 } ``` diff --git a/docs/api/info.md b/docs/api/info.md index 1ca4d16a..76828d38 100644 --- a/docs/api/info.md +++ b/docs/api/info.md @@ -37,15 +37,14 @@ All monetary values in responses use **Decimal.js** for precision to eliminate I ### GET /api/info/banks -List all bank accounts in the workspace. +List all bank accounts in the workspace. Canonical collection body `{ items, total }` (unpaginated β€” `total` is the row count). **Response:** `200 OK` ```json { - "banks": [ - { "id": 1, "name": "Main Account", "balance": 5000.00 } - ] + "items": ["BE68539007547034", "BE71096123456769"], + "total": 2 } ``` @@ -57,19 +56,21 @@ List all supported bank adapters. **Response:** `200 OK` +Canonical collection body `{ items, total }`. + ```json { - "adapters": [ - { "key": "belfius", "name": "Belfius", "adapter_class": "BelfiusAdapter" }, - { "key": "revolut", "name": "Revolut", "adapter_class": "RevolutAdapter" }, - { "key": "ing", "name": "ING", "adapter_class": "INGAdapter" }, - { "key": "kbc", "name": "KBC", "adapter_class": "KBCAdapter" }, - { "key": "bnp", "name": "BNP Paribas Fortis", "adapter_class": "BNPAdapter" }, - { "key": "sabb", "name": "SABB", "adapter_class": "SABBAdapter" }, - { "key": "wise", "name": "Wise", "adapter_class": "WiseAdapter" }, - { "key": "vision", "name": "Vision", "adapter_class": "VisionAdapter" } + "items": [ + { "key": "belfius", "name": "Belfius" }, + { "key": "revolut", "name": "Revolut" }, + { "key": "ing", "name": "ING" }, + { "key": "kbc", "name": "KBC" }, + { "key": "bnp", "name": "BNP Paribas Fortis" }, + { "key": "sabb", "name": "SABB" }, + { "key": "wise", "name": "Wise" }, + { "key": "vision", "name": "Vision" } ], - "total_count": 8 + "total": 8 } ``` diff --git a/docs/api/marketLookup.md b/docs/api/marketLookup.md index 301febcc..4cd66e55 100644 --- a/docs/api/marketLookup.md +++ b/docs/api/marketLookup.md @@ -84,7 +84,7 @@ Use `detail=basic` for price-only views such as benchmark strips, watchlist prev ```json { - "quotes": [ + "items": [ { "symbol": "AAPL", "name": "Apple Inc.", @@ -127,17 +127,18 @@ Use `detail=basic` for price-only views such as benchmark strips, watchlist prev } ] } - ] + ], + "total": 1 } ``` **Response β€” `detail=basic`:** `200 OK` -Same envelope shape (`{ "quotes": [ … ] }`), but each quote object contains only the core price fields. The fundamentals/analyst fields (`marketCap`, `pe`, `forwardPE`, `dividendYield`, `eps`, `beta`, `priceToBook`, `analystConsensus`, `recentAnalystActions`) are **omitted**. +Same canonical collection body (`{ "items": [ … ], "total": n }`), but each quote object contains only the core price fields. The fundamentals/analyst fields (`marketCap`, `pe`, `forwardPE`, `dividendYield`, `eps`, `beta`, `priceToBook`, `analystConsensus`, `recentAnalystActions`) are **omitted**. ```json { - "quotes": [ + "items": [ { "symbol": "AAPL", "name": "Apple Inc.", @@ -156,7 +157,8 @@ Same envelope shape (`{ "quotes": [ … ] }`), but each quote object contains on "high52w": 199.62, "low52w": 164.08 } - ] + ], + "total": 1 } ``` @@ -197,7 +199,7 @@ Get historical price chart data for a symbol. { "symbol": "AAPL", "currency": "USD", - "points": [ + "items": [ { "time": 1709246400000, "close": 175.43, @@ -205,7 +207,8 @@ Get historical price chart data for a symbol. "low": 173.00, "volume": 52436789 } - ] + ], + "total": 1 } ``` @@ -228,7 +231,7 @@ Get news articles for one or more symbols. ```json { - "articles": [ + "items": [ { "title": "Apple Reports Strong Q4 Earnings", "link": "https://finance.yahoo.com/...", @@ -237,7 +240,8 @@ Get news articles for one or more symbols. "thumbnail": "https://image.com/thumb.jpg", "relatedSymbols": ["AAPL"] } - ] + ], + "total": 1 } ``` @@ -294,8 +298,8 @@ Backend import route tests were updated to validate the unified API response env ### Earlier Notes (2026-04-10) - Quote + summary mapping behavior when provider responses are available. -- Quote failure fallback behavior returning `{"quotes": []}` for partial failure tolerance. +- Quote failure fallback behavior returning `{"items": [], "total": 0}` for partial failure tolerance. - `GET /api/market/news` deduplication by title and server-side thumbnail normalization. -- News search failure tolerance returning `{"articles": []}`. +- News search failure tolerance returning `{"items": [], "total": 0}`. Code links: [[apps/node-backend/tests/routes/marketLookup.test.js]], [[apps/node-backend/src/routes/marketLookup.js]], [[apps/node-backend/tests/routes/import.test.js]] diff --git a/docs/api/plannedTransactions.md b/docs/api/plannedTransactions.md index 8fa3fef0..04dc7632 100644 --- a/docs/api/plannedTransactions.md +++ b/docs/api/plannedTransactions.md @@ -225,10 +225,10 @@ This endpoint is read-only and never mutates any state. The caller (frontend `Ma **Query Parameters:** none -**Response:** +**Response:** canonical collection body `{ items, total }` (unpaginated β€” `total` is the row count). ```json { - "suggestions": [ + "items": [ { "planned": { "id": 42, @@ -255,7 +255,7 @@ This endpoint is read-only and never mutates any state. The caller (frontend `Ma - Only active, unexecuted planned payments are considered. - Transaction lookback is 45 days (sourced from `transactionRepository.listRecentUnlinked({ sinceDate })`). - Loan-type planned payments (`is_loan = true`) are excluded. -- Returns an empty `suggestions` array when `autoClearPlannedOnMatch` is `false`. +- Returns an empty `items` array when `autoClearPlannedOnMatch` is `false`. - Recipient cluster roots are resolved via `recipientRepository.getClusterRootMap` so clustered aliases are compared correctly. **Response Status Codes:** diff --git a/docs/api/research.md b/docs/api/research.md index 57322d40..75a02a94 100644 --- a/docs/api/research.md +++ b/docs/api/research.md @@ -600,7 +600,7 @@ The cross-provider symbol map is the fool-proof anchor against silent wrong-inst List stored mappings for an instrument. **Query:** `instrument_key` (required), `key_type` (`isin` | `internal`, default `isin`). -**Response:** `{ mappings: [ { id, instrument_key, key_type, provider, provider_symbol, resolved_name, exchange, currency, status, verified_at } ] }` β€” `status` ∈ `confirmed` | `auto` | `failed`. +**Response:** canonical collection body `{ items: [ { id, instrument_key, key_type, provider, provider_symbol, resolved_name, exchange, currency, status, verified_at } ], total }` β€” `status` ∈ `confirmed` | `auto` | `failed`. ### POST /api/research/mappings/resolve @@ -616,7 +616,7 @@ Auto-propose a per-provider symbol by running each search-capable, keyed provide Persist user-confirmed mappings (upsert one row per provider, default `status=confirmed`). -**Body:** `{ instrument_key, key_type?, mappings: [ { provider, providerSymbol, resolvedName?, exchange?, currency? } ] }` (non-empty array). **Response:** `{ mappings: [...] }` β€” the full stored set after upsert. +**Body:** `{ instrument_key, key_type?, mappings: [ { provider, providerSymbol, resolvedName?, exchange?, currency? } ] }` (non-empty array). **Response:** canonical collection body `{ items: [...], total }` β€” the full stored set after upsert. ### DELETE /api/research/mappings/:id @@ -637,11 +637,11 @@ Manage the keyed providers' API keys from the app (or via the root `.env`, ADR-0 ### GET /api/research/provider-keys -Returns `{ providers: [{ provider, label, envVar, configured, source, masked }] }` β€” `source` ∈ `settings` | `env` | `none`; `masked` shows only the last 4 characters. +Returns the canonical collection body `{ items: [{ provider, label, envVar, configured, source, masked }], total }` β€” `source` ∈ `settings` | `env` | `none`; `masked` shows only the last 4 characters. ### PUT /api/research/provider-keys/:provider -Body `{ api_key }`. Stores/replaces the key for `provider` (one of `twelve_data` / `finnhub` / `fmp` / `alpha_vantage` / `fred`) and returns the updated masked statuses. `400` on unknown provider or empty key. +Body `{ api_key }`. Stores/replaces the key for `provider` (one of `twelve_data` / `finnhub` / `fmp` / `alpha_vantage` / `fred`) and returns the updated masked statuses in the same `{ items, total }` body as the GET. `400` on unknown provider or empty key. ### DELETE /api/research/provider-keys/:provider diff --git a/docs/reference/api-client-methods.md b/docs/reference/api-client-methods.md index 2e0bbe9c..f6f54bf3 100644 --- a/docs/reference/api-client-methods.md +++ b/docs/reference/api-client-methods.md @@ -112,7 +112,7 @@ The legacy `apiClient` singleton (`[[apps/frontend/src/lib/api.ts]]`, 1243 lines | `createWatchlistItem(data)` | POST /api/watchlist | `WatchlistItem` | | `updateWatchlistItem(id, data)` | PATCH /api/watchlist/:id | `WatchlistItem` | | `deleteWatchlistItem(id)` | DELETE /api/watchlist/:id | `void` | -| `getMarketQuotes(symbols)` | GET /api/market/quotes | `{ quotes: Array<{symbol, price, change, changePercent}> }` | +| `getMarketQuotes(symbols)` | GET /api/market/quote | `Array<{symbol, price, change, changePercent}>` (unwrapped from the `{ items, total }` body) | **Phase 3.6 Enhancement**: WatchlistPage refactored to use typed `apiClient` watchlist methods instead of scattered raw `fetch()` calls. Enables shared retry logic, timeout handling, and React Query integration. `getMarketQuotes()` fetches live quotes for multiple symbols via comma-separated list. @@ -143,7 +143,7 @@ The legacy `apiClient` singleton (`[[apps/frontend/src/lib/api.ts]]`, 1243 lines | Method | Backend Route | Return Type | Notes | |--------|----------|-------------|-------| | `getStatistics(params?)` | GET /api/info | Statistics summary | Direct | -| `getSupportedParsers()` | GET /api/info/supported-adapters | `{ adapters, total_count }` | Direct | +| `getSupportedParsers()` | GET /api/info/supported-adapters | `SupportedAdapter[]` (unwrapped from the `{ items, total }` body) | Direct | | `getTransactionSummary(params?)` | GET /api/info/transaction-summary | Summary stats | Direct | | `getTransactionCount()` | GET /api/info/transaction-count | `{ total_transactions }` | Direct | | `getMonthlyFinancialSummary(params?)` | GET /api/aggregations/monthly-summary | Monthly data | Phase G: Envelope unwrapped | @@ -185,7 +185,7 @@ The legacy `apiClient` singleton (`[[apps/frontend/src/lib/api.ts]]`, 1243 lines | Method | Endpoint | Return Type | |--------|----------|-------------| -| `getMarketNews(symbols?, count?)` | GET /api/market/news | `{ articles }` | +| `getMarketNews(symbols?, count?)` | GET /api/market/news | `MarketNewsArticle[]` (unwrapped from the `{ items, total }` body) | ### Electron (Desktop Only) diff --git a/i18n/source/en.json b/i18n/source/en.json index dbaeb8f9..a3b0ae36 100644 --- a/i18n/source/en.json +++ b/i18n/source/en.json @@ -205,6 +205,7 @@ "addInv.label.ticker": "Ticker / Symbol", "addInv.label.totalCost": "Total Cost", "addInv.label.units": "Units / Shares", + "addInv.nameRequired": "Name is required", "addInv.placeholder.cadastralIncome": "e.g. 1250", "addInv.placeholder.jsonEndpoint": "https://api.example.com/price", "addInv.placeholder.location": "e.g. Brussels, Belgium", @@ -1045,6 +1046,7 @@ "invDetail.unitsAt": "{units} units @ {price}", "invDetail.unitsHeld": "Units Held", "invDetail.unrealizedGain": "Unrealized Gain", + "invEdit.nameRequired": "Name is required", "invEdit.symbolRequired": "Ticker / symbol is required", "invEdit.title": "Edit Investment", "invEdit.toast.updated": "Investment \"{name}\" updated", @@ -1652,6 +1654,7 @@ "portfolioImport.defaultAssetClass": "Default asset class", "portfolioImport.defaultType": "Default type", "portfolioImport.desc": "Import trades (buys, sells, dividends, fees) from a broker or exchange CSV. Map the columns to your portfolio fields.", + "portfolioImport.duplicateColumnsWarning": "The same CSV column is mapped to multiple fields: {columns}. Every mapped field will copy that column's value β€” double-check this is intended.", "portfolioImport.newCustom": "New custom parser", "portfolioImport.parserNamePlaceholder": "e.g. Degiro, Binance", "portfolioImport.parserSource": "Parser", diff --git a/i18n/source/nl.json b/i18n/source/nl.json index 442b1a1f..00cc83b4 100644 --- a/i18n/source/nl.json +++ b/i18n/source/nl.json @@ -205,6 +205,7 @@ "addInv.label.ticker": "Ticker / Symbool", "addInv.label.totalCost": "Totale kostprijs", "addInv.label.units": "Eenheden / Aandelen", + "addInv.nameRequired": "Naam is verplicht", "addInv.placeholder.cadastralIncome": "bijv. 1250", "addInv.placeholder.jsonEndpoint": "https://api.example.com/price", "addInv.placeholder.location": "bijv. Brussel, BelgiΓ«", @@ -1045,6 +1046,7 @@ "invDetail.unitsAt": "{units} eenheden @ {price}", "invDetail.unitsHeld": "Gehouden eenheden", "invDetail.unrealizedGain": "Ongerealiseerde winst", + "invEdit.nameRequired": "Naam is verplicht", "invEdit.symbolRequired": "Ticker / symbool is verplicht", "invEdit.title": "Belegging bewerken", "invEdit.toast.updated": "Belegging \"{name}\" bijgewerkt", @@ -1652,6 +1654,7 @@ "portfolioImport.defaultAssetClass": "Standaard activaklasse", "portfolioImport.defaultType": "Standaardtype", "portfolioImport.desc": "Importeer transacties (aankopen, verkopen, dividenden, kosten) uit een CSV van een broker of beurs. Wijs de kolommen toe aan je portefeuillevelden.", + "portfolioImport.duplicateColumnsWarning": "Dezelfde CSV-kolom is aan meerdere velden gekoppeld: {columns}. Elk gekoppeld veld neemt de waarde van die kolom over β€” controleer of dit de bedoeling is.", "portfolioImport.newCustom": "Nieuwe aangepaste parser", "portfolioImport.parserNamePlaceholder": "bijv. Degiro, Binance", "portfolioImport.parserSource": "Parser", diff --git a/openapi.yaml b/openapi.yaml index 39dcf600..890e4264 100644 --- a/openapi.yaml +++ b/openapi.yaml @@ -656,8 +656,6 @@ components: type: number currency: type: string - balance: - type: number category_id: type: integer nullable: true @@ -3636,7 +3634,7 @@ paths: tags: [PlannedTransactions] responses: "200": - description: Due-soon list + description: Due-soon list (canonical `{items, total}` body; `days` echoes the effective window) content: application/json: schema: @@ -3645,9 +3643,17 @@ paths: - type: object properties: data: - type: array - items: - $ref: "#/components/schemas/PlannedTransaction" + type: object + required: [items, total, days] + properties: + items: + type: array + items: + $ref: "#/components/schemas/PlannedTransaction" + total: + type: integer + days: + type: integer /api/planned-transactions/match-suggestions: get: @@ -3662,7 +3668,7 @@ paths: tags: [PlannedTransactions] responses: "200": - description: Match suggestions + description: Match suggestions (canonical `{items, total}` body) content: application/json: schema: @@ -3671,31 +3677,37 @@ paths: - type: object properties: data: - type: array - items: - type: object - properties: - planned: + type: object + required: [items, total] + properties: + total: + type: integer + items: + type: array + items: type: object properties: - id: { type: integer } - recipient_id: { type: integer, nullable: true } - recipient_name: { type: string, nullable: true } - amount: { type: number } - planned_date: { type: string } - currency: { type: string, nullable: true } - is_recurring: { type: boolean } - candidates: - type: array - items: - type: object - properties: - id: { type: integer } - recipient_name: { type: string, nullable: true } - amount: { type: number } - transaction_date: { type: string } - currency: { type: string, nullable: true } - memo: { type: string, nullable: true } + planned: + type: object + properties: + id: { type: integer } + recipient_id: { type: integer, nullable: true } + recipient_name: { type: string, nullable: true } + amount: { type: number } + planned_date: { type: string } + currency: { type: string, nullable: true } + is_recurring: { type: boolean } + candidates: + type: array + items: + type: object + properties: + id: { type: integer } + recipient_name: { type: string, nullable: true } + amount: { type: number } + transaction_date: { type: string } + currency: { type: string, nullable: true } + memo: { type: string, nullable: true } /api/planned-transactions/{id}: parameters: @@ -3824,11 +3836,24 @@ paths: tags: [Info] responses: "200": - description: Bank list + description: Bank list (canonical `{items, total}` body) content: application/json: schema: - $ref: "#/components/schemas/Envelope" + allOf: + - $ref: "#/components/schemas/Envelope" + - type: object + properties: + data: + type: object + required: [items, total] + properties: + items: + type: array + items: + type: string + total: + type: integer /api/info/supported-adapters: get: @@ -3837,11 +3862,29 @@ paths: tags: [Info] responses: "200": - description: Adapter list + description: Adapter list (canonical `{items, total}` body) content: application/json: schema: - $ref: "#/components/schemas/Envelope" + allOf: + - $ref: "#/components/schemas/Envelope" + - type: object + properties: + data: + type: object + required: [items, total] + properties: + items: + type: array + items: + type: object + required: [key, name] + properties: + key: { type: string } + name: { type: string } + adapter_class: { type: string } + total: + type: integer /api/info/transaction-count: get: @@ -4767,11 +4810,24 @@ paths: enum: [basic, full] responses: "200": - description: Quote data + description: Quote data (canonical `{items, total}` body) content: application/json: schema: - $ref: "#/components/schemas/Envelope" + allOf: + - $ref: "#/components/schemas/Envelope" + - type: object + properties: + data: + type: object + required: [items, total] + properties: + items: + type: array + items: + type: object + total: + type: integer /api/market/chart: get: @@ -4794,11 +4850,32 @@ paths: type: string responses: "200": - description: OHLC chart data + description: OHLC chart data (canonical `{items, total}` series with `symbol`/`currency` alongside) content: application/json: schema: - $ref: "#/components/schemas/Envelope" + allOf: + - $ref: "#/components/schemas/Envelope" + - type: object + properties: + data: + type: object + required: [items, total] + properties: + symbol: { type: string } + currency: { type: string } + items: + type: array + items: + type: object + properties: + time: { type: number } + close: { type: number } + high: { type: number } + low: { type: number } + volume: { type: number } + total: + type: integer /api/market/news: get: @@ -4813,11 +4890,24 @@ paths: type: string responses: "200": - description: News items + description: News items (canonical `{items, total}` body) content: application/json: schema: - $ref: "#/components/schemas/Envelope" + allOf: + - $ref: "#/components/schemas/Envelope" + - type: object + properties: + data: + type: object + required: [items, total] + properties: + items: + type: array + items: + type: object + total: + type: integer # ─── Watchlist ──────────────────────────────────────────────────────────────── /api/watchlist: @@ -5423,7 +5513,7 @@ paths: tags: [AI] responses: "200": - description: Model list + description: Model list (canonical `{items, total}` body) content: application/json: schema: @@ -5433,12 +5523,14 @@ paths: properties: data: type: object - required: [models] + required: [items, total] properties: - models: + items: type: array items: $ref: "#/components/schemas/OllamaModel" + total: + type: integer /api/ai/conversations: get: @@ -6753,11 +6845,24 @@ paths: tags: [Admin] responses: "200": - description: Request metrics + description: Request metrics (canonical `{items, total}` body) content: application/json: schema: - $ref: "#/components/schemas/Envelope" + allOf: + - $ref: "#/components/schemas/Envelope" + - type: object + properties: + data: + type: object + required: [items, total] + properties: + items: + type: array + items: + type: object + total: + type: integer /api/admin/endpoints: get: @@ -7100,7 +7205,7 @@ paths: default: isin responses: "200": - description: "Stored mappings ({ mappings: [...] })" + description: "Stored mappings (canonical `{items, total}` body)" content: application/json: schema: @@ -7131,7 +7236,7 @@ paths: type: object responses: "200": - description: Updated mapping set + description: Updated mapping set (canonical `{items, total}` body) content: application/json: schema: @@ -7229,7 +7334,7 @@ paths: tags: [Research] responses: "200": - description: "{ providers: [{ provider, label, envVar, configured, source, masked }] }" + description: "Canonical `{items, total}` body; each item is { provider, label, envVar, configured, source, masked }" content: application/json: schema: @@ -7258,7 +7363,7 @@ paths: type: string responses: "200": - description: Updated masked statuses + description: Updated masked statuses (canonical `{items, total}` body) content: application/json: schema: diff --git a/scripts/with-test-db.sh b/scripts/with-test-db.sh index 83a13726..49017270 100755 --- a/scripts/with-test-db.sh +++ b/scripts/with-test-db.sh @@ -74,7 +74,12 @@ docker run -d --rm \ printf '[test-db] Waiting for postgres' i=0 -until docker exec "$CONTAINER" pg_isready -U vision_test -d vision_test >/dev/null 2>&1; do +# Probe over TCP (-h 127.0.0.1): during initdb the entrypoint runs a TEMPORARY +# server that answers on the unix socket only β€” a socket probe reports ready, +# then the temp server shuts down and the migration's first TCP connect dies +# with "Connection terminated unexpectedly". TCP only answers once the final +# server is up. +until docker exec "$CONTAINER" pg_isready -h 127.0.0.1 -U vision_test -d vision_test >/dev/null 2>&1; do i=$((i + 1)) if [ "$i" -gt 60 ]; then echo ' timed out.' >&2