feat(fx): real applied rate for FCY conversions (issue #253)

CLAUDE.md

@@ -298,6 +298,14 @@ When adding a new section (like 721), follow this checklist:
 - **Legal-citation (corrected, retained)**: divisa is **Art. 33.1** (transmisión − adquisición), timing **Art. 14.2.e** ("se imputará en el momento del cobro o del pago"), FIFO-for-currency rationale **DGT V2324-10**; NOT Art. 37.1.l (which governs "incorporaciones que no derivan de transmisión"). The stale "Art. 37.1.l" comment in `report.ts`'s FX block was corrected alongside this work (see the #225 citation correction above).
 - **Scope/filter (unchanged)**: only `assetCategory` STK/FUND/BOND, `currency ≠ EUR`, ecb-resolvable; crypto permutas (proceeds in a coin, not fiat — crypto valuation path), options/FOP/FSFOP, and the CASH conversions already handled by `extractFxEvents` are excluded; SHORT legs are skipped (inflow at open, outflow at close — the long-side carry model can't represent them). Gated by monodivisa (`skipFx`), like the rest of the FX engine.
 
+### Real applied FX rate for actual conversions (issue #253 — carves out the "never fxRateToBase / ECB default" rule)
+- **What**: a CASH EUR↔FCY conversion may now carry `Trade.realEurAmount` — the REAL, spread-laden EUR principal the broker actually moved on THAT conversion, the separately-itemized `commission`/fee EXCLUDED. When present (finite and positive), the FX engine values that conversion at the effective real rate `realEurAmount / |quantity|` instead of the ECB mid-rate, so the broker's hidden FX spread is captured. The commission is STILL applied on top (Art. 35), so `realEurAmount + commission` = the full real cost of the conversion — the fee is never folded into `realEurAmount` (no double-count). This is a real cash amount FROM the statement — it is **NOT** IBKR's `fxRateToBase`, which remains banned (the "always ECB, never fxRateToBase" rule under *Financial Precision* still holds for every implicit/derived rate). Absent → the engine falls back to ECB.
+- **Why**: the broker's FX spread is a deductible cost — **Art. 35.1.a** ("importe real de la enajenación") read with **35.1.b** ("gastos … inherentes"). ECB-mid silently DROPS that spread, overstating the divisa gain (or understating a loss) on a real conversion the taxpayer actually paid the spread on. Reserved for ACTUAL EUR↔FCY conversions where a real euro amount exists in the statement; **ECB stays the valuation** for FCY that arrives with NO euro price (stock proceeds, dividends, interest) and as the fallback whenever no `realEurAmount` is supplied. The maintainer already endorsed this carve-out (issue #242 → #253).
+- **V0282-22 nuance**: ECB is the *official* valuation rate, but the "importe real" is, by construction, "ECB minus the disclosed spread" — so this is the SAME figure expressed via the verifiable cash amount on the statement, not a competing rate. It does NOT overturn the ECB-default doctrine; it just prefers the broker's own euro cash figure when the statement discloses it.
+- **HOW / the load-bearing INVARIANT (symmetry)**: ONE shared private helper `FxFifoEngine.effectiveRate(event)` (in `fx-fifo.ts`) returns `realEurAmount/|quantity|` when present else `event.ecbRate`, and is called by BOTH `addLot` (acquire) AND `consumeLots` (dispose). So the acquire and dispose legs of a tracked round-trip are ALWAYS valued on the SAME basis — never a half-real/half-ECB mix, which is exactly the asymmetry that would fabricate a phantom gain. When both legs carry `realEurAmount`, the realized FX gain is `realEurReceived − realEurPaid` EXACTLY. Absent field → `effectiveRate` returns `ecbRate` → byte-identical to the prior pure-ECB behavior (the zero-change regression guard).
+- **Scope / who populates it**: `extractFxEvents` reads `trade.realEurAmount` (parsing it with `decimal.js`; a non-finite/non-positive/garbage value is dropped → ECB fallback) and attaches it to BOTH the acquire and dispose events it emits. **Lightyear** populates the field today: its conversion rows come in a same-timestamp PAIR (one leg per currency), and `parsers/lightyear.ts` harvests the EUR leg's `|Net Amt.|` (real principal, fee excluded) and stamps it on the non-EUR CASH trade. FCY↔FCY conversions (no EUR leg) leave it unset → ECB. Every OTHER broker falls back to ECB until its parser sets the field. Skipped under monodivisa (`skipFx` — the whole FX engine is off), like the rest of the FX engine.
+- **Pinned by**: `tests/engine/fx-fifo.test.ts` ("real applied EUR amount (FX spread capture — issue #253)") at the raw-`processEvents` level; `tests/parsers/lightyear-fx-realrate.test.ts` (the parser stamps the field, and `realEurAmount + |fee| == |EUR Gross|`); and `tests/integration/fx-real-rate-e2e.test.ts` end-to-end through `generateTaxReport` and `lightyearParser.parse` → `generateTaxReport` (spread capture = smaller gain by exactly the spread, symmetric reconciliation = realReceived − realPaid, no-field byte-identical, no fee double-count).
+
 ### FX-FIFO movement trace (opt-in audit ledger — issue #230)
 - **What**: an opt-in `ReportOptions.fxTrace` flag on `generateTaxReport`; when true, `TaxSummary.fxTrace` is populated with an `FxTraceEvent[]` — the full all-year FX-FIFO movement ledger (kinds `acquire`/`dispose`/`park`/`unpark`/`discard`/`profit`, each carrying the running pool and parked FCY balances after the event). Produced by the engine itself: `FxFifoEngine.enableTrace()` records every movement, `getTrace()` returns it. Serialize the array with `serializeFxTrace(trace, "jsonl" | "csv")` (type `FxTraceFormat`) in `src/generators/fx-trace.ts`.
 - **Why**: lets a developer/advisor audit exactly how a 1633/1637 figure was built and reconcile it by hand, and makes the carry-basis engine mechanically testable via golden-ledger tests. Born from issue #230 — the reporter was keeping a hand `console.log` ledger to verify the FX math; this turns that into a first-class, lossless artifact.

src/engine/fx-fifo.ts

@@ -63,6 +63,8 @@ export interface FxEvent {
   trigger: FxTrigger;
   /** Commission in EUR (positive = cost paid). Increases cost basis on BUY, reduces proceeds on SELL. */
   commissionEur?: Decimal;
+  /** Real EUR value applied to this conversion (spread-laden), BEFORE commission. When present, the effective rate realEurAmount/|quantity| is used instead of ecbRate (issue #253). */
+  realEurAmount?: Decimal;
   /**
    * Discriminator for the carry-basis stock events. Absent → a plain
    * acquire/dispose driven by signed `quantity` (the original behavior).
@@ -406,10 +408,28 @@ export class FxFifoEngine {
         }
       }
 
+      // Real EUR cash the broker applied to THIS conversion (FX principal, spread
+      // embedded, separate commission EXCLUDED). When present and a finite positive
+      // amount, the engine values the conversion at the effective real rate
+      // (realEurAmount/|quantity|) instead of ecbRate (issue #253). Both legs of a
+      // tracked round-trip carry it so acquire and dispose value on the same basis.
+      // Not finite/positive → omit it → ECB fallback. Commission handling unchanged.
+      let realEurAmount: Decimal | undefined;
+      if (trade.realEurAmount !== undefined) {
+        try {
+          const real = new Decimal(trade.realEurAmount);
+          if (real.isFinite() && real.abs().greaterThan(0)) {
+            realEurAmount = real.abs();
+          }
+        } catch {
+          // Unparseable string (decimal.js throws on garbage) → omit → ECB fallback.
+        }
+      }
+
       if (acquiring) {
-        events.push({ date, currency: trade.currency, quantity: amount, ecbRate, trigger: "conversion", commissionEur });
+        events.push({ date, currency: trade.currency, quantity: amount, ecbRate, trigger: "conversion", commissionEur, realEurAmount });
       } else {
-        events.push({ date, currency: trade.currency, quantity: amount.negated(), ecbRate, trigger: "conversion", commissionEur });
+        events.push({ date, currency: trade.currency, quantity: amount.negated(), ecbRate, trigger: "conversion", commissionEur, realEurAmount });
       }
     }
 
@@ -752,12 +772,33 @@ export class FxFifoEngine {
       || exch === "FXCONV" || notes.includes("AFX");
   }
 
+  /** Effective EUR-per-FCY rate for an event: the broker's real applied rate
+   *  (realEurAmount / |quantity|) when supplied (issue #253), else the ECB rate.
+   *  ONE helper used by BOTH addLot and consumeLots so the acquire and dispose
+   *  legs are ALWAYS valued on the same basis — never a half-real/half-ECB mix. */
+  private static effectiveRate(event: FxEvent): Decimal {
+    // A Decimal is always truthy, so re-check finite & strictly-positive here
+    // (defense-in-depth, like the rest of this file): a 0/negative/NaN
+    // realEurAmount must fall back to ECB, never zero out (or sign-flip) the
+    // conversion. A real EUR principal is positive by definition, so a negative
+    // is rejected, not silently absorbed via abs().
+    if (
+      event.realEurAmount &&
+      event.realEurAmount.isFinite() &&
+      event.realEurAmount.greaterThan(0) &&
+      event.quantity.abs().greaterThan(0)
+    ) {
+      return event.realEurAmount.div(event.quantity.abs());
+    }
+    return event.ecbRate;
+  }
+
   private addLot(event: FxEvent): void {
     // Defense-in-depth: never create a lot for a non-positive quantity — costPerUnit
     // would be 0/0 = NaN and silently poison all later FIFO math for this currency.
     if (!event.quantity.greaterThan(0)) return;
     // Commission increases the EUR cost of acquiring the lot
-    const baseCost = event.quantity.mul(event.ecbRate);
+    const baseCost = event.quantity.mul(FxFifoEngine.effectiveRate(event));
     const totalCost = event.commissionEur ? baseCost.plus(event.commissionEur) : baseCost;
     const costPerUnit = totalCost.div(event.quantity);
 
@@ -787,7 +828,8 @@ export class FxFifoEngine {
       entry.count++;
       entry.totalQty = entry.totalQty.plus(remaining);
       this.fxMissing.set(event.currency, entry);
-      const proceedsEur = remaining.mul(event.ecbRate);
+      const effRate = FxFifoEngine.effectiveRate(event);
+      const proceedsEur = remaining.mul(effRate);
       const netProceeds = event.commissionEur ? proceedsEur.minus(event.commissionEur) : proceedsEur;
       this.disposals.push({
         currency: event.currency,
@@ -801,16 +843,17 @@ export class FxFifoEngine {
         holdingPeriodDays: 0,
         lotId: "UNKNOWN",
       });
-      this.record("dispose", event, { quantityFcy: remaining, rate: event.ecbRate, costBasisEur: netProceeds, proceedsEur: netProceeds, gainLossEur: new Decimal(0), lotId: "UNKNOWN", note: "sin lotes previos → ganancia FX = 0" });
+      this.record("dispose", event, { quantityFcy: remaining, rate: effRate, costBasisEur: netProceeds, proceedsEur: netProceeds, gainLossEur: new Decimal(0), lotId: "UNKNOWN", note: "sin lotes previos → ganancia FX = 0" });
       return;
     }
 
+    const effRate = FxFifoEngine.effectiveRate(event);
     while (remaining.greaterThan(0) && lots.length > 0) {
       const lot = lots[0]!;
       const consumed = Decimal.min(remaining, lot.quantity);
 
       // Commission reduces proceeds, distributed proportionally across consumed lots
-      let proceedsEur = consumed.mul(event.ecbRate);
+      let proceedsEur = consumed.mul(effRate);
       if (event.commissionEur) {
         const proportion = consumed.div(totalQty);
         proceedsEur = proceedsEur.minus(event.commissionEur.mul(proportion));
@@ -840,15 +883,15 @@ export class FxFifoEngine {
       }
 
       remaining = remaining.minus(consumed);
-      this.record("dispose", event, { quantityFcy: consumed, rate: event.ecbRate, costBasisEur, proceedsEur, gainLossEur: proceedsEur.minus(costBasisEur), lotId: lotIdConsumed, lotAcquireDate: lot.acquireDate });
+      this.record("dispose", event, { quantityFcy: consumed, rate: effRate, costBasisEur, proceedsEur, gainLossEur: proceedsEur.minus(costBasisEur), lotId: lotIdConsumed, lotAcquireDate: lot.acquireDate });
     }
 
     if (remaining.greaterThan(0)) {
       const entry = this.fxMissing.get(event.currency) ?? { count: 0, totalQty: new Decimal(0) };
       entry.count++;
       entry.totalQty = entry.totalQty.plus(remaining);
       this.fxMissing.set(event.currency, entry);
-      let proceedsEur = remaining.mul(event.ecbRate);
+      let proceedsEur = remaining.mul(effRate);
       if (event.commissionEur) {
         const proportion = remaining.div(totalQty);
         proceedsEur = proceedsEur.minus(event.commissionEur.mul(proportion));
@@ -865,7 +908,7 @@ export class FxFifoEngine {
         holdingPeriodDays: 0,
         lotId: "UNKNOWN",
       });
-      this.record("dispose", event, { quantityFcy: remaining, rate: event.ecbRate, costBasisEur: proceedsEur, proceedsEur, gainLossEur: new Decimal(0), lotId: "UNKNOWN", note: "lotes insuficientes → ganancia FX = 0" });
+      this.record("dispose", event, { quantityFcy: remaining, rate: effRate, costBasisEur: proceedsEur, proceedsEur, gainLossEur: new Decimal(0), lotId: "UNKNOWN", note: "lotes insuficientes → ganancia FX = 0" });
     }
   }
 

src/parsers/lightyear.ts

@@ -125,21 +125,51 @@ function parseLightyearCsv(lines: string[]): Statement {
   const trades: Trade[] = [];
   const cashTransactions: CashTransaction[] = [];
 
-  // Pre-scan: collect FX conversion fees by timestamp.
+  // Pre-scan: collect FX conversion fees AND the real EUR principal by timestamp.
   // Lightyear emits conversion pairs (one leg per currency). The fee can appear
   // on either leg (often on the EUR leg, which we skip). Group by timestamp
   // and extract the fee so it can be applied to the non-EUR trade.
   const conversionFees = new Map<string, { fee: string; currency: string }>();
+  // The EUR leg's |Net Amt.| is the real spread-laden EUR principal Lightyear
+  // actually moved (fee EXCLUDED — the engine applies our -Fee commission on
+  // top). Stored per timestamp so the non-EUR trade can be valued at the real
+  // applied rate instead of ECB (Art. 35.1 LIRPF; issue #253). FCY↔FCY
+  // conversions have no EUR leg → left unset → engine falls back to ECB.
+  const conversionEurAmounts = new Map<string, string>();
+  // Timestamps with MORE THAN ONE EUR conversion leg (two different conversions
+  // in the same second). The pairing key is only second-resolution, so we cannot
+  // tell which non-EUR leg owns which EUR principal — attaching the wrong one
+  // would mis-value an entire conversion into casillas 1633/1637. Such timestamps
+  // are excluded so the engine uses the ECB rate (a correct mid-rate beats a
+  // wrong real principal).
+  const ambiguousEurTimestamps = new Set<string>();
   for (let i = 1; i < lines.length; i++) {
     const line = lines[i]!.trim();
     if (!line) continue;
     const fields = parseCsvLine(line, ",");
     const txType = (fields[cols.type] ?? "").trim().toLowerCase();
     if (txType !== "conversion") continue;
-    const feeDec = toFiniteDecimal(fields[cols.fee] ?? "0");
-    if (feeDec.isZero()) continue;
     const dateRaw = (fields[cols.date] ?? "").trim();
     const ccy = (fields[cols.ccy] ?? "EUR").trim();
+
+    // Harvest the EUR leg's real Net Amt. (the fee-excluded principal). No fee
+    // gate — a fee-free conversion still carries a real EUR principal.
+    if (ccy === "EUR") {
+      if (conversionEurAmounts.has(dateRaw)) {
+        // A second EUR leg at this timestamp → ambiguous pairing; drop to ECB.
+        ambiguousEurTimestamps.add(dateRaw);
+      } else {
+        // Use the EUR leg's Net Amt. ONLY — it is the real principal with the fee
+        // EXCLUDED (the engine applies our -Fee commission on top). Gross is
+        // fee-INCLUSIVE, so falling back to it would double-count the fee; if Net
+        // is empty we omit the field and let the engine use ECB instead.
+        const eurNet = toFiniteDecimal(fields[cols.netAmount] ?? "0");
+        if (!eurNet.isZero()) conversionEurAmounts.set(dateRaw, eurNet.abs().toString());
+      }
+    }
+
+    const feeDec = toFiniteDecimal(fields[cols.fee] ?? "0");
+    if (feeDec.isZero()) continue;
     const existing = conversionFees.get(dateRaw);
     if (!existing || feeDec.abs().greaterThan(new Decimal(existing.fee).abs())) {
       conversionFees.set(dateRaw, { fee: feeDec.abs().toString(), currency: ccy });
@@ -256,6 +286,13 @@ function parseLightyearCsv(lines: string[]): Statement {
       const commissionVal = pairedFee ? pairedFee.fee : new Decimal(fee).abs().toString();
       const commCurrency = pairedFee ? pairedFee.currency : currency;
 
+      // Real EUR principal from the paired EUR leg (|Net Amt.|, spread embedded,
+      // fee excluded). Absent for FCY↔FCY conversions, or dropped when >1 EUR leg
+      // shares this timestamp (ambiguous pairing) → omit so the engine uses ECB.
+      const realEurAmount = ambiguousEurTimestamps.has(dateRaw)
+        ? undefined
+        : conversionEurAmounts.get(dateRaw);
+
       trades.push({
         tradeID: `lightyear-fx-${reference || `${tradeDate}-${currency}-${i}`}`,
         accountId: "",
@@ -278,6 +315,7 @@ function parseLightyearCsv(lines: string[]): Statement {
         exchange: "LIGHTYEAR",
         commissionCurrency: commCurrency,
         commission: `-${commissionVal}`,
+        ...(realEurAmount ? { realEurAmount } : {}),
         taxes: "0",
         multiplier: "1",
       });

src/types/ibkr.ts

@@ -90,6 +90,20 @@ export interface Trade {
   underlyingIsin?: string;
   /** IBKR order ID — same value across partial-fill executions of one order. Used by the parser to collapse executions into one trade. */
   ibOrderID?: string;
+  /**
+   * Real EUR value the broker actually applied to a EUR↔FCY CASH conversion —
+   * the spread-laden euro amount moved, BEFORE the separately-itemized
+   * `commission`/fee (which the FX engine still applies on top, Art. 35). When
+   * present, the FX FIFO engine values THIS conversion at the effective real rate
+   * (`realEurAmount / |quantity|`) instead of the ECB reference rate, so the
+   * broker's hidden FX spread is captured as a deductible cost (Art. 35.1 LIRPF;
+   * issue #253). Absent → the engine falls back to the official ECB rate (the
+   * default for every other conversion and for valuing FCY that arrives with no
+   * euro price). This is a real cash amount FROM the statement — NOT IBKR's
+   * `fxRateToBase` (which remains banned). Only meaningful on `assetCategory:
+   * "CASH"` conversion trades; ignored elsewhere and under monodivisa (skipFx).
+   */
+  realEurAmount?: string;
 }
 
 export interface CashTransaction {