From 6c5117e67b0e74182bcee3f47e605ec08394a40c Mon Sep 17 00:00:00 2001 From: wegamekinglc Date: Wed, 15 Jul 2026 13:11:27 +0800 Subject: [PATCH 01/19] docs: specify repository documentation reconciliation --- ...repository-documentation-reconciliation.md | 142 ++++++++++++++++++ 1 file changed, 142 insertions(+) create mode 100644 .codex/artifacts/specs/repository-documentation-reconciliation.md diff --git a/.codex/artifacts/specs/repository-documentation-reconciliation.md b/.codex/artifacts/specs/repository-documentation-reconciliation.md new file mode 100644 index 000000000..ad0c7edfe --- /dev/null +++ b/.codex/artifacts/specs/repository-documentation-reconciliation.md @@ -0,0 +1,142 @@ +# Repository Documentation Reconciliation - Specification + +## Source + +- User request: repository-wide documentation review and update, approved 2026-07-15. +- Baseline: merged `master` at `1589089bdf10df352ce5cf9cde963fd6b51a4f95`. +- Published-document set: `.github/scripts/check_docs.py` (`DOCS`, 33 Markdown files). +- Prior audit: `.codex/artifacts/reviews/codebase-documentation-audit.md` at commit `7b959542`. +- Latest capability references: `docs/methodology/xccy_calibration.md`, + `docs/methodology/yield_curve_jacobian.md`, and public C++/Python/Excel surfaces. + +## Problem Statement + +DAL's automated documentation checker passes, but it verifies structural integrity rather +than full agreement with the current implementation. Since the prior repository audit, +substantial numerical, runtime, packaging, curve, and XCCY work has landed. Most detailed +documentation is current, but several public claims, architecture flows, API spellings, +surface-availability statements, example listings, and indexes are incomplete or stale. + +The repository needs a source-backed audit of every published Markdown file, followed by +focused edits that describe the current library without turning overview pages into copies +of the methodology documents. + +## Goals + +- Audit all 33 Markdown files selected by `.github/scripts/check_docs.py` against the + current implementation. +- Revalidate every still-relevant finding in the 2026-07-10 codebase documentation audit. +- Correct inaccurate API names, quantitative claims, runtime ownership descriptions, + architecture flows, build commands, and cross-surface availability statements. +- Make major current capabilities discoverable from the root and documentation indexes. +- Record one auditable disposition for every published document: changed, verified current, + or intentionally unchanged with evidence. +- Publish the reconciliation as a documentation-only follow-up pull request from `master`. + +## Non-Goals + +- No C++, Python, Excel, web, CMake, workflow, or generated-file behavior changes. +- No attempt to add the missing Excel worksheet getter for the joint XCCY effective inverse; + document the current surface accurately instead. +- No hand edits under `dal-cpp/dal/auto/` or `dal-excel/auto/`. +- No edits to `CLAUDE.md`, `.claude/`, `AGENTS.md`, third-party documentation, build output, + or Codex skill instructions. +- No speculative API promises, migration roadmap, or implementation history in current-state + documentation. Historical context remains in `CHANGELOG.md`. +- No wholesale prose rewrite when a document is already accurate. + +## Functional Requirements + +- **FR1 - Complete published-document inventory:** Derive the audit set from the `DOCS` + expression in `.github/scripts/check_docs.py`, not from an independently maintained list. + The review artifact shall contain exactly one disposition for each selected document. +- **FR2 - Source-backed ground truth:** Validate claims against the closest authoritative + source: public headers for C++, pybind11 and `dal/api.py` for Python, handwritten plus + generated registrations for Excel, FastAPI/React source for web, CMake/presets/scripts for + build instructions, and tests/examples/benchmarks for observable workflows. +- **FR3 - Prior-audit reconciliation:** Recheck the findings in + `.codex/artifacts/reviews/codebase-documentation-audit.md` against current `master`. Current + docs shall not repeat limitations that have been fixed or omit material constraints that + remain. +- **FR4 - Root and index discoverability:** Keep `README.md` concise while adding links for + major current capabilities that are otherwise absent, including cross-currency pricing and + calibration. Correct the `docs/README.md` heading hierarchy and remove its obsolete claim + that analytic Jacobians are exclusive to `LOG_DISCOUNT`. +- **FR5 - Architecture accuracy:** `docs/architecture.md` shall describe immutable + operation-level `MarketFixingSnapshot_` ownership separately from the mutable process-wide + fixing store. Its calibration flow shall distinguish generic single/staged calibration, + staged XCCY basis calibration, and simultaneous domestic/foreign/basis XCCY calibration. +- **FR6 - Exact API spelling and availability:** Developer-facing identifiers shall match + compilable or callable surface names. In particular, C++ notional-mode constants shall use + `XccyNotionalMode_::Value_::*`. Documentation shall distinguish core/public staged XCCY + options and matrices from the narrower Python and Excel staged result surfaces. +- **FR7 - XCCY fixing contract:** The XCCY methodology shall document canonical + `FX[foreign/domestic]` names, reciprocal reverse-FX lookup, bidirectional reciprocity within + the implemented tolerance, positive finite snapshot observations, duplicate behavior, and + the unsettled-cashflow dependency closure for historical requests. +- **FR8 - Jacobian and inverse semantics:** The yield-curve Jacobian and XCCY methodology + shall state joint matrix placement, dimensions, solved-state construction, exact/bumped/ + approximate population rules, and the existing `tolerance_` scaling convention without + implying unavailable binding accessors. +- **FR9 - Examples and performance navigation:** Current documentation shall list all three + registered C++ XCCY examples and the installed-surface Python joint example. It shall + describe `xccy_perf` as a 24-row execution-smoke surface, including its started-MTM basket + and reset-aware calibration coverage, without presenting it as a paired regression gate. +- **FR10 - Component README accuracy:** Reconcile the core, public, Python, Excel, and web + READMEs with their actual current surfaces. The Python testing section shall include curve + and XCCY coverage. The Excel docs shall explicitly state which joint matrices are and are not + worksheet-visible. +- **FR11 - Changelog separation:** `CHANGELOG.md` may update its current-capability baseline + and retain qualifying dated history. Current-state documents shall not contain delivery + chronology, superseded plans, or historical implementation narrative. +- **FR12 - Verified no-change dispositions:** A document that needs no edit shall remain + untouched and be marked verified-current in the audit artifact with its inspected source + authorities. Passing `check_docs.py` alone is not sufficient evidence. + +## Non-Functional Requirements + +- **Accuracy:** Every changed technical claim must have direct evidence in current source, + tests, examples, generated registration, or build configuration. +- **Compatibility:** Documentation-only changes must not alter public or internal behavior. +- **Current-state style:** Methodology owns mathematical contracts and invariants; overview + pages summarize and link instead of duplicating detailed prose. +- **Markdown:** Preserve GitHub-renderable math, valid relative links and anchors, aligned + tables, no trailing whitespace, and final newlines. +- **Scope control:** The final tracked diff may contain published Markdown and Codex audit/ + planning artifacts only. Generated and production files must remain unchanged. +- **Reviewability:** Prefer small thematic commits so API corrections, architecture/ + methodology reconciliation, and navigation updates can be reviewed independently. + +## Inputs and Outputs + +| Input | Authority | Output | +|------------------------|------------------------------------------------------------------------|---------------------------------------------------------------------------| +| Published Markdown set | `.github/scripts/check_docs.py::DOCS` | One 33-row disposition matrix | +| C++ surface | Core/public headers and generated enums | Exact types, constants, ownership, and solver contracts | +| Python surface | `dal-python/src/bindings/`, `dal-python/src/dal/`, tests, examples | Callable names, result properties, and supported workflows | +| Excel surface | `dal-excel/src/`, interface markup, generated help, smoke tests | Worksheet functions, settings, selectors, and documented limitations | +| Build/runtime surface | CMake, presets, scripts, workflows, web source | Current setup, ownership, execution, and CI descriptions | +| Numerical behavior | Methodology implementation plus independent tests | Correct formulas, matrix semantics, tolerances, and applicability | + +## Acceptance Criteria + +- [ ] The audit artifact contains exactly one evidence-backed disposition for each of the 33 + Markdown files selected by `.github/scripts/check_docs.py`. +- [ ] Every still-relevant finding from the prior codebase documentation audit is either + corrected in current docs or recorded as already resolved with current evidence. +- [ ] The confirmed root/index, architecture, enum-spelling, XCCY fixing, joint inverse, + example, benchmark, Python testing, Excel availability, and changelog gaps are resolved. +- [ ] Root and component overview pages remain concise and link to methodology/API documents + for detailed contracts. +- [ ] `CLAUDE.md`, `.claude/`, generated files, third-party docs, production source, build + configuration, and workflows have no tracked changes. +- [ ] `python3 .github/scripts/check_docs.py` passes. +- [ ] `git diff --check` passes for the complete branch diff. +- [ ] Documented runnable commands changed by the branch are executed or validated against the + already-built current targets, including XCCY examples and `xccy_perf` when referenced. +- [ ] A fresh DAL reviewer reports no blocking documentation or source-consistency findings. +- [ ] The final branch is pushed as a new documentation-only PR from merged `master`. + +## Open Questions + +None. The user selected the repository-wide audit and approved the documentation-only design. From f6fed84b32edcd9b8a436a294df4f10e83f3f729 Mon Sep 17 00:00:00 2001 From: wegamekinglc Date: Wed, 15 Jul 2026 13:56:05 +0800 Subject: [PATCH 02/19] docs: plan repository documentation reconciliation --- ...repository-documentation-reconciliation.md | 521 ++++++++++++++++++ 1 file changed, 521 insertions(+) create mode 100644 .codex/artifacts/plans/2026-07-15-repository-documentation-reconciliation.md diff --git a/.codex/artifacts/plans/2026-07-15-repository-documentation-reconciliation.md b/.codex/artifacts/plans/2026-07-15-repository-documentation-reconciliation.md new file mode 100644 index 000000000..35aa8ba8c --- /dev/null +++ b/.codex/artifacts/plans/2026-07-15-repository-documentation-reconciliation.md @@ -0,0 +1,521 @@ +# Repository Documentation Reconciliation Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Reconcile every published DAL Markdown document with merged `master`, preserve historical planning artifacts outside `docs/`, and publish a source-backed documentation-only pull request. + +**Architecture:** Treat current headers, bindings, generated registrations, examples, tests, scripts, and workflows as ground truth. Keep root and component pages concise, place mathematical and ownership contracts in methodology/architecture pages, and record one disposition for each of the 33 baseline documents in a durable audit artifact. + +**Tech Stack:** GitHub-flavored Markdown, DAL C++17/public headers, pybind11 bindings, Excel/Machinist registrations, CMake/build scripts, Python documentation checker. + +## Global Constraints + +- Baseline is merged `master` at `1589089bdf10df352ce5cf9cde963fd6b51a4f95`. +- The controlling specification is `.codex/artifacts/specs/repository-documentation-reconciliation.md`. +- No C++, Python, Excel, web, CMake, workflow, or generated-file behavior changes. +- Do not edit `CLAUDE.md`, `.claude/`, `AGENTS.md`, third-party documentation, build output, generated files, or Codex skill instructions. +- Current-state docs contain no delivery chronology or superseded implementation narrative; history belongs in `CHANGELOG.md` or `.codex/artifacts/`. +- Root/component overviews summarize and link; methodology owns formulas, invariants, limitations, matrix units, and numerical contracts. +- Every changed technical claim must be traceable to current source, tests, examples, generated registration, or build configuration. +- Preserve GitHub-renderable math, valid relative links/anchors, aligned tables, final newlines, and no trailing whitespace. +- The final tracked diff may contain published Markdown and `.codex/artifacts/{plans,reviews,specs}/` only. + +--- + +### Task 1: Move historical implementation artifacts out of published documentation + +**Files:** +- Move: `docs/superpowers/plans/2026-07-12-unified-yield-curve-interpolation-aad.md` -> `.codex/artifacts/plans/2026-07-12-unified-yield-curve-interpolation-aad.md` +- Move: `docs/superpowers/plans/2026-07-12-ycinstrument-pricing-performance.md` -> `.codex/artifacts/plans/2026-07-12-ycinstrument-pricing-performance.md` +- Move: `docs/superpowers/plans/2026-07-12-zero-rate-parameterization.md` -> `.codex/artifacts/plans/2026-07-12-zero-rate-parameterization.md` +- Move: `docs/superpowers/specs/2026-07-12-ycinstrument-pricing-performance-design.md` -> `.codex/artifacts/specs/2026-07-12-ycinstrument-pricing-performance-design.md` +- Move: `docs/superpowers/specs/2026-07-12-zero-rate-parameterization-design.md` -> `.codex/artifacts/specs/2026-07-12-zero-rate-parameterization-design.md` + +**Interfaces:** +- Consumes: the current-state documentation rule and the baseline 33-file set selected by `.github/scripts/check_docs.py`. +- Produces: preserved planning history under the canonical Codex artifact directories and a 28-file current-state published-doc set. + +- [ ] **Step 1: Run the structural red check** + +```bash +python3 - <<'PY' +from pathlib import Path +historical = sorted(Path("docs/superpowers").rglob("*.md")) +assert not historical, "historical implementation artifacts remain in docs: " + ", ".join(map(str, historical)) +PY +``` + +Expected: FAIL listing exactly the five assigned plan/spec files. + +- [ ] **Step 2: Confirm that no destination collision or inbound published link exists** + +```bash +for source in docs/superpowers/plans/*.md docs/superpowers/specs/*.md; do + destination=".codex/artifacts/${source#docs/superpowers/}" + test ! -e "$destination" + rg -n --fixed-strings "$source" README.md CONTRIBUTING.md CHANGELOG.md dal-*/README.md docs --glob '*.md' || true +done +``` + +Expected: every destination is absent; no current-state document depends on the historical path. + +- [ ] **Step 3: Move each artifact without rewriting its historical content** + +```bash +mkdir -p .codex/artifacts/plans .codex/artifacts/specs +git mv docs/superpowers/plans/2026-07-12-unified-yield-curve-interpolation-aad.md .codex/artifacts/plans/ +git mv docs/superpowers/plans/2026-07-12-ycinstrument-pricing-performance.md .codex/artifacts/plans/ +git mv docs/superpowers/plans/2026-07-12-zero-rate-parameterization.md .codex/artifacts/plans/ +git mv docs/superpowers/specs/2026-07-12-ycinstrument-pricing-performance-design.md .codex/artifacts/specs/ +git mv docs/superpowers/specs/2026-07-12-zero-rate-parameterization-design.md .codex/artifacts/specs/ +``` + +- [ ] **Step 4: Run the structural green check and documentation checker** + +```bash +python3 - <<'PY' +import importlib.util +from pathlib import Path +assert not list(Path("docs/superpowers").rglob("*.md")) +spec = importlib.util.spec_from_file_location("check_docs", ".github/scripts/check_docs.py") +module = importlib.util.module_from_spec(spec) +spec.loader.exec_module(module) +assert len(module.DOCS) == 28, len(module.DOCS) +PY +python3 .github/scripts/check_docs.py +git diff --check +``` + +Expected: all commands exit 0; documentation integrity reports 28 Markdown files. + +- [ ] **Step 5: Commit** + +```bash +git add .codex/artifacts/plans .codex/artifacts/specs docs/superpowers +git commit -m "docs: move historical plans out of published docs" +``` + +### Task 2: Refresh root navigation, architecture, and current capability baseline + +**Files:** +- Modify: `README.md` +- Modify: `CHANGELOG.md` +- Modify: `docs/README.md` +- Modify: `docs/architecture.md` + +**Interfaces:** +- Consumes: current XCCY source/examples, fixing-snapshot ownership, current analytic-Jacobian representations, and the resolved runtime/build findings from the prior audit. +- Produces: concise root discoverability plus authoritative runtime ownership and calibration-flow navigation. + +- [ ] **Step 1: Run content red checks** + +```bash +rg -n "XCCY|Cross-Currency" README.md +rg -n "Why `LOG_DISCOUNT` is the parameterization that supports the analytic Jacobian" docs/README.md +rg -n "MarketFixingSnapshot_|CalibrateJointXccyMarket|CalibrateCrossCurrencyMarket" docs/architecture.md +``` + +Expected: the root search returns no match; the obsolete `LOG_DISCOUNT` sentence matches; architecture lacks the snapshot and XCCY entry points. + +- [ ] **Step 2: Update the root README without duplicating methodology** + +Add cross-currency pricing/calibration to the opening capability sentence. Under **Examples**, add a short cross-currency paragraph with direct links to: + +```markdown +- [reset-aware pricing](dal-cpp/examples/xccy_reset_pricing/) +- [staged basis calibration](dal-cpp/examples/xccy_curve_calibration/) +- [joint domestic/foreign/basis calibration](dal-cpp/examples/xccy_mtm_calibration/) +- [Python joint calibration](dal-python/examples/007.xccy_joint_calibration.py) +``` + +Add this concise methodology entry to the abbreviated list: + +```markdown +- [Cross-Currency Pricing and Calibration](docs/methodology/xccy_calibration.md) — fixed, resettable, and MTM swaps; immutable fixing snapshots; staged basis and simultaneous domestic/foreign/basis calibration +``` + +- [ ] **Step 3: Correct the documentation index and heading hierarchy** + +Promote `### Methodology (`methodology/`)` and `### Experimental (`experimental/`)` to level-two headings. Replace the log-discount exclusivity bullet with: + +```markdown + - Persistent log-discount coordinates, interpolation/extrapolation, basis weights, and participation in the shared analytic-Jacobian curve factory +``` + +Extend the XCCY and yield-Jacobian index summaries to mention immutable operation snapshots, joint matrix range layout, and effective-inverse scaling without copying formulas. + +- [ ] **Step 4: Add snapshot ownership and branch the architecture calibration flow** + +Add this runtime-state row, padded to the existing table widths: + +```markdown +| Market-fixing snapshot | Immutable operation-level value retained by reset-aware pricing/calibration results. An explicit snapshot is authoritative; when omitted, staged/joint XCCY calibration gathers required historical requests and copies the process-wide store once before solving. | +``` + +Replace the single calibration diagram with one common validation/residual prefix and three named branches: + +```text +instrument/convention builders + -> validated spec, curve layout, and model-rate residuals + -> single/staged curves: CalibrateCurve / CalibrateMultiCurve + -> staged XCCY basis: CalibrateCrossCurrencyMarket + -> supplied domestic/foreign blocks + basis parameters + one fixing snapshot + -> joint XCCY: CalibrateJointXccyMarket + -> domestic declarations + foreign declarations + basis declaration + -> one fixing snapshot + named parameter/residual ranges + -> exact or approximate underdetermined solve + -> eligible AAD analytic Jacobian or explicit bumped mode + -> solved curves, diagnostics, and optional matrices +``` + +Link the paragraph to `methodology/xccy_calibration.md` as well as the existing yield-curve pages. + +- [ ] **Step 5: Reconcile the changelog baseline and qualifying XCCY entry** + +Replace the baseline cross-currency bullet with current-state wording covering fixed/resettable/MTM pricing, immutable timestamped rate/FX snapshots, staged basis fitting, simultaneous domestic/foreign/basis calibration, and named Jacobian ranges. + +Remove the obsolete baseline bullet that presents analytic calibration as a `CurveJacobianMode`-only capability; analytic Jacobians now cover every implemented curve representation subject to normal eligibility gates. In the dated XCCY entry, avoid enum shorthand that looks like a callable C++ identifier and qualify the cross-surface statement: public C++ and joint Python expose both joint matrices, while Excel exposes the joint forward Jacobian/ranges but no effective-inverse worksheet getter. + +- [ ] **Step 6: Verify and commit** + +```bash +python3 .github/scripts/check_docs.py +git diff --check +rg -n "Cross-Currency Pricing and Calibration" README.md docs/README.md CHANGELOG.md +rg -n "Market-fixing snapshot|CalibrateCrossCurrencyMarket|CalibrateJointXccyMarket" docs/architecture.md +git add README.md CHANGELOG.md docs/README.md docs/architecture.md +git commit -m "docs: refresh architecture and navigation" +``` + +### Task 3: Correct quantitative methodology contracts + +**Files:** +- Modify: `docs/experimental/replicate-ptirds-single-currency-curve.md` +- Modify: `docs/methodology/aad.md` +- Modify: `docs/methodology/interpolation.md` +- Modify: `docs/methodology/matrix.md` +- Modify: `docs/methodology/random.md` + +**Interfaces:** +- Consumes: `/tmp/dal-doc-audit-quant.md` and the cited current source/tests. +- Produces: exact current-state numerical and runtime semantics without changing implementation. + +- [ ] **Step 1: Run terminology/formula red checks** + +```bash +rg -n "all-days|matches calendar = all|does better" docs/experimental/replicate-ptirds-single-currency-curve.md +rg -n "first-order accurate|not-a-knot|O\\(n m\\)" docs/methodology/interpolation.md docs/methodology/matrix.md +rg -n "full `SkipTo` support|re-seed via `Branch`/`Clone`|repositions the generator without replay" docs/methodology/random.md +``` + +Expected: each stale claim is present before editing. + +- [ ] **Step 2: Correct PTIRDS calendar reproduction and neutralize comparison language** + +State that the external target uses an every-day calendar, while DAL reproduces the supplied explicit dates with `Holidays::None()` plus `Unadjusted` accrual/payment/index conventions because `Holidays::None()` removes named holidays but still treats weekends as non-business days. Replace “does better” with the factual statement that eligible `LOG_DISCOUNT` calibration uses an AAD-derived analytic Jacobian. + +- [ ] **Step 3: Distinguish full-tape rewind from checkpoint rewind** + +Rename the AAD memory heading to `Mark / RewindToMark`. State exactly: + +```markdown +`RewindToMark(tape)` discards nodes recorded after the current mark and is used after each Monte Carlo path. `Rewind(tape)` resets the tape to the beginning of its recording and is used before a fresh simulation or calibration recording. +``` + +- [ ] **Step 4: Correct interpolation accuracy, formula, and boundary semantics** + +Describe piecewise-linear interpolation as degree-one/piecewise affine with `O(h^2)` interpolation error for a sufficiently smooth scalar function. Replace the cubic evaluation formula with: + +```math +f(x)=a f_i+b f_{i+1}-\frac{a b h^2}{6}\left[(1+a)f_i''+(1+b)f_{i+1}''\right]. +``` + +Describe order-3 endpoint boundaries as pinning the endpoint segment's third derivative to the supplied value; remove “not-a-knot family.” + +- [ ] **Step 5: Correct matrix notation and complexity** + +After the document's existing definitions (`a` super-diagonal, `c` sub-diagonal), use: + +```math +x_i=\frac{b_i-c_{i-1}x_{i-1}}{\beta_i}, +\qquad +x_{i-1}\leftarrow x_{i-1}-\frac{a_{i-1}}{\beta_{i-1}}x_i. +``` + +State that band Cholesky factorization is `O(n m^2)` for lower bandwidth `m`; band triangular solve/multiply is `O(n m)`. Explain that dense `CholeskyImpl` clips negative pivot residuals to zero and regularizes the reciprocal diagonal; it does not factor an explicitly shifted `A + lambda I` matrix. + +- [ ] **Step 6: Document implementation-specific random seeking** + +State that `SkipTo` is implementation-specific: Sobol reconstructs state directly; `ShuffledIRN_::SkipTo` is a no-op; its `Clone` restarts from the original seed; `Branch` creates another seeded generator. For MRG32, state that the current skip accounting matches antithetic `FillUniform`, while `FillNormal` consumes a fresh uniform per component and bypasses that cache, so MRG32 seeking must not be promised as replay-equivalent normal-path substreams. Keep Sobol as the verified normal-path seeking surface. + +- [ ] **Step 7: Verify and commit** + +```bash +python3 .github/scripts/check_docs.py +git diff --check +rg -n "RewindToMark|O\\(h\\^2\\)|O\\(n m\\^2\\)|implementation-specific" docs/methodology +git add docs/experimental/replicate-ptirds-single-currency-curve.md docs/methodology/aad.md docs/methodology/interpolation.md docs/methodology/matrix.md docs/methodology/random.md +git commit -m "docs: correct numerical methodology contracts" +``` + +### Task 4: Complete XCCY fixing, Jacobian, example, and performance methodology + +**Files:** +- Modify: `docs/methodology/xccy_calibration.md` +- Modify: `docs/methodology/yield_curve_jacobian.md` + +**Interfaces:** +- Consumes: core XCCY/fixing implementations, `/tmp/dal-doc-audit-curves.md`, and existing generic inverse-risk convention. +- Produces: authoritative core contracts and one concise cross-surface availability summary for Task 5 to mirror. + +- [ ] **Step 1: Run XCCY methodology red checks** + +```bash +rg -n "FX\\[foreign/domestic\\]|reciprocal|1e-10|totalParameters x totalResiduals|xccy_perf|007.xccy_joint_calibration" docs/methodology/xccy_calibration.md docs/methodology/yield_curve_jacobian.md +``` + +Expected: the required terms are absent or incomplete. + +- [ ] **Step 2: Complete reset and immutable-snapshot rules** + +Document all of these current rules together: + +- `FxIndexName(domestic, foreign)` is `FX[foreign/domestic]`. +- Direct lookup wins; if absent, the reciprocal of the reverse canonical observation is used. +- If both directions exist at one timestamp, `abs(direct * reverse - 1) <= 1e-10` is required. +- Index names are non-empty, timestamps valid, and observations positive and finite. +- Core/Python map inputs provide at most one value per pair; Excel parallel arrays reject duplicates explicitly. +- Historical requirements are deduplicated and include only observations feeding unsettled coupons, notionals, or MTM reset dependencies. +- Explicit snapshots, including explicit empty snapshots, are authoritative; omitted snapshots are copied once from global fixings. + +- [ ] **Step 3: Add matrix dimensions, solved-state construction, scaling, and population rules** + +State staged shapes as `nInstruments x nBasisParameters` and `nBasisParameters x nInstruments`. State joint shapes as `totalResiduals x totalParameters` and `totalParameters x totalResiduals`. Explain that the exposed forward Jacobian is unscaled, while the effective inverse is the weighted pseudoinverse of the solver's tolerance-scaled Jacobian captured at the accepted exact solution. Include: + +```math +\Delta x = \mathrm{effJacobianInverse}\,\Delta q / \mathrm{tolerance}. +``` + +State: exact analytic may produce both matrices; exact bumped produces only the effective inverse; approximate produces neither; options suppress each independently. Both matrices live on the top-level joint result, not per-group diagnostics. + +- [ ] **Step 4: Add a compact surface-availability table** + +Use these exact rows: + +| Surface | Staged XCCY | Joint XCCY | +|---------|-------------|------------| +| Core/public C++ | Full options, forward Jacobian, and effective inverse | Full options and both top-level matrices | +| Python | Default staged solve with market/FX-forward and fit diagnostics; no staged matrix bindings | Options, named ranges, forward Jacobian, and effective inverse | +| Excel | Basis handle and fit diagnostics; no staged matrix views | Options plus forward-Jacobian/range views; no effective-inverse worksheet getter | + +- [ ] **Step 5: Complete examples and performance smoke navigation** + +List `xccy_curve_calibration`, `xccy_reset_pricing`, `xccy_mtm_calibration`, and `dal-python/examples/007.xccy_joint_calibration.py`. Correct the joint example wording to “five declaration blocks across the domestic, foreign, and basis groups.” Add a **Performance Smoke Surface** subsection: `xccy_perf` emits 24 rows covering four pricing cases (future fixed/resettable/MTM and started MTM), staged calibration including the reset-aware analytic case, and joint calibration; Linux/Windows execute it to completion, but it is not in the paired base/head regression allowlist. + +- [ ] **Step 6: Verify documented workflows and commit** + +```bash +build/Release-linux/dal-cpp/examples/xccy_curve_calibration/xccy_curve_calibration +build/Release-linux/dal-cpp/examples/xccy_reset_pricing/xccy_reset_pricing +build/Release-linux/dal-cpp/examples/xccy_mtm_calibration/xccy_mtm_calibration +PYTHONPATH="$PWD/build/stage/Release-linux" dal-python/.venv/bin/python dal-python/examples/007.xccy_joint_calibration.py +DAL_NUM_THREADS=4 build/Release-linux/dal-cpp/benchmarks/xccy_perf/xccy_perf | python3 -c 'import re,sys; row=re.compile(r"^\\S(?:.*?\\S)?\\s+[0-9]+(?:\\.[0-9]+)?\\s+(?:ns|us|ms|s)\\s+[0-9]+(?:\\.[0-9]+)?\\s+(?:ns|us|ms|s)\\s+[0-9]+(?:\\.[0-9]+)?\\s+(?:ns|us|ms|s)\\s+[0-9]+\\s*$"); names=[line.rsplit(None,7)[0] for line in sys.stdin if row.match(line)]; assert len(names)==len(set(names))==24, (len(names),len(set(names)))' +python3 .github/scripts/check_docs.py +git diff --check +git add docs/methodology/xccy_calibration.md docs/methodology/yield_curve_jacobian.md +git commit -m "docs: complete XCCY methodology coverage" +``` + +### Task 5: Reconcile public API and Python/Excel README availability + +**Files:** +- Modify: `docs/public-api.md` +- Modify: `dal-python/README.md` +- Modify: `dal-excel/README.md` + +**Interfaces:** +- Consumes: the Task 4 terminology/table, exact pybind11 fields, and exact Excel accepted selectors/settings. +- Produces: copy-safe C++ identifiers and binding-specific capability statements without promising missing accessors. + +- [ ] **Step 1: Run API-surface red checks** + +```bash +rg -n "XccyNotionalMode_::\\{FIXED|effJacobianInverse|staged" docs/public-api.md dal-python/README.md dal-excel/README.md +``` + +Expected: invalid grouped C++ spelling exists; staged/joint matrix distinctions are absent or incomplete. + +- [ ] **Step 2: Correct C++ enum spelling and staged/joint API availability** + +In `docs/public-api.md`, spell all constants fully: + +```text +XccyNotionalMode_::Value_::FIXED +XccyNotionalMode_::Value_::RESETTABLE +XccyNotionalMode_::Value_::MARK_TO_MARKET +``` + +State that C++ staged diagnostics own forward/effective-inverse matrices and options through the included core types; joint results own both top-level matrices. Existing `JointXccyResult*` facade helpers expose the documented handles/vectors/ranges, while C++ reads `effJacobianInverse_` directly because no dedicated facade getter exists. + +- [ ] **Step 3: Correct Python staged/joint wording and test inventory** + +State that staged Python exposes the one-argument default solve, calibrated market, FX forwards, and scalar/vector fit diagnostics only. State that joint Python exposes `JointXccyCalibrationOptions_`, `jacobian_at_solution`, `eff_jacobian_inverse`, and named ranges. Add the joint inverse dimensions and `/ tolerance_` risk-transform warning with a methodology link. Extend **Testing** with curve construction/calibration, staged XCCY, resettable/MTM snapshots, and joint XCCY diagnostics/matrix coverage. + +- [ ] **Step 4: Complete Excel snapshot and matrix-visibility wording** + +Add duplicate `(index name, timestamp)` rejection, canonical/reverse FX lookup, and the `1e-10` reciprocal rule. State explicitly: + +- staged `XCCYCALIBRATIONRESULT.GET` exposes fit vectors/scalars only and neither matrix; +- joint settings can request both computations; +- `JOINTXCCYCALIBRATIONRESULT.GET("jacobian")` exposes the forward matrix and ranges; +- no worksheet selector exposes the retained joint effective inverse. + +- [ ] **Step 5: Verify identifiers against source and commit** + +```bash +rg -n "enum class Value_|FIXED|RESETTABLE|MARK_TO_MARKET" dal-cpp/dal/auto/MG_XccyNotionalMode_enum.hpp +rg -n "jacobian_at_solution|eff_jacobian_inverse|CalibrateXccyMarket|CalibrateJointXccyMarket" dal-python/src/bindings/curve.cpp +rg -n "acceptedAttributes|effJacobianInverse|computeEffJacobianInverse|jacobian" dal-excel/src/__xccycalibration.cpp +python3 .github/scripts/check_docs.py +git diff --check +git add docs/public-api.md dal-python/README.md dal-excel/README.md +git commit -m "docs: clarify XCCY API availability" +``` + +### Task 6: Record the 33-document audit and run the final documentation gate + +**Files:** +- Create: `.codex/artifacts/reviews/repository-documentation-reconciliation.md` + +**Interfaces:** +- Consumes: baseline audit reports `/tmp/dal-doc-audit-quant.md`, `/tmp/dal-doc-audit-curves.md`, `/tmp/dal-doc-audit-platform.md` when available, all Task 1-5 commits, the controlling spec, and the prior audit. +- Produces: final evidence-backed disposition matrix and branch-level verification record. + +- [ ] **Step 1: Create the audit artifact with exactly 33 baseline dispositions** + +Use this structure: + +```markdown +# DAL Repository Documentation Reconciliation + +- Baseline: `1589089b...` +- Published baseline set: 33 files from `.github/scripts/check_docs.py` + +## Executive Summary +The baseline set contains 14 changed documents, five historical artifacts moved to +`.codex/artifacts/`, and 14 documents verified current without edits. + +## Document Dispositions +| Baseline document | Disposition | Source evidence | Result | +|-------------------|-------------|-----------------|--------| +Use the exact 33-row mapping below and fill each evidence/result cell from the three +source-backed audit reports and the Task 1-5 diffs. + +## Prior Audit Reconciliation +| 2026-07-10 finding | Current status | Evidence | +|--------------------|----------------|----------| +Group and record the prior numerical, runtime, packaging, web, build, API, and +documentation findings. Mark fixed behavior as resolved with current source/test +evidence; retain the documented MRG32 normal-path seeking limitation as current. + +## Verification +Record the documentation checker count, diff/scope checks, three C++ example runs, +Python example run, 24-row benchmark parse, and reviewer verdict. +``` + +The exact baseline disposition mapping is: + +| Baseline document | Disposition | +|-------------------|-------------| +| `CHANGELOG.md` | `changed` | +| `CONTRIBUTING.md` | `verified-current` | +| `README.md` | `changed` | +| `dal-cpp/README.md` | `verified-current` | +| `dal-excel/README.md` | `changed` | +| `dal-public/README.md` | `verified-current` | +| `dal-python/README.md` | `changed` | +| `dal-web/README.md` | `verified-current` | +| `docs/README.md` | `changed` | +| `docs/architecture.md` | `changed` | +| `docs/experimental/aad-analytic-jacobian-curve-calibration.md` | `verified-current` | +| `docs/experimental/replicate-ptirds-single-currency-curve.md` | `changed` | +| `docs/installation.md` | `verified-current` | +| `docs/methodology/aad.md` | `changed` | +| `docs/methodology/black_scholes.md` | `verified-current` | +| `docs/methodology/dupire.md` | `verified-current` | +| `docs/methodology/interpolation.md` | `changed` | +| `docs/methodology/log_discount_curve.md` | `verified-current` | +| `docs/methodology/matrix.md` | `changed` | +| `docs/methodology/pde.md` | `verified-current` | +| `docs/methodology/quadrature.md` | `verified-current` | +| `docs/methodology/random.md` | `changed` | +| `docs/methodology/script_engine.md` | `verified-current` | +| `docs/methodology/underdetermined_search.md` | `verified-current` | +| `docs/methodology/xccy_calibration.md` | `changed` | +| `docs/methodology/yield_curve.md` | `verified-current` | +| `docs/methodology/yield_curve_jacobian.md` | `changed` | +| `docs/public-api.md` | `changed` | +| `docs/superpowers/plans/2026-07-12-unified-yield-curve-interpolation-aad.md` | `moved-to-.codex` | +| `docs/superpowers/plans/2026-07-12-ycinstrument-pricing-performance.md` | `moved-to-.codex` | +| `docs/superpowers/plans/2026-07-12-zero-rate-parameterization.md` | `moved-to-.codex` | +| `docs/superpowers/specs/2026-07-12-ycinstrument-pricing-performance-design.md` | `moved-to-.codex` | +| `docs/superpowers/specs/2026-07-12-zero-rate-parameterization-design.md` | `moved-to-.codex` | + +Use only `changed`, `moved-to-.codex`, and `verified-current` dispositions. Cite symbols/files, not source line numbers, in the durable audit. + +- [ ] **Step 2: Prove matrix completeness against the baseline file list** + +```bash +python3 - <<'PY' +from pathlib import Path +import importlib.util +import re +audit = Path('.codex/artifacts/reviews/repository-documentation-reconciliation.md').read_text() +rows = [line for line in audit.splitlines() if line.startswith('| `') and ('changed' in line or 'moved-to-.codex' in line or 'verified-current' in line)] +assert len(rows) == 33, len(rows) +paths = [re.match(r"\| `([^`]+)`", row).group(1) for row in rows] +assert len(paths) == len(set(paths)) == 33 +spec = importlib.util.spec_from_file_location('check_docs', '.github/scripts/check_docs.py') +module = importlib.util.module_from_spec(spec) +spec.loader.exec_module(module) +current = {module.relative(path) for path in module.DOCS} +moved = { + 'docs/superpowers/plans/2026-07-12-unified-yield-curve-interpolation-aad.md', + 'docs/superpowers/plans/2026-07-12-ycinstrument-pricing-performance.md', + 'docs/superpowers/plans/2026-07-12-zero-rate-parameterization.md', + 'docs/superpowers/specs/2026-07-12-ycinstrument-pricing-performance-design.md', + 'docs/superpowers/specs/2026-07-12-zero-rate-parameterization-design.md', +} +assert set(paths) == current | moved, (set(paths) ^ (current | moved)) +PY +``` + +Expected: exit 0 with 33 unique rows. + +- [ ] **Step 3: Run final scope and documentation checks** + +```bash +python3 .github/scripts/check_docs.py +git diff --check 1589089b..HEAD +git diff --name-only 1589089b..HEAD | while IFS= read -r path; do case "$path" in *.md) ;; *) printf '%s\n' "$path";; esac; done +git diff --name-only 1589089b..HEAD -- CLAUDE.md .claude AGENTS.md dal-cpp/dal/auto dal-excel/auto +``` + +Expected: docs checker passes for 28 current-state Markdown files; all three diff/scope commands emit no unexpected paths. + +- [ ] **Step 4: Commit the audit record** + +```bash +git add .codex/artifacts/reviews/repository-documentation-reconciliation.md +git commit -m "docs: record repository documentation audit" +``` + +- [ ] **Step 5: Prepare final review evidence** + +```bash +git log --oneline 1589089b..HEAD +git diff --stat 1589089b..HEAD +git status --short +``` + +Expected: six scoped documentation commits after the spec commit, a clean worktree, and no production/generated changes. From ca147d8ffc138b97ad5bae7b60fa421d88ae6fc6 Mon Sep 17 00:00:00 2001 From: wegamekinglc Date: Wed, 15 Jul 2026 14:00:43 +0800 Subject: [PATCH 03/19] docs: move historical plans out of published docs --- .../plans/2026-07-12-unified-yield-curve-interpolation-aad.md | 0 .../plans/2026-07-12-ycinstrument-pricing-performance.md | 0 .../artifacts}/plans/2026-07-12-zero-rate-parameterization.md | 0 .../specs/2026-07-12-ycinstrument-pricing-performance-design.md | 0 .../specs/2026-07-12-zero-rate-parameterization-design.md | 0 5 files changed, 0 insertions(+), 0 deletions(-) rename {docs/superpowers => .codex/artifacts}/plans/2026-07-12-unified-yield-curve-interpolation-aad.md (100%) rename {docs/superpowers => .codex/artifacts}/plans/2026-07-12-ycinstrument-pricing-performance.md (100%) rename {docs/superpowers => .codex/artifacts}/plans/2026-07-12-zero-rate-parameterization.md (100%) rename {docs/superpowers => .codex/artifacts}/specs/2026-07-12-ycinstrument-pricing-performance-design.md (100%) rename {docs/superpowers => .codex/artifacts}/specs/2026-07-12-zero-rate-parameterization-design.md (100%) diff --git a/docs/superpowers/plans/2026-07-12-unified-yield-curve-interpolation-aad.md b/.codex/artifacts/plans/2026-07-12-unified-yield-curve-interpolation-aad.md similarity index 100% rename from docs/superpowers/plans/2026-07-12-unified-yield-curve-interpolation-aad.md rename to .codex/artifacts/plans/2026-07-12-unified-yield-curve-interpolation-aad.md diff --git a/docs/superpowers/plans/2026-07-12-ycinstrument-pricing-performance.md b/.codex/artifacts/plans/2026-07-12-ycinstrument-pricing-performance.md similarity index 100% rename from docs/superpowers/plans/2026-07-12-ycinstrument-pricing-performance.md rename to .codex/artifacts/plans/2026-07-12-ycinstrument-pricing-performance.md diff --git a/docs/superpowers/plans/2026-07-12-zero-rate-parameterization.md b/.codex/artifacts/plans/2026-07-12-zero-rate-parameterization.md similarity index 100% rename from docs/superpowers/plans/2026-07-12-zero-rate-parameterization.md rename to .codex/artifacts/plans/2026-07-12-zero-rate-parameterization.md diff --git a/docs/superpowers/specs/2026-07-12-ycinstrument-pricing-performance-design.md b/.codex/artifacts/specs/2026-07-12-ycinstrument-pricing-performance-design.md similarity index 100% rename from docs/superpowers/specs/2026-07-12-ycinstrument-pricing-performance-design.md rename to .codex/artifacts/specs/2026-07-12-ycinstrument-pricing-performance-design.md diff --git a/docs/superpowers/specs/2026-07-12-zero-rate-parameterization-design.md b/.codex/artifacts/specs/2026-07-12-zero-rate-parameterization-design.md similarity index 100% rename from docs/superpowers/specs/2026-07-12-zero-rate-parameterization-design.md rename to .codex/artifacts/specs/2026-07-12-zero-rate-parameterization-design.md From 1f5aa58cf56f89d9569a0f3b1bbfb13d673331de Mon Sep 17 00:00:00 2001 From: wegamekinglc Date: Wed, 15 Jul 2026 14:11:16 +0800 Subject: [PATCH 04/19] docs: refresh architecture and navigation --- CHANGELOG.md | 28 +++++++++++++++------------- README.md | 10 +++++++++- docs/README.md | 13 +++++++------ docs/architecture.md | 30 ++++++++++++++++++++---------- 4 files changed, 51 insertions(+), 30 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index c59118844..bb4d218d3 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -27,31 +27,33 @@ here as the baseline rather than dated releases: calibrated to market instruments. See `docs/methodology/yield_curve.md`. - **Underdetermined Search** — constrained least-change solver for over-parameterised nonlinear calibration. See `docs/methodology/underdetermined_search.md`. -- **Cross-Currency Calibration** — XCCY basis-curve fitting across two currencies. See - `docs/methodology/xccy_calibration.md`. +- **Cross-Currency Pricing and Calibration** — fixed, resettable, and mark-to-market + swap pricing with immutable timestamped rate/FX fixing snapshots, staged basis + fitting, simultaneous domestic/foreign/basis calibration, and named joint + parameter/residual ranges. See `docs/methodology/xccy_calibration.md`. - **Interpolation** — linear, log-linear, cubic-spline, and mixed 1D interpolators plus bilinear 2D interpolation. See `docs/methodology/interpolation.md`. - **Log-Discount Curve** — node log-discount-factor parameterisation with `LogDfScheme_` interpolation schemes and scalar-generic passive/AAD evaluation. See `docs/methodology/log_discount_curve.md`. -- **Yield-Curve Jacobian and Inverse-Jacobian Risk** — AAD forward Jacobian and the - inverse-Jacobian IR-risk transform, including the `effJacobianInverse_` unit convention. - See `docs/methodology/yield_curve_jacobian.md`. +- **Yield-Curve Jacobian and Inverse-Jacobian Risk** — AAD forward Jacobians for every + implemented curve representation subject to the normal eligibility gates, plus the + inverse-Jacobian IR-risk transform and its `effJacobianInverse_` unit convention. See + `docs/methodology/yield_curve_jacobian.md`. - **Script Engine** — events-table to AST pipeline, visitor passes (domain analysis, constant-condition folding), and the fuzzy evaluator for pathwise AAD through discontinuous payoffs. See `docs/methodology/script_engine.md`. -- **Analytic Jacobian for curve calibration (CurveJacobianMode flag)** — optional analytic - Jacobian mode for yield-curve calibration. See - `docs/methodology/yield_curve_jacobian.md`. ## 2026-07 - `curve`: Added reset-aware cross-currency pricing and simultaneous domestic, - foreign, and basis calibration. `XccyNotionalMode_` now defines `FIXED`, - `RESETTABLE`, and `MARK_TO_MARKET`; explicit rate/FX fixing identities and one - immutable timestamped snapshot support already-started swaps; and the joint - solver exposes named parameter/residual ranges plus analytic or bumped - Jacobians through public C++, Python, and Excel. **Breaking:** the two + foreign, and basis calibration. Fixed, resettable, and mark-to-market notional + modes replace the prior boolean configuration; explicit rate/FX fixing identities + and one immutable timestamped snapshot support already-started swaps; and the + joint solver exposes named parameter/residual ranges plus analytic or bumped + Jacobians. Public C++ and joint Python expose both retained joint matrices. Excel + exposes the joint forward Jacobian and ranges, but has no worksheet getter for the + effective inverse. **Breaking:** the two `CrossCurrencyConvention_` booleans `resettableNotional_` and `markToMarketNotional_` are replaced by the enum in `CrossCurrencySwapConfig_`. The legacy fixed-notional convenience constructor diff --git a/README.md b/README.md index 29587dbab..60f62003f 100644 --- a/README.md +++ b/README.md @@ -5,7 +5,7 @@ [![Codacy Grade](https://app.codacy.com/project/badge/Grade/9c84afd2bb534c6c87584e5d6e4cc420)](https://app.codacy.com/app/wegamekinglc/Derivatives-Algorithms-Lib) [![Coverage Status](https://coveralls.io/repos/github/wegamekinglc/Derivatives-Algorithms-Lib/badge.svg?branch=master)](https://coveralls.io/github/wegamekinglc/Derivatives-Algorithms-Lib?branch=master) -A C++17 quantitative finance library with built-in Automatic Adjoint Differentiation (AAD). Features include yield curve construction, Monte Carlo simulation, finite difference PDE solvers, a scripting engine for exotic payoffs with tree-walk and compiled evaluators, and parallel model evaluation. +A C++17 quantitative finance library with built-in Automatic Adjoint Differentiation (AAD). Features include yield curve construction, cross-currency pricing and calibration, Monte Carlo simulation, finite difference PDE solvers, a scripting engine for exotic payoffs with tree-walk and compiled evaluators, and parallel model evaluation. ## Quick Start @@ -93,6 +93,13 @@ d_vol : 58.7140 More examples: [Python](dal-python/examples/), [Excel](dal-excel/examples/), [C++](dal-cpp/examples/). The C++ Monte Carlo script examples show both tree-walk and compiled evaluator output where applicable. +Cross-currency examples: + +- [reset-aware pricing](dal-cpp/examples/xccy_reset_pricing/) +- [staged basis calibration](dal-cpp/examples/xccy_curve_calibration/) +- [joint domestic/foreign/basis calibration](dal-cpp/examples/xccy_mtm_calibration/) +- [Python joint calibration](dal-python/examples/007.xccy_joint_calibration.py) + ### Script Engine Modes Monte Carlo script valuation defaults to the tree-walk evaluator (`compiled=false`). @@ -152,6 +159,7 @@ Methodology notes (see the index above for the full list): - [AAD](docs/methodology/aad.md) — Automatic adjoint differentiation: expression templates, tape, propagation - [Yield Curve](docs/methodology/yield_curve.md) and [Yield-Curve Jacobian](docs/methodology/yield_curve_jacobian.md) — discount curves, calibration, Jacobian / inverse-Jacobian risk +- [Cross-Currency Pricing and Calibration](docs/methodology/xccy_calibration.md) — fixed, resettable, and MTM swaps; immutable fixing snapshots; staged basis and simultaneous domestic/foreign/basis calibration - [Interpolation](docs/methodology/interpolation.md) — linear, log-linear, cubic interpolators - [PDE](docs/methodology/pde.md) — finite-difference meshers and coordinate maps - [Script Engine](docs/methodology/script_engine.md) — expression scripting, fuzzy AAD evaluation, and compiled evaluator parity diff --git a/docs/README.md b/docs/README.md index 77310af99..34d45a7d1 100644 --- a/docs/README.md +++ b/docs/README.md @@ -34,7 +34,7 @@ additions — is recorded in the repo-root [CHANGELOG.md](../CHANGELOG.md). - Web UI installation and startup - Verification and troubleshooting -### Methodology (`methodology/`) +## Methodology (`methodology/`) Deep dives into the quantitative methods and algorithms implemented in DAL: @@ -59,11 +59,11 @@ Deep dives into the quantitative methods and algorithms implemented in DAL: - Solver controls structure and Broyden update regime - Application to yield curve calibration via smoothness penalties -- **[xccy_calibration.md](methodology/xccy_calibration.md)** — Cross-Currency Calibration +- **[xccy_calibration.md](methodology/xccy_calibration.md)** — Cross-Currency Pricing and Calibration - Fixed, resettable-notional, and mark-to-market cross-currency pricing - - Timestamped immutable rate and FX fixing snapshots + - Immutable operation-level snapshots of timestamped rate and FX fixings - Staged basis calibration and simultaneous domestic/foreign/basis calibration - - Joint parameter/residual ranges and analytic or bumped Jacobians + - Joint parameter/residual matrix ranges, analytic or bumped Jacobians, and effective-inverse scaling - **[interpolation.md](methodology/interpolation.md)** — Interpolation - Linear, log-linear, cubic-spline, and mixed one-dimensional interpolators @@ -83,7 +83,7 @@ Deep dives into the quantitative methods and algorithms implemented in DAL: - `LogDfScheme_` interpolation schemes (`LOG_LINEAR`, `LOG_CUBIC_NATURAL`, `MIXED`) - Thomas algorithm for the natural-cubic system, basis weights, and `fppCoef_` matrix - Serialization version design (v1 without scheme, v2 with named scheme) - - Why `LOG_DISCOUNT` is the parameterization that supports the analytic Jacobian + - Persistent log-discount coordinates, interpolation/extrapolation, basis weights, and participation in the shared analytic-Jacobian curve factory - **[pde.md](methodology/pde.md)** — PDE Framework - Coordinate maps, including identity, sinh, and endpoint-exact concentrating maps @@ -94,6 +94,7 @@ Deep dives into the quantitative methods and algorithms implemented in DAL: - **[yield_curve_jacobian.md](methodology/yield_curve_jacobian.md)** — Yield-Curve Jacobian and Inverse-Jacobian Risk - Forward residual Jacobian via AAD reverse sweep vs finite-difference bump + - Staged and joint matrix dimensions, including named joint parameter/residual ranges - Inverse-Jacobian IR-risk transform `r = gᵀ · effJacobianInverse_ / tolerance_` - Why `effJacobianInverse_` carries an extra `tolerance_` factor (solver residual scaling) @@ -129,7 +130,7 @@ Deep dives into the quantitative methods and algorithms implemented in DAL: - Sobol inverse-CDF policy table and clone-equivalent state/flag preservation - Path seeking via direct state reconstruction (`SobolSet_::SkipTo`, MRG32k32a matrix jump) -### Experimental (`experimental/`) +## Experimental (`experimental/`) Reference studies and capability explorations that are not normative methodology: diff --git a/docs/architecture.md b/docs/architecture.md index 135cdba05..ffca1e1f0 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -61,12 +61,16 @@ Callers embedding the library need to preserve that distinction. | Evaluation date | Process-wide store value. Reads use the global-store mutex; mutation, scoped overrides, and native valuation also participate in the re-entrant valuation/mutation barrier. | | Accounting date | Process-wide store value. Reads and writes use the global-store mutex but do not participate in the valuation/mutation barrier. | | Fixings | Process-wide store. Reads use the global-store mutex, but the final `StoreFixings` write does not. Fixings mutation is outside the evaluation-date synchronization contract, so callers must externally serialize it with other fixings access. | +| Market-fixing snapshot | Immutable operation-level value retained by reset-aware pricing/calibration results. An explicit snapshot is authoritative; when omitted, staged/joint XCCY calibration gathers required historical requests and copies the process-wide store once before solving. | | Calendar/currency/index registration | Initialized once per process by `InitGlobalData` / Python module initialization. | | Thread pool | Process-wide singleton, inactive until explicit start or first task. Lifecycle and queue transitions are synchronized. | | AAD tape | One tape per executing thread (`thread_local`). Calibration guards and simulation batches rewind their thread's tape before reuse. | | Script compiled stacks | Operand stacks belong to each `EvalState_`; recursive compiled evaluation reuses that state, and AAD stack values cannot outlive the task/tape that owns them. | | Excel object repository | Host/environment-owned repository of storable handles used between worksheet calls. | +`MarketFixingSnapshot_` therefore gives one pricing or calibration operation a stable +fixing view rather than a live view of the process-wide fixing store. + ### Evaluation-date synchronization Evaluation-date access uses two native synchronization mechanisms with one @@ -159,21 +163,27 @@ for each worker's path batch. ```text instrument/convention builders - -> CurveCalibrationSpec_ or public builder - -> validation and knot construction - -> residual map: model rate - market rate - -> Underdetermined::Find or Underdetermined::Approximate - -> eligible AAD analytic Jacobian - -> finite-difference fallback - -> calibrated curve and diagnostics + -> validated spec, curve layout, and model-rate residuals + -> single/staged curves: CalibrateCurve / CalibrateMultiCurve + -> staged XCCY basis: CalibrateCrossCurrencyMarket + -> supplied domestic/foreign blocks + basis parameters + one fixing snapshot + -> joint XCCY: CalibrateJointXccyMarket + -> domestic declarations + foreign declarations + basis declaration + -> one fixing snapshot + named parameter/residual ranges + -> exact or approximate underdetermined solve + -> eligible AAD analytic Jacobian or explicit bumped mode + -> solved curves, diagnostics, and optional matrices ``` Exact calibration uses the weighted least-change solver and can expose the forward Jacobian plus effective inverse Jacobian. Approximate calibration trades fit against distance from the reference parameters. Staged multi-curve calibration -solves stages in order; joint calibration stacks declarations into one residual -system. See [yield-curve construction](methodology/yield_curve.md) and -[yield-curve Jacobian](methodology/yield_curve_jacobian.md). +solves stages in order; staged XCCY supplies the domestic and foreign curve blocks +while fitting a basis curve, and joint XCCY stacks domestic, foreign, and basis +declarations into one residual system. See +[yield-curve construction](methodology/yield_curve.md), +[yield-curve Jacobian](methodology/yield_curve_jacobian.md), and +[cross-currency pricing and calibration](methodology/xccy_calibration.md). ## Generated Code From 4d0e60b742f1c20f7eaf719fc985fb2adec21ea6 Mon Sep 17 00:00:00 2001 From: wegamekinglc Date: Wed, 15 Jul 2026 14:22:11 +0800 Subject: [PATCH 05/19] docs: correct numerical methodology contracts --- .../replicate-ptirds-single-currency-curve.md | 25 ++++++----- docs/methodology/aad.md | 6 ++- docs/methodology/interpolation.md | 17 ++++---- docs/methodology/matrix.md | 36 ++++++++-------- docs/methodology/random.md | 43 +++++++++++-------- 5 files changed, 72 insertions(+), 55 deletions(-) diff --git a/docs/experimental/replicate-ptirds-single-currency-curve.md b/docs/experimental/replicate-ptirds-single-currency-curve.md index de9c959c3..b09dd5b45 100644 --- a/docs/experimental/replicate-ptirds-single-currency-curve.md +++ b/docs/experimental/replicate-ptirds-single-currency-curve.md @@ -24,10 +24,9 @@ This is an excellent DAL exercise because it stresses several subsystems at once - **Calibration** — a single global least-squares/least-change solve over all curve nodes simultaneously, exactly the regime DAL's underdetermined search targets. - **Date machinery** — IMM-style node dates, single-business-day and stub swaps, - Act/365F, an all-days (no-holiday) calendar, annual frequency, zero payment lag. -- **AAD** — DAL's differentiator supplies an analytic Jacobian for the solver on - eligible `LOG_DISCOUNT` specs, which is where DAL does *better* than the reference's - bumped/auto-diff solve. + Act/365F, an external every-day calendar reproduced with explicit-date `Unadjusted` + conventions, annual frequency, and zero payment lag. +- **AAD** — eligible `LOG_DISCOUNT` calibration uses an AAD-derived analytic Jacobian. ## 2. The Target (numerical acceptance criteria) @@ -235,9 +234,12 @@ observed max `|err|` of ~`5.2e-7` (`log_linear`), ~`4.6e-7` (`log_cubic`), and - **Act/365F** is a first-class `DayBasis_` alternative (`ACT_365F`), in `dal-cpp/dal/time/daybasis.hpp`. -- **All-days / no-holiday calendar:** `Holidays::None()` exists and is used by curve - instruments today (`dal-cpp/dal/time/holidays.hpp`, - `dal-cpp/dal/curve/ycinstrument.cpp`). This matches calendar = "all". +- **Every-day target calendar:** the external target treats every day as a business + day. DAL reproduces the supplied explicit dates with `Holidays::None()` plus + `Unadjusted` accrual, payment, and index conventions. `Holidays::None()` removes + named holidays, but weekends remain non-business days unless date adjustment is + bypassed (`dal-cpp/dal/time/holidays.hpp`, + `dal-cpp/dal/curve/ycinstrument.cpp`). - **Annual frequency:** `PeriodLength_("12M")`; swap leg conventions accept it (`dal-cpp/dal/curve/ycinstrument.cpp`). - **Payment lag:** `RateLegConvention_::paymentLag_` flows through @@ -298,7 +300,7 @@ implementation; all required items are available or have an explicit DAL analogu | IMM / stub swaps via explicit dates | yes | `Swap_(tradeDate, start, maturity, ...)`, `dal-cpp/dal/curve/ycinstrument.hpp` | explicit effective/termination dates per leg | | 1-business-day swap | yes | degenerate single-period `Swap_`, `dal-cpp/dal/curve/ycinstrument.cpp` | 1-day span; annuity > 0 path | | Act/365F day count | yes | `ACT_365F`, `dal-cpp/dal/time/daybasis.hpp` | reuse | -| All-days / no-holiday calendar | yes | `Holidays::None()`, `dal-cpp/dal/time/holidays.hpp` | matches calendar = "all" | +| Every-day target calendar | yes | `Holidays::None()` plus `Unadjusted`, `dal-cpp/tests/curve/test_ptirds_curve.cpp` | preserves the supplied dates; `Holidays::None()` alone still excludes weekends | | Annual frequency, payment lag 0 | yes | `RateLegConvention_`, `dal-cpp/dal/curve/ycinstrument.cpp` | `PeriodLength_("12M")`, `paymentLag_ = 0` | | Global simultaneous solve over nodes | yes | `Underdetermined::Find`, `dal-cpp/dal/curve/calibration.cpp` | one global solve, not a sequential bootstrap | | Levenberg-Marquardt least-squares | analog | underdetermined least-change EXACT/APPROXIMATE, `dal-cpp/dal/math/optimization/underdetermined.hpp` | equivalent for the square 13×13 case | @@ -340,8 +342,11 @@ numerical validation lives in the core C++ test suite. ### 4. Instrument construction for the PTIRDS set - The 13 swaps are built via the explicit-date `Swap_` constructor (`dal-cpp/dal/curve/ycinstrument.hpp`): annual fixed/float legs, Act/365F, - `Holidays::None()`, payment lag 0, explicit IMM stub effective/termination dates, - and the degenerate 1-business-day swap. The stub day-count context + `Holidays::None()`, `Unadjusted` accrual/payment/index conventions, payment lag 0, + explicit IMM stub effective/termination dates, and the degenerate 1-business-day + swap. The external target uses an every-day calendar; the explicit `Unadjusted` + conventions are required because `Holidays::None()` removes named holidays but + still treats weekends as non-business days. The stub day-count context (`SchedulePeriod_::dayCountContext_`, a `Handle_` built by `BuildSinglePeriodSchedule` in `dal-cpp/dal/curve/calibration_internal.hpp`) reproduces the reference accruals. diff --git a/docs/methodology/aad.md b/docs/methodology/aad.md index 7fd2532a7..9745ca767 100644 --- a/docs/methodology/aad.md +++ b/docs/methodology/aad.md @@ -142,15 +142,17 @@ Seeding the $j$-th output adjoint to $1$ (others $0$) and propagating recovers t $j$-th row of the Jacobian; doing all $m$ together in one sweep recovers the whole Jacobian at the cost of one reverse pass with vector arithmetic. -## Memory: Checkpointing via Mark / Rewind +## Memory: Checkpointing via Mark / RewindToMark The tape grows with the number of operations, so long simulations would exhaust memory if every step were kept. The algorithm uses a **checkpoint** discipline: - A **mark** records a position on the tape. -- **Rewind** discards everything recorded after the mark, reusing that memory, +- **`RewindToMark`** discards everything recorded after the mark, reusing that memory, without disturbing the adjoints already accumulated before the mark. +`RewindToMark(tape)` discards nodes recorded after the current mark and is used after each Monte Carlo path. `Rewind(tape)` resets the tape to the beginning of its recording and is used before a fresh simulation or calibration recording. + This lets a repeated computation (e.g. one Monte Carlo path) record, propagate, and then rewind, so the tape size is bounded by the work of a *single* repetition rather than the whole simulation. Propagation can therefore be partitioned into diff --git a/docs/methodology/interpolation.md b/docs/methodology/interpolation.md index 92920d7ec..7f6f6f49d 100644 --- a/docs/methodology/interpolation.md +++ b/docs/methodology/interpolation.md @@ -63,8 +63,9 @@ yield-curve layer; it prevents passive and AAD paths from maintaining separate f ## Linear -Piecewise-linear interpolation between knots — the kernel above. It is exact at knots, -continuous, and first-order accurate between them. `IsInBounds` returns true on +Piecewise-linear interpolation between knots — the degree-one, piecewise-affine kernel +above. It is exact at knots, continuous, and has $O(h^2)$ interpolation error for a +sufficiently smooth scalar function. `IsInBounds` returns true on $[x_1, x_N]$; outside that range the kernel clamps to the nearest endpoint value (flat extrapolation via the `LowerBound` edge cases). @@ -97,7 +98,7 @@ on the *Numerical Recipes* `spline`/`splint` pair). Evaluation between knots use Hermite form $$ -f(x) = a\,f_i + b\,f_{i+1} - \tfrac{h^2}{6}\Big((1+a)a\,f''_i + (1+b)b\,f''_{i+1}\Big), +f(x)=a f_i+b f_{i+1}-\frac{a b h^2}{6}\left[(1+a)f_i''+(1+b)f_{i+1}''\right] $$ with $h = x_{i+1}-x_i$, $b = (x-x_i)/h$, $a = 1-b$. @@ -105,11 +106,11 @@ with $h = x_{i+1}-x_i$, $b = (x-x_i)/h$, $a = 1-b$. The two end conditions are supplied as a `Boundary_(order, value)` pair, where `order` selects which derivative is pinned at the boundary: -| `order_` | Boundary condition | -|----------|-------------------------------------------------| -| 1 | first derivative $f'(x_1) = $ `value_` | -| 2 | second derivative $f''(x_1) = $ `value_` | -| 3 | third derivative fixed (not-a-knot family) | +| `order_` | Boundary condition | +|----------|---------------------------------------------------------| +| 1 | first derivative $f'(x_1) = $ `value_` | +| 2 | second derivative $f''(x_1) = $ `value_` | +| 3 | endpoint segment's third derivative pinned to `value_` | `Boundary_(2, 0.0)` on both ends gives the classic natural spline (zero end curvature). diff --git a/docs/methodology/matrix.md b/docs/methodology/matrix.md index f3b502eb5..504d144df 100644 --- a/docs/methodology/matrix.md +++ b/docs/methodology/matrix.md @@ -76,17 +76,18 @@ recurrence $$ \beta_0 = d_0, \qquad \beta_i = d_i - \frac{a_{i-1}\,c_{i-1}}{\beta_{i-1}}, \qquad -\ell_i = \frac{c_i}{\beta_i} +\ell_i = \frac{c_{i-1}}{\beta_{i-1}} $$ with $d$ the diagonal, $a$ the super-diagonal, and $c$ the sub-diagonal. The implementation in `TridagBetaInverse` (`dal-cpp/dal/math/matrix/banded.cpp`) stores the inverses -$1/\beta_i$ rather than the $\ell_i$ directly, so that the subsequent forward and backward -substitutions reduce to +$1/\beta_i$ rather than the $\ell_i$ directly. The forward substitution starts with +$x_0=b_0/\beta_0$; subsequent forward and backward steps are $$ -x_0 = \frac{b_0}{\beta_0}, \quad x_i = \frac{b_i - a_{i-1}\,x_{i-1}}{\beta_i}, \qquad -x_{i-1} \mathrel{-}= \frac{c_{i-1}}{\beta_{i-1}}\,x_i . +x_i=\frac{b_i-c_{i-1}x_{i-1}}{\beta_i}, +\qquad +x_{i-1}\leftarrow x_{i-1}-\frac{a_{i-1}}{\beta_{i-1}}x_i $$ This is the Thomas algorithm. It is $O(n)$ in time and $O(n)$ in memory, and it requires no @@ -109,20 +110,21 @@ L_{i,j} = \frac{1}{L_{j,j}}\!\left( A_{i,j} - \sum_{k` | Knuth-style lagged additive (IRN55) with shuffling, modulus $2^{30}$ | No-op (re-seed via `Branch`/`Clone` for substreams) | -| `MRG32` | `MRG32k32a_` | L'Ecuyer combined multiple recursive (MRG32k32a), periods $\approx 2^{191}$ | Matrix exponentiation of the recurrence | +| `IRN` | `ShuffledIRN_<55,31,128>` | Knuth-style lagged additive (IRN55) with shuffling, modulus $2^{30}$ | No-op; `Clone` restarts from the original seed and `Branch` creates another seeded generator | +| `MRG32` | `MRG32k32a_` | L'Ecuyer combined multiple recursive (MRG32k32a), periods $\approx 2^{191}$ | Matrix jump with antithetic-`FillUniform` accounting; not replay-equivalent for normal paths | `PseudoRandom_` adds antithetic variates on top of the underlying engine: `FillUniform` alternates between drawing $u_i$ and emitting the antithetic $1 - u_i$ from its cache, halving the number of engine calls when antithetic -sampling is in use. The `precise_` flag selects a higher-accuracy +sampling is in use. `FillNormal`, however, consumes a fresh `NextUniform()` per +component and bypasses that cache. The `precise_` flag selects a higher-accuracy inverse-normal-CDF inside `FillNormal`. ### `IRN` — Shuffled Lagged Additive @@ -255,17 +257,20 @@ replacing it with the state before output. Shuffling breaks the short-range correlations that plain lagged generators exhibit. The output is the shuffled value mapped to $(0,1)$ with the $2(2u+1)/2^{31}$ form that keeps both endpoints excluded. `SkipTo` is a no-op for this engine. +`Clone` reconstructs the generator from its original seed instead of preserving +the advanced state; `Branch` creates another seeded generator. ### `MRG32` — Combined Multiple Recursive `MRG32k32a_` is L'Ecuyer's MRG32k32a combined multiple-recursive generator: two order-3 recurrences with moduli $m_1 = 4294967087$ and $m_2 = 4294944443$ -are combined as $u = ((x - y) \bmod m_1) / (m_1 + 1)$. `SkipTo` advances the -recurrence by $n \cdot \text{NDim}$ points using binary exponentiation of the -$3 \times 3$ companion matrices of each recurrence, reduced modulo $m_1$ and -$m_2$, then applies the resulting matrices to the initial state. This is the -same path-seeking principle as the Sobol `Seek`: jump to the target state -directly rather than iterating through every intermediate point. +are combined as $u = ((x - y) \bmod m_1) / (m_1 + 1)$. `SkipTo` uses binary +exponentiation of the $3 \times 3$ companion matrices to jump from the initial +state. Its skip count halves $n \cdot \text{NDim}$ to match the generated/cached +antithetic pattern of `FillUniform`. Because `FillNormal` instead consumes a fresh +uniform for every component, MRG32 seeking is not replay-equivalent for normal-path +substreams and must not be used as such. Sobol remains the verified normal-path +seeking surface. ## Selection Guidance @@ -277,9 +282,11 @@ directly rather than iterating through every intermediate point. reduction techniques that rely on statistical independence (antithetic, control variates with estimated coefficients), and for regression-based estimators where low-discrepancy bias is undesirable. Between the two - engines, `MRG32` has the stronger theoretical guarantees and full `SkipTo` - support for substream parallelism; `IRN` is retained as a lightweight - Knuth-style alternative. + engines, `MRG32` has the stronger theoretical recurrence, and its current + `SkipTo` accounting matches antithetic `FillUniform` rather than + replay-equivalent normal-path substreams; `IRN` is retained as a lightweight + Knuth-style alternative with no seek implementation. Use Sobol when direct + normal-path seeking is required. ## Benchmark Coverage @@ -303,6 +310,6 @@ over a 100K-path batch, the dominant Monte Carlo inner loop. - [Script engine](script_engine.md) — the Monte Carlo driver that consumes these generators; the Brownian bridge is most effective when wrapped around a Sobol sequence for path-dependent payoffs. -- [AAD methodology](aad.md) — `SkipTo` and the per-thread tape make pathwise-adjoint - Monte Carlo parallelisable without synchronisation during the forward or reverse - passes. +- [AAD methodology](aad.md) — Sobol's direct `SkipTo` and the per-thread tape make + pathwise-adjoint Monte Carlo parallelisable without synchronisation during the + forward or reverse passes. From 427385092103ef669afc77bbb15694128fc8ec48 Mon Sep 17 00:00:00 2001 From: wegamekinglc Date: Wed, 15 Jul 2026 14:34:06 +0800 Subject: [PATCH 06/19] docs: tighten numerical runtime semantics --- docs/methodology/aad.md | 5 +++- docs/methodology/interpolation.md | 10 +++---- docs/methodology/matrix.md | 7 ++--- docs/methodology/random.md | 44 +++++++++++++++++++++---------- 4 files changed, 43 insertions(+), 23 deletions(-) diff --git a/docs/methodology/aad.md b/docs/methodology/aad.md index 9745ca767..f585019fc 100644 --- a/docs/methodology/aad.md +++ b/docs/methodology/aad.md @@ -151,7 +151,10 @@ memory if every step were kept. The algorithm uses a **checkpoint** discipline: - **`RewindToMark`** discards everything recorded after the mark, reusing that memory, without disturbing the adjoints already accumulated before the mark. -`RewindToMark(tape)` discards nodes recorded after the current mark and is used after each Monte Carlo path. `Rewind(tape)` resets the tape to the beginning of its recording and is used before a fresh simulation or calibration recording. +`RewindToMark(tape)` discards nodes recorded after the current mark and is called +before each Monte Carlo path, so it clears the preceding path's recording between +paths. `Rewind(tape)` resets the tape to the beginning of its recording and is used +before a fresh simulation or calibration recording. This lets a repeated computation (e.g. one Monte Carlo path) record, propagate, and then rewind, so the tape size is bounded by the work of a *single* repetition diff --git a/docs/methodology/interpolation.md b/docs/methodology/interpolation.md index 7f6f6f49d..5cf8772c3 100644 --- a/docs/methodology/interpolation.md +++ b/docs/methodology/interpolation.md @@ -106,11 +106,11 @@ with $h = x_{i+1}-x_i$, $b = (x-x_i)/h$, $a = 1-b$. The two end conditions are supplied as a `Boundary_(order, value)` pair, where `order` selects which derivative is pinned at the boundary: -| `order_` | Boundary condition | -|----------|---------------------------------------------------------| -| 1 | first derivative $f'(x_1) = $ `value_` | -| 2 | second derivative $f''(x_1) = $ `value_` | -| 3 | endpoint segment's third derivative pinned to `value_` | +| `order_` | Boundary condition | +|----------|------------------------------------------------------------------------------------------| +| 1 | `lhs` pins $f'(x_1)$; `rhs` pins $f'(x_N)$ to its respective `value_` | +| 2 | `lhs` pins $f''(x_1)$; `rhs` pins $f''(x_N)$ to its respective `value_` | +| 3 | each endpoint segment's third derivative is pinned to the respective `lhs`/`rhs` `value_` | `Boundary_(2, 0.0)` on both ends gives the classic natural spline (zero end curvature). diff --git a/docs/methodology/matrix.md b/docs/methodology/matrix.md index 504d144df..aac2137b6 100644 --- a/docs/methodology/matrix.md +++ b/docs/methodology/matrix.md @@ -90,9 +90,10 @@ x_i=\frac{b_i-c_{i-1}x_{i-1}}{\beta_i}, x_{i-1}\leftarrow x_{i-1}-\frac{a_{i-1}}{\beta_{i-1}}x_i $$ -This is the Thomas algorithm. It is $O(n)$ in time and $O(n)$ in memory, and it requires no -pivoting: it is valid only when every $\beta_i$ is non-zero, which holds in particular for -**diagonally dominant** and for **symmetric positive-definite** tri-diagonal systems — the +This is the Thomas algorithm. It is $O(n)$ in time and $O(n)$ in memory. Pivoting is not +used: the factorization is valid only when every $\beta_i$ is non-zero, +which holds in particular for **strictly diagonally dominant** and for **symmetric +positive-definite** tri-diagonal systems — the two cases that dominate finite-difference PDE discretisations and natural-spline construction. The `TriDecomp_` factorization wraps the asymmetric case; `TriDecompSymm_` collapses `above_` and `below_` to one vector for the symmetric case, where left and right diff --git a/docs/methodology/random.md b/docs/methodology/random.md index 46423d862..7de43f0de 100644 --- a/docs/methodology/random.md +++ b/docs/methodology/random.md @@ -24,8 +24,8 @@ $$ `NDim()` is the number of variates produced per call (one draw of a multi-dimensional point). `SkipTo(n)` behavior is implementation-specific: Sobol reconstructs its state directly, `ShuffledIRN_::SkipTo` is a no-op, and -MRG32's current accounting is tied to antithetic uniform draws rather than a -general replay-equivalent path contract. +MRG32 replays `FillUniform` only for even path offsets on a fresh generator; its +current accounting is not a general replay-equivalent path contract. `FillUniform` writes variates in $(0,1)$; `FillNormal` writes standard normal variates, obtained either by inverse-CDF inversion of the uniform variates or by a direct transformation, depending on the generator. @@ -238,7 +238,7 @@ enum, whose alternatives are `IRN` and `MRG32`. | `RNGType_` | Implementation | Period / structure | `SkipTo` | |------------|-----------------------|----------------------------------------------|-----------------------------------------------------| | `IRN` | `ShuffledIRN_<55,31,128>` | Knuth-style lagged additive (IRN55) with shuffling, modulus $2^{30}$ | No-op; `Clone` restarts from the original seed and `Branch` creates another seeded generator | -| `MRG32` | `MRG32k32a_` | L'Ecuyer combined multiple recursive (MRG32k32a), periods $\approx 2^{191}$ | Matrix jump with antithetic-`FillUniform` accounting; not replay-equivalent for normal paths | +| `MRG32` | `MRG32k32a_` | L'Ecuyer combined multiple recursive (MRG32k32a), periods $\approx 2^{191}$ | Matrix jump; fresh-generator `FillUniform` replay at even path offsets only; no normal-path replay | `PseudoRandom_` adds antithetic variates on top of the underlying engine: `FillUniform` alternates between drawing $u_i$ and emitting the antithetic @@ -255,7 +255,7 @@ $\text{irn}_i \leftarrow (\text{irn}_i + \text{irn}_{i+31}) \bmod 2^{30}$, with a length-128 shuffle table that returns the entry indexed by the new state, replacing it with the state before output. Shuffling breaks the short-range correlations that plain lagged generators exhibit. The output is the shuffled -value mapped to $(0,1)$ with the $2(2u+1)/2^{31}$ form that keeps both +value mapped to $(0,1)$ as `(2 * ret_val + 1) / 2^31`, which keeps both endpoints excluded. `SkipTo` is a no-op for this engine. `Clone` reconstructs the generator from its original seed instead of preserving the advanced state; `Branch` creates another seeded generator. @@ -264,12 +264,28 @@ the advanced state; `Branch` creates another seeded generator. `MRG32k32a_` is L'Ecuyer's MRG32k32a combined multiple-recursive generator: two order-3 recurrences with moduli $m_1 = 4294967087$ and $m_2 = 4294944443$ -are combined as $u = ((x - y) \bmod m_1) / (m_1 + 1)$. `SkipTo` uses binary -exponentiation of the $3 \times 3$ companion matrices to jump from the initial -state. Its skip count halves $n \cdot \text{NDim}$ to match the generated/cached -antithetic pattern of `FillUniform`. Because `FillNormal` instead consumes a fresh -uniform for every component, MRG32 seeking is not replay-equivalent for normal-path -substreams and must not be used as such. Sobol remains the verified normal-path +are combined by the implementation's piecewise expression + +$$ +u = +\begin{cases} +\dfrac{x-y}{m_1+1}, & x>y, \\ +\dfrac{x-y+m_1}{m_1+1}, & x\le y. +\end{cases} +$$ + +The equality case therefore returns $m_1/(m_1+1)$ rather than zero. `SkipTo` +uses binary exponentiation of the $3 \times 3$ companion matrices to jump from +the initial recurrence state. On a fresh generator, `SkipTo(n)` is replay-equivalent +for `FillUniform` only when $n$ is an even path offset: the skipped paths then form +complete generated/antithetic pairs, so the recurrence jump consumes +$n\,\text{NDim}/2$ engine values. An odd offset would require the cached uniform +vector and antithetic toggle from the preceding generated path. `SkipTo` calls +`Reset()` for the recurrence but does not reset or reconstruct `anti_` or `cache_`, +so odd offsets are not replay-equivalent; for the same reason, seeking on a reused +generator has no replay-equivalent contract. `FillNormal` consumes a fresh uniform +for every component and bypasses the cache entirely, so MRG32 seeking is not +replay-equivalent for normal-path substreams. Sobol remains the verified normal-path seeking surface. ## Selection Guidance @@ -283,10 +299,10 @@ seeking surface. control variates with estimated coefficients), and for regression-based estimators where low-discrepancy bias is undesirable. Between the two engines, `MRG32` has the stronger theoretical recurrence, and its current - `SkipTo` accounting matches antithetic `FillUniform` rather than - replay-equivalent normal-path substreams; `IRN` is retained as a lightweight - Knuth-style alternative with no seek implementation. Use Sobol when direct - normal-path seeking is required. + `SkipTo` accounting replays `FillUniform` only at even path offsets on a fresh + generator, not on odd offsets, reused generators, or normal-path substreams; + `IRN` is retained as a lightweight Knuth-style alternative with no seek + implementation. Use Sobol when direct normal-path seeking is required. ## Benchmark Coverage From b42bef0c296d23123fe7c52edf4647ba279a9c79 Mon Sep 17 00:00:00 2001 From: wegamekinglc Date: Wed, 15 Jul 2026 14:47:39 +0800 Subject: [PATCH 07/19] docs: complete XCCY methodology coverage --- docs/methodology/xccy_calibration.md | 90 +++++++++++++++++++----- docs/methodology/yield_curve_jacobian.md | 22 +++++- 2 files changed, 90 insertions(+), 22 deletions(-) diff --git a/docs/methodology/xccy_calibration.md b/docs/methodology/xccy_calibration.md index a655f03f7..fb5cdc413 100644 --- a/docs/methodology/xccy_calibration.md +++ b/docs/methodology/xccy_calibration.md @@ -73,7 +73,9 @@ holiday calendar, business-day convention, and fixing hour/minute. The reset for domestic period $i\ge1$ is effective on that period's accrual start; its FX fixing date is the lagged and adjusted effective date. `FixingIdentity_` gives the index name and hour/minute for each domestic and foreign floating-rate -fixing. `FxIndexName(pair)` gives the FX fixing index name. +fixing. `FxIndexName(domestic, foreign)` gives the canonical FX fixing index +name `FX[foreign/domestic]`, matching the domestic-per-foreign market +convention. At valuation time $v$: @@ -82,22 +84,31 @@ At valuation time $v$: active forward value; and - a fixing after $v$ uses the active forward value. -Only a historical fixing attached to a non-settled cashflow or reset is -required. Missing required observations fail with the index, timestamp, and -pricing context. +Historical requests are deduplicated and include only observations that still +feed unsettled coupons, domestic notionals, or MTM reset dependencies. Missing +required observations fail with the index, timestamp, and pricing context. ## Immutable Market-Fixing Snapshot `MarketFixingSnapshot_` is an immutable nested map `index name -> DateTime_ -> value`. The same snapshot can contain rate and FX -observations and is retained by the calibration result. The snapshot currently -accepts positive values only: the constructor rejects non-positive entries, so -rate fixings from negative-rate regimes (for example EURIBOR, ESTR, TIBOR, or -SARON over 2014–2022) are not accepted. Supplying a snapshot -makes it authoritative, including when it is explicitly empty. When a snapshot -is omitted, staged and joint XCCY calibration first collect all historical -requests across all instruments and copy those values from the process-wide -fixing store once. Later global mutations cannot change the captured solve. +observations and is retained by the calibration result. Index names must be +non-empty, timestamps valid, and observations positive and finite. The +positive-value requirement means that rate fixings from negative-rate regimes +(for example EURIBOR, ESTR, TIBOR, or SARON over 2014–2022) are not accepted. + +Direct lookup wins. If that observation is absent and the name is canonical FX, +lookup returns the reciprocal of the reverse canonical observation. When both +directions exist at the same timestamp, the snapshot requires +`abs(direct * reverse - 1) <= 1e-10`. + +Core and Python map inputs can contain at most one value for each `(index name, +timestamp)` pair. The Excel parallel-array adapter rejects duplicate pairs +explicitly. Supplying a snapshot makes it authoritative, including when it is +explicitly empty. When a snapshot is omitted, staged and joint XCCY calibration +first collect and deduplicate all required historical observations across all +instruments and copy them from the process-wide fixing store once. Later global +mutations cannot change the captured solve. ## Staged Basis Calibration @@ -115,6 +126,10 @@ approximate mode minimizes the fit subject to smoothness regularization. The result contains the basis-bearing `CrossCurrencyMarket_`, FX forwards at the basis knots, fit diagnostics, and optional matrices. +For `nInstruments` quotes and `nBasisParameters` basis parameters, the staged +forward Jacobian has shape `nInstruments x nBasisParameters`; the effective +inverse has shape `nBasisParameters x nInstruments`. + ## Joint Domestic, Foreign, and Basis Calibration `CalibrateJointXccyMarket` solves one residual system over three ordered groups: @@ -142,13 +157,26 @@ come from pricing and curve routing, not regularization. The result retains both solved currency blocks, the basis curve, FX forwards, the fixing snapshot, group and joint diagnostics, market/model/residual vectors, ranges, and optional forward/effective-inverse Jacobian matrices. +`effJacobianInverse_` has shape `totalParameters x totalResiduals`. Both +matrices live on the top-level joint result rather than inside the domestic, +foreign, or XCCY group diagnostics. ## Analytic and Bumped Jacobians -Both staged and joint calibration accept `ANALYTIC` and `BUMPED`. In exact mode, -analytic calibration can populate the forward Jacobian at the solution; either -mode can populate the solver's effective inverse when requested. Approximate -mode does not expose either matrix. +Both staged and joint calibration accept `ANALYTIC` and `BUMPED`. At the +accepted exact solution, the exposed forward matrix is the unscaled analytic +residual Jacobian. The effective inverse is the weighted pseudoinverse formed +from the solver's tolerance-scaled Jacobian at that same solved state; it is not +the literal inverse of the exposed forward matrix. For a raw decimal quote +perturbation $\Delta q$, the parameter move is + +$$ +\Delta x = \mathrm{effJacobianInverse}\,\Delta q / \mathrm{tolerance}. +$$ + +Exact analytic calibration may produce both matrices. Exact bumped calibration +produces only the effective inverse. Approximate calibration produces neither, +and the forward/inverse options can suppress their computations independently. Joint XCCY analytic calibration is fail-fast. Every domestic and foreign declaration must satisfy the joint curve AAD gates, including `ACT_365F`, a @@ -159,17 +187,41 @@ If any analytic gate fails, the exception identifies the ineligible currency, declaration, instrument, or reset. Select `BUMPED` explicitly to run the same valid residual system without the analytic eligibility requirement. +## Surface Availability + +| Surface | Staged XCCY | Joint XCCY | +|---------|-------------|------------| +| Core/public C++ | Full options, forward Jacobian, and effective inverse | Full options and both top-level matrices | +| Python | Default staged solve with market/FX-forward and fit diagnostics; no staged matrix bindings | Options, named ranges, forward Jacobian, and effective inverse | +| Excel | Basis handle and fit diagnostics; no staged matrix views | Options plus forward-Jacobian/range views; no effective-inverse worksheet getter | + ## Runnable Examples +- `xccy_curve_calibration` performs staged basis-only calibration against + supplied domestic and foreign curve blocks and prints fit and FX-forward + diagnostics. - `xccy_reset_pricing` prices future fixed, resettable, and MTM swaps against piecewise-constant discount, projection, and basis curves, validates their reset and notional behavior, and prices an already-started MTM swap from an immutable snapshot of historical domestic-rate, foreign-rate, and FX fixings. - `xccy_mtm_calibration` builds known domestic, foreign, and basis curves, derives self-consistent quotes, supplies an immutable fixing snapshot for an - already-started MTM swap, and recovers all three parameter blocks in one joint - calibration. It prints the convergence residual, Jacobian shape, block ranges, - and parameter-recovery errors. + already-started MTM swap, and recovers five declaration blocks across the + domestic, foreign, and basis groups in one joint calibration. It prints the + convergence residual, Jacobian shape, block ranges, and parameter-recovery + errors. +- `dal-python/examples/007.xccy_joint_calibration.py` runs joint calibration + through the installed Python surface and prints convergence, matrix + dimensions, named ranges, and FX forwards. + +## Performance Smoke Surface + +`xccy_perf` emits 24 unique timing rows. They cover four pricing cases (future +fixed, resettable, and MTM plus started MTM), staged calibration including the +reset-aware analytic case, and joint calibration. Linux and Windows CI execute +the benchmark to completion, but it is not in the paired base/head regression +allowlist; this target is execution-smoke and reporting coverage rather than a +regression threshold gate. ## See Also diff --git a/docs/methodology/yield_curve_jacobian.md b/docs/methodology/yield_curve_jacobian.md index d648d4689..4f05ff72d 100644 --- a/docs/methodology/yield_curve_jacobian.md +++ b/docs/methodology/yield_curve_jacobian.md @@ -363,13 +363,29 @@ The immutable fixing snapshot is passive data: historical observations have no parameter adjoints, while future or unsupplied at-valuation observations remain active forward values. +`effJacobianInverse_` has the complementary shape +`totalParameters x totalResiduals` and follows the same published parameter and +residual range order. Both matrices are retained on the top-level +`JointXccyCalibrationResult_`, not on its per-group diagnostics. At the accepted +exact solution, the forward matrix is the unscaled analytic Jacobian, while the +effective inverse is the weighted pseudoinverse of the solver's +tolerance-scaled Jacobian. They are therefore not literal inverses in their +exposed forms. A raw decimal quote perturbation maps to parameter coordinates as + +$$ +\Delta x = \mathrm{effJacobianInverse}\,\Delta q / \mathrm{tolerance}. +$$ + Unlike generic joint multi-curve calibration, requested XCCY `ANALYTIC` mode is fail-fast. An unsupported declaration, day basis, instrument route, or malformed XCCY plan raises an eligibility error naming the offending group. `BUMPED` remains available for every otherwise-valid spec. In exact bumped mode, -`jacobianAtSolution_` is empty while `effJacobianInverse_` may still be retained. -Approximate mode exposes neither matrix. The two matrix computations can also be -disabled independently with `JointXccyCalibrationOptions_`. +`jacobianAtSolution_` is empty while `effJacobianInverse_` is retained when +requested. Exact analytic mode may retain both matrices; approximate mode +exposes neither. `JointXccyCalibrationOptions_` can suppress the two matrix +computations independently. Core/public C++ and joint Python expose both +top-level matrices. The Excel joint worksheet surface exposes the forward +Jacobian and named ranges but has no effective-inverse getter. The regression suite compares the full analytic stack against two-sided central differences and verifies that the published parameter and residual ranges From 216a7134647f42c26939e8aa7fa12716bd7c4e9d Mon Sep 17 00:00:00 2001 From: wegamekinglc Date: Wed, 15 Jul 2026 14:54:25 +0800 Subject: [PATCH 08/19] docs: align XCCY example output description --- docs/methodology/xccy_calibration.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/methodology/xccy_calibration.md b/docs/methodology/xccy_calibration.md index fb5cdc413..1f4994f89 100644 --- a/docs/methodology/xccy_calibration.md +++ b/docs/methodology/xccy_calibration.md @@ -198,8 +198,8 @@ valid residual system without the analytic eligibility requirement. ## Runnable Examples - `xccy_curve_calibration` performs staged basis-only calibration against - supplied domestic and foreign curve blocks and prints fit and FX-forward - diagnostics. + supplied domestic and foreign curve blocks and prints fit diagnostics, + including the FX spot and maximum residual, plus elapsed time. - `xccy_reset_pricing` prices future fixed, resettable, and MTM swaps against piecewise-constant discount, projection, and basis curves, validates their reset and notional behavior, and prices an already-started MTM swap from an From 4b2528c623e20f7204c1d0fcd2f46bde4134932f Mon Sep 17 00:00:00 2001 From: wegamekinglc Date: Wed, 15 Jul 2026 15:01:46 +0800 Subject: [PATCH 09/19] docs: clarify XCCY API availability --- dal-excel/README.md | 30 +++++++++++++++++------- dal-python/README.md | 15 +++++++++++- docs/public-api.md | 55 +++++++++++++++++++++++++++++++++++--------- 3 files changed, 80 insertions(+), 20 deletions(-) diff --git a/dal-excel/README.md b/dal-excel/README.md index 2f0bba1be..141656c58 100644 --- a/dal-excel/README.md +++ b/dal-excel/README.md @@ -32,9 +32,11 @@ worksheet calls: Curve workflows use convention/instrument constructors followed by `CALIBRATE.SINGLECURVE`, `CALIBRATE.XCCYMARKET`, or -`CALIBRATE.JOINTXCCY`; result accessors return diagnostics, matrices, ranges, or -curve handles. The [public API guide](../docs/public-api.md#excel) lists the -primary worksheet families. +`CALIBRATE.JOINTXCCY`; result accessors return diagnostics, supported matrix or +range views, and curve handles. Matrix visibility differs between staged and +joint XCCY as described below. The +[public API guide](../docs/public-api.md#excel) lists the primary worksheet +families. ## Resettable and Joint XCCY Functions @@ -47,8 +49,17 @@ domestic and foreign rate-fixing identities. Pass that handle to `MARKETFIXINGSNAPSHOT.NEW` takes parallel index-name, fixing-time, and value ranges and returns one immutable snapshot handle. The arrays must have equal length, timestamps must be valid, and observations must be positive and finite. -The snapshot can contain both rate and canonical FX names such as -`FX[EUR/USD]`. +Repeated `(index name, timestamp)` rows are rejected. The canonical name for a +domestic/foreign pair is `FX[foreign/domestic]`, for example `FX[EUR/USD]` for +USD/EUR. Lookup uses the requested direction when present and otherwise uses +the reciprocal of the reverse canonical observation. If both directions are +present at one timestamp, their product must differ from one by no more than +`1e-10`. + +Staged `CALIBRATE.XCCYMARKET` returns a basis-curve handle plus fit diagnostics. +`XCCYCALIBRATIONRESULT.GET` exposes `marketRates`, `modelRates`, `residuals`, +`maxAbsResidual`, and `rmsResidual`; neither the staged forward Jacobian nor the +staged effective inverse is worksheet-visible. `CALIBRATE.JOINTXCCY` performs one domestic/foreign/basis solve. Its result supports dedicated handle getters: @@ -57,10 +68,13 @@ supports dedicated handle getters: - `JOINTXCCYCALIBRATIONRESULT.GET.FOREIGNBLOCK` - `JOINTXCCYCALIBRATIONRESULT.GET.BASISCURVE` -`JOINTXCCYCALIBRATIONRESULT.GET` returns matrix views selected by +Joint settings can request both forward-Jacobian and effective-inverse +computation. `JOINTXCCYCALIBRATIONRESULT.GET` returns views selected by `fxForwards`, `marketRates`, `modelRates`, `residuals`, `jacobian`, -`parameterRanges`, or `residualRanges`. The generated HTML under `auto/` is the -exact argument and settings-key reference. +`parameterRanges`, or `residualRanges`; `jacobian` is the worksheet-visible +forward matrix and the range selectors publish its named layout. No worksheet +selector exposes the retained joint effective inverse. The generated HTML +under `auto/` is the exact argument and settings-key reference. ## Layout and Generated Registration diff --git a/dal-python/README.md b/dal-python/README.md index 51cfca03e..e729d7c67 100644 --- a/dal-python/README.md +++ b/dal-python/README.md @@ -306,12 +306,17 @@ python -m pytest tests -k "test_date" -v ``` Tests are located in `tests/` and cover: + - Date arithmetic and comparisons - Vector and matrix operations - Model construction (BS, Dupire) - Monte Carlo pricing accuracy vs Black-Scholes analytical formulas - AAD Greek computation and validation - Random number generator properties +- Curve construction plus single and staged multi-curve calibration +- Staged XCCY basis calibration and fit diagnostics +- Resettable/MTM XCCY construction with immutable fixing snapshots +- Joint domestic/foreign/basis XCCY calibration, including matrix and named-range contracts ## Project Structure @@ -422,6 +427,10 @@ result = dal.calibrate_curve( Python exposes single, staged multi-curve, staged XCCY basis, and simultaneous joint XCCY calibration. A base curve is multiplied into the calibrated component; it is not a replacement for the pricing discount curve required by a forward-curve stage. +The staged `CalibrateXccyMarket(spec)` binding is the one-argument default solve: +it returns the calibrated market, FX forwards, and scalar/vector fit diagnostics +only. Staged matrix options and matrix fields remain available only through the +C++ surface. ### Resettable and Joint XCCY Calibration @@ -449,7 +458,11 @@ foreign curve blocks, `fx_forward_curve`, basis curve, retained snapshot, group diagnostics, full market/model/residual vectors, analytic Jacobian, effective inverse, and named `parameter_ranges` / `residual_ranges`. Pass `JointXccyCalibrationOptions_` to select `ANALYTIC` or `BUMPED` and to disable -either diagnostic matrix. +either diagnostic matrix. The `eff_jacobian_inverse` matrix has shape +`totalParameters x totalResiduals` and is the weighted inverse of the solver's +tolerance-scaled Jacobian. Transforming a raw decimal quote bump therefore +requires division by the spec's `tolerance_`; see the +[Jacobian methodology](../docs/methodology/yield_curve_jacobian.md#joint-xccy-jacobian-layout). The runnable [joint XCCY calibration example](examples/007.xccy_joint_calibration.py) uses an explicit fixing snapshot for a started MTM trade. It prints convergence, diff --git a/docs/public-api.md b/docs/public-api.md index 5e2fbe45b..32e8a8459 100644 --- a/docs/public-api.md +++ b/docs/public-api.md @@ -149,14 +149,33 @@ For staged calibration, assemble `MultiCurveCalibrationSpec_` and call - `JointXccyCalibrationSpecBuilder_` / `CalibrateJointXccyMarket` solves the domestic declarations, foreign declarations, and basis declaration together. -`CrossCurrencySwapConfigBuilder_` selects `XccyNotionalMode_::{FIXED, -RESETTABLE,MARK_TO_MARKET}`, explicit domestic/foreign `FixingIdentity_` values, -and an `FxResetConvention_`. `MarketFixingSnapshotNew` creates an immutable -rate-and-FX observation set for in-progress swaps. Joint results expose the -three solved curve handles plus FX forwards, market/model/residual vectors, the -forward Jacobian, and named parameter/residual ranges through -`JointXccyResult*` accessors. See -[cross-currency pricing and calibration](methodology/xccy_calibration.md). +`CrossCurrencySwapConfigBuilder_` selects +`XccyNotionalMode_::Value_::FIXED`, +`XccyNotionalMode_::Value_::RESETTABLE`, or +`XccyNotionalMode_::Value_::MARK_TO_MARKET`, explicit domestic/foreign +`FixingIdentity_` values, and an `FxResetConvention_`. +`MarketFixingSnapshotNew` creates an immutable rate-and-FX observation set for +in-progress swaps. + +The public XCCY header includes the core staged and joint result types. Staged +C++ callers can use `CrossCurrencyCalibrationOptions_`; the returned +`CrossCurrencyCalibrationDiagnostics_` owns the optional forward Jacobian and +effective inverse. Joint results similarly own both matrices at the top level. +The `JointXccyResult*` facade helpers expose the three solved curve handles, FX +forwards, market/model/residual vectors, the forward Jacobian, and named +parameter/residual ranges. There is no dedicated effective-inverse facade +helper; C++ consumers read +`JointXccyCalibrationResult_::effJacobianInverse_` directly. + +For staged XCCY, the forward/inverse shapes are +`nInstruments x nBasisParameters` and +`nBasisParameters x nInstruments`. For joint XCCY they are +`totalResiduals x totalParameters` and +`totalParameters x totalResiduals`. The effective inverse is based on the +solver's tolerance-scaled Jacobian, so a raw decimal quote-risk transform also +divides by the calibration `tolerance_`. See +[cross-currency pricing and calibration](methodology/xccy_calibration.md) and +the [Jacobian methodology](methodology/yield_curve_jacobian.md#joint-xccy-jacobian-layout). Set `parameterization_ = CurveParameterization_::Value_::ZERO_RATE` to calibrate future zero-rate nodes. `initialGuess_` and `initialGuessPerNode_` are decimal continuously @@ -262,14 +281,23 @@ curve = dal.DiscountZeroRate_New( The returned `DiscountZeroRate_` exposes read-only `anchor_date`, `node_dates`, `zero_rates`, `day_count`, and `log_df_scheme` properties. -Python exposes the joint XCCY declarations, builder, options, calibration entry -point, and result surface with both trailing-underscore and snake-case aliases. +Python staged XCCY exposes only the one-argument default +`CalibrateXccyMarket(spec)` solve. Its result provides the calibrated market, +FX forwards, and scalar/vector fit diagnostics; staged matrix options and +matrix fields remain C++-only. + +Python joint XCCY exposes the declarations, builder, +`JointXccyCalibrationOptions_`, calibration entry point, and result surface +with both trailing-underscore and snake-case aliases. `JointXccyCalibrationResult_` provides `domestic_curve_block`, `foreign_curve_block`, `basis_curve`, `fx_forward_curve`, `fixings`, group diagnostics, `market_rates`, `model_rates`, `residuals`, `jacobian_at_solution`, `eff_jacobian_inverse`, `parameter_ranges`, and `residual_ranges`. `CalibrateJointXccyMarket(spec, options)` selects analytic or -bumped Jacobians and optional matrix construction. +bumped Jacobians and optional matrix construction. The effective inverse has +shape `totalParameters x totalResiduals`; applying it to a raw decimal quote +bump requires division by the spec's `tolerance_`, as described in the +[Jacobian methodology](methodology/yield_curve_jacobian.md#joint-xccy-jacobian-layout). See [dal-python/README.md](../dal-python/README.md) for package-focused examples. @@ -323,8 +351,13 @@ foreign discount-instrument/knot group, configured XCCY instruments, basis knots, an optional immutable snapshot handle, and two-column settings. Dedicated result functions return the domestic block, foreign block, and basis curve handles. +Staged `XCCYCALIBRATIONRESULT.GET` exposes fit vectors and scalars but neither +the staged forward Jacobian nor the staged effective inverse. `JOINTXCCYCALIBRATIONRESULT.GET` returns `fxForwards`, `marketRates`, `modelRates`, `residuals`, `jacobian`, `parameterRanges`, or `residualRanges`. +Joint settings can request both matrix computations, but `jacobian` is the only +matrix available through a worksheet selector; there is no +`effJacobianInverse` worksheet getter. Generated function help under `dal-excel/auto/*.htm` is the argument-level catalog used by Excel registration. From ab4ca65157672550ec1920a1c005fe8daaad51a5 Mon Sep 17 00:00:00 2001 From: wegamekinglc Date: Wed, 15 Jul 2026 15:08:38 +0800 Subject: [PATCH 10/19] docs: distinguish staged XCCY entry points --- docs/public-api.md | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/docs/public-api.md b/docs/public-api.md index 32e8a8459..5d087d967 100644 --- a/docs/public-api.md +++ b/docs/public-api.md @@ -144,8 +144,9 @@ The facade separates construction from solving: For staged calibration, assemble `MultiCurveCalibrationSpec_` and call `CalibrateMultiCurveBundle`. Cross-currency calibration has two paths: -- `CrossCurrencyCalibrationSpecBuilder_` / `CalibrateXccyMarket` calibrates a - basis curve over supplied domestic and foreign blocks. +- `CrossCurrencyCalibrationSpecBuilder_` / the one-argument public convenience + facade `CalibrateXccyMarket(spec)` calibrates a basis curve over supplied + domestic and foreign blocks. - `JointXccyCalibrationSpecBuilder_` / `CalibrateJointXccyMarket` solves the domestic declarations, foreign declarations, and basis declaration together. @@ -158,7 +159,9 @@ For staged calibration, assemble `MultiCurveCalibrationSpec_` and call in-progress swaps. The public XCCY header includes the core staged and joint result types. Staged -C++ callers can use `CrossCurrencyCalibrationOptions_`; the returned +C++ callers that need `CrossCurrencyCalibrationOptions_` use the core +`CalibrateCrossCurrencyMarket(spec, options)` entry point; there is no +`CalibrateXccyMarket(spec, options)` public-facade overload. The returned `CrossCurrencyCalibrationDiagnostics_` owns the optional forward Jacobian and effective inverse. Joint results similarly own both matrices at the top level. The `JointXccyResult*` facade helpers expose the three solved curve handles, FX From f4a59b009a9f410e786c5ebb95d5e5939390695c Mon Sep 17 00:00:00 2001 From: wegamekinglc Date: Wed, 15 Jul 2026 15:24:02 +0800 Subject: [PATCH 11/19] docs: record repository documentation audit --- ...repository-documentation-reconciliation.md | 76 +++++++++++++++++++ 1 file changed, 76 insertions(+) create mode 100644 .codex/artifacts/reviews/repository-documentation-reconciliation.md diff --git a/.codex/artifacts/reviews/repository-documentation-reconciliation.md b/.codex/artifacts/reviews/repository-documentation-reconciliation.md new file mode 100644 index 000000000..1259f47fe --- /dev/null +++ b/.codex/artifacts/reviews/repository-documentation-reconciliation.md @@ -0,0 +1,76 @@ +# DAL Repository Documentation Reconciliation + +- Baseline: `1589089bdf10df352ce5cf9cde963fd6b51a4f95` +- Published baseline set: 33 files from `.github/scripts/check_docs.py` +- Reconciliation head before this audit record: `ab4ca65135276cb9443afc9decdf490e921c9d72` + +## Executive Summary + +The baseline set contains 14 changed documents, five historical artifacts moved intact to +`.codex/artifacts/`, and 14 documents verified current without edits. The reconciliation +uses public headers, bindings, generated registrations, source, tests, examples, benchmarks, +build configuration, and the accepted Task 1-5 diffs as evidence; structural documentation +checks alone are not treated as proof of technical accuracy. + +## Document Dispositions + +| Baseline document | Disposition | Source evidence | Result | +|-------------------|-------------|-----------------|--------| +| `CHANGELOG.md` | `changed` | XCCY pricing/calibration source; `XccyNotionalMode_::Value_`; Python and Excel joint-result registrations | Current capability baseline now covers fixed, resettable, and MTM XCCY, immutable snapshots, staged/joint calibration, and the exact per-surface matrix split; dated history remains qualified. | +| `CONTRIBUTING.md` | `verified-current` | `build_linux.sh`; `CMakePresets.json`; `CalibrationTest`; `PublicApiTest`; `.github/scripts/check_benchmark_regressions.py` | Build, generation, targeted-test, and paired benchmark instructions match the current developer workflow. | +| `README.md` | `changed` | Registered XCCY examples; Python example 007; XCCY methodology | The overview, example navigation, and methodology shortlist now make cross-currency pricing and calibration discoverable without duplicating the detailed contract. | +| `dal-cpp/README.md` | `verified-current` | `dal-cpp/CMakeLists.txt`; `Platform.cmake`; `ThreadPool_`; generation targets | Target ownership, lazy/configurable concurrency, build options, and generated-code workflow match the core implementation. | +| `dal-excel/README.md` | `changed` | `__curveprotocol.cpp`; `__xccycalibration.cpp`; generated joint getter; `ExcelApiTest` | Snapshot duplicate/reciprocal rules and staged-versus-joint worksheet matrix visibility now match the registered Excel surface. | +| `dal-public/README.md` | `verified-current` | `dal-public/CMakeLists.txt`; installed consumer; public builders, XCCY helpers, and `PublicApiTest` | The facade, compatibility scope, installed target, header families, and tests are described accurately. | +| `dal-python/README.md` | `changed` | `dal-python/src/bindings/curve.cpp`; curve and XCCY pytest suites; Python example 007 | Staged diagnostics-only and joint options/matrix availability are separated, inverse scaling is warned, and the test inventory includes curve and XCCY coverage. | +| `dal-web/README.md` | `verified-current` | `app.native_runtime`; `DalGateway`; `valuation.py`; native valuation binding; start scripts | Native-only preflight, staged runtime, async thread handoff, GIL release, gateway serialization, and matrix support match the current web implementation. | +| `docs/README.md` | `changed` | Curve parameterization/factory source; XCCY and Jacobian methodology | Heading hierarchy is corrected, LOG_DISCOUNT exclusivity is removed, and XCCY snapshot/range/inverse topics are indexed. | +| `docs/architecture.md` | `changed` | `MarketFixingSnapshot_`; global fixing store; generic, staged-XCCY, and joint-XCCY calibration entry points | Mutable process state is distinguished from immutable operation snapshots, and calibration architecture now branches by implemented workflow. | +| `docs/experimental/aad-analytic-jacobian-curve-calibration.md` | `verified-current` | `calibration.cpp`; `jointcalibration.cpp`; `aadjacobian.cpp`; analytic-Jacobian tests | The former rollout note is a concise redirect to the current normative methodology and no longer labels shipped behavior experimental. | +| `docs/experimental/replicate-ptirds-single-currency-curve.md` | `changed` | `PTIRDSCurveTest`; `Holidays::IsBusinessDay`; calibration source | The every-day target is now distinguished from `Holidays::None()` plus `Unadjusted` DAL conventions, and comparison language is neutral. | +| `docs/installation.md` | `verified-current` | `build_linux.sh`; presets; exported DAL CMake packages; installed consumer; web preflight | Core/full/distribution builds, staged installs, downstream consumption, CPU tuning, tests, and native web startup are current. | +| `docs/methodology/aad.md` | `changed` | `Rewind`; `RewindToMark`; `MCSimulation`; tape backends | Full-recording rewind and per-path checkpoint rewind are now named and timed according to the tape implementation. | +| `docs/methodology/black_scholes.md` | `verified-current` | Black/Bachelier option and implied-volatility source; real-domain round-trip tests | Formula, units, intrinsic handling, and Bachelier real-forward/strike support match the corrected implementation. | +| `docs/methodology/dupire.md` | `verified-current` | `IVS_::Call`; `IVS_::LocalVol`; rate-aware local-vol and repricing tests | Discounting, carry, strike derivative, grid layout, and tail behavior match the current rate-aware implementation. | +| `docs/methodology/interpolation.md` | `changed` | `Cubic1_`; `NaturalCubicWeightGeometry_`; mixed interpolation source/tests | Linear error order, cubic curvature factor, endpoint third-derivative semantics, and linear-head/cubic-tail orientation are accurate. | +| `docs/methodology/log_discount_curve.md` | `verified-current` | log-DF and zero-rate curve implementations; parameterization and PTIRDS tests | Anchor layout, interpolation schemes, extrapolation, AAD propagation, persistence, and zero-rate mapping remain current. | +| `docs/methodology/matrix.md` | `changed` | tri-diagonal solve; `BandedCholesky_`; `CholeskyImpl`; asymmetric and Krylov tests | Thomas notation, factorization/solve complexity, strict-dominance qualification, and dense regularization now describe the algorithms exactly. | +| `docs/methodology/pde.md` | `verified-current` | PDE grid/operators/theta scheme source and tests; `european_fd` | Coordinate maps, nonuniform operators, prepared-state contract, boundary ownership, and example flow match the implementation. | +| `docs/methodology/quadrature.md` | `verified-current` | quadrature source; Hermite moment and Simpson refinement tests | Gauss-Hermite exactness, Simpson point semantics, and fourth-order convergence match source and independent tests. | +| `docs/methodology/random.md` | `changed` | Sobol and pseudo-random implementations; simulation batch driver; RNG tests | Seeking is implementation-specific; IRN clone/branch behavior and the bounded MRG32 uniform replay contract are exact, while normal-path replay equivalence is explicitly not promised. | +| `docs/methodology/script_engine.md` | `verified-current` | parser/preprocessor; tree, compiled, and fuzzy visitors; simulation parity tests | Language phases, eager booleans, time split, compilation, batching, RNG selection, and AAD execution remain source-accurate. | +| `docs/methodology/underdetermined_search.md` | `verified-current` | `BacktrackMinimum`; exact/approximate solver; inverse/Jacobian tests | Scaling, minimum-change solve, corrected quadratic minimizer, diagnostics, and controls match the optimizer. | +| `docs/methodology/xccy_calibration.md` | `changed` | fixing snapshot and XCCY pricing/calibration source/tests; bindings; examples; `xccy_perf` and CI wiring | Canonical/reverse FX rules, immutable dependency closure, staged/joint matrix semantics and availability, four examples, and the 24-row smoke surface are complete. | +| `docs/methodology/yield_curve.md` | `verified-current` | curve construction/calibration headers; analytic/inverse tests; registered yield-curve examples | Representation layouts, single/staged/generic-joint solves, Jacobian population, risk scaling, and examples remain current. | +| `docs/methodology/yield_curve_jacobian.md` | `changed` | joint XCCY result/solver source; `TestXccyJointJacobian`; Python and Excel registrations | Joint range placement, complementary dimensions, accepted-solution construction, tolerance scaling, population modes, and binding availability are explicit. | +| `docs/public-api.md` | `changed` | generated notional enum; core/public XCCY headers; Python bindings; Excel selectors | C++ constants are copy-safe, core versus facade staged entry points are distinguished, and staged/joint matrices are documented per surface without inventing getters. | +| `docs/superpowers/plans/2026-07-12-unified-yield-curve-interpolation-aad.md` | `moved-to-.codex` | Task 1 hash comparison and 100% rename | Preserved intact at `.codex/artifacts/plans/2026-07-12-unified-yield-curve-interpolation-aad.md`; removed from the current-state published set. | +| `docs/superpowers/plans/2026-07-12-ycinstrument-pricing-performance.md` | `moved-to-.codex` | Task 1 hash comparison and 100% rename | Preserved intact at `.codex/artifacts/plans/2026-07-12-ycinstrument-pricing-performance.md`; removed from the current-state published set. | +| `docs/superpowers/plans/2026-07-12-zero-rate-parameterization.md` | `moved-to-.codex` | Task 1 hash comparison and 100% rename | Preserved intact at `.codex/artifacts/plans/2026-07-12-zero-rate-parameterization.md`; removed from the current-state published set. | +| `docs/superpowers/specs/2026-07-12-ycinstrument-pricing-performance-design.md` | `moved-to-.codex` | Task 1 hash comparison and 100% rename | Preserved intact at `.codex/artifacts/specs/2026-07-12-ycinstrument-pricing-performance-design.md`; removed from the current-state published set. | +| `docs/superpowers/specs/2026-07-12-zero-rate-parameterization-design.md` | `moved-to-.codex` | Task 1 hash comparison and 100% rename | Preserved intact at `.codex/artifacts/specs/2026-07-12-zero-rate-parameterization-design.md`; removed from the current-state published set. | + +## Prior Audit Reconciliation + +| 2026-07-10 finding group | Current status | Evidence | +|--------------------------|----------------|----------| +| Numerical and simulation correctness | Resolved except for one documented current RNG limitation | Rate-aware Dupire is gated by flat-vol local-vol and repricing tests; `BacktrackMinimum` has an independent oracle test; Bachelier supports real forward/strike pairs; `ValueByMonteCarlo` rejects non-positive path counts with public/Python/Excel tests; Sobol policy/clone, mixed orientation, and quadrature exactness have focused tests. `docs/methodology/random.md` now retains the current limitation: MRG32 seeking is not replay-equivalent for normal-path substreams. | +| Runtime and concurrency | Resolved and documented | `ThreadPool_` is lazy, lifecycle-locked, and capped by `DAL_NUM_THREADS`; native valuation uses `py::gil_scoped_release`; evaluation-date and gateway locks retain the intentional process serialization boundary. Core, web, architecture, and Python documentation agree. | +| Web product behavior | Resolved and documented | `app.native_runtime` and both launchers enforce the native-only contract; `DalGateway` constructs mutable/nested `DoubleMatrix_` surfaces; `valuation.py` moves native work off the event loop while the gateway lock states the serialization boundary. | +| Packaging and installation | Resolved for the documented contract | `dal-cppConfig.cmake` and `dal-publicConfig.cmake` exports plus `tests/installed-consumer/` establish relocatable downstream CMake consumption. Full builds provision Python dependencies and install into `build/stage/` rather than relying on source-root artifacts. | +| Build configuration | Resolved and documented | `core-dev`, `full-dev`, and `distribution` presets split workflows; `Platform.cmake` applies configuration-aware optimization and makes `DAL_ENABLE_NATIVE_ARCH` opt-in; installation and contributor guides describe the same staged/generation behavior. | +| Public API and distribution metadata | Resolved or accurately bounded | The public layer is documented as a source-compatible convenience facade rather than an ABI boundary; Monte Carlo path validation is enforced before conversion; root and Python metadata use MIT. XCCY docs now spell generated enums exactly and distinguish unavailable Python/Excel matrix accessors from core/public fields. | +| Documentation governance and coverage | Resolved for current published docs | The shipped analytic-Jacobian note redirects to methodology; commands, component links, architecture, contribution, installation, and public API guides exist and are checker-owned. Historical implementation plans/specs moved intact to `.codex/artifacts/`; current-state docs contain no rollout narrative. | + +## Verification + +| Gate | Result | +|------|--------| +| Baseline disposition completeness | The exact plan assertion passed: 33 unique rows equal the current 28-file checker set plus the five moved baseline paths. | +| Documentation checker | `python3 .github/scripts/check_docs.py` passed for 28 current-state Markdown files. | +| Branch whitespace and scope | `git diff --check 1589089b..HEAD` passed; the complete branch diff contains only Markdown, and protected `CLAUDE.md`, `.claude/`, `AGENTS.md`, `dal-cpp/dal/auto`, and `dal-excel/auto` paths are unchanged. | +| C++ XCCY examples | `xccy_curve_calibration`, `xccy_reset_pricing`, and `xccy_mtm_calibration` each exited 0; outputs included the staged 15-instrument fit, four pricing cases, and the converged `25x25` joint solve with five named ranges. | +| Python XCCY example | Installed-surface `dal-python/examples/007.xccy_joint_calibration.py` exited 0 with a converged `3x3` solve, named ranges, and FX forwards. | +| XCCY benchmark smoke | The executable output parsed with the corrected single-backslash whitespace regex to exactly 24 unique timing rows. The doubled raw-regex backslashes in the written task command are not executable evidence. | +| Task reviews | Task 1 and Task 2 were accepted clean; Tasks 3, 4, and 5 were accepted after scoped follow-up commits and re-review. The accepted head before this artifact is `ab4ca651`. | +| Whole-branch review | Not yet performed. A fresh final reviewer is intentionally scheduled after this audit commit. | From cb3c9913aba1b72b616cf96f69725eae5e4d6e80 Mon Sep 17 00:00:00 2001 From: wegamekinglc Date: Wed, 15 Jul 2026 15:36:29 +0800 Subject: [PATCH 12/19] docs: make audit evidence durable --- ...repository-documentation-reconciliation.md | 66 +++++++++---------- 1 file changed, 33 insertions(+), 33 deletions(-) diff --git a/.codex/artifacts/reviews/repository-documentation-reconciliation.md b/.codex/artifacts/reviews/repository-documentation-reconciliation.md index 1259f47fe..796f9e5b5 100644 --- a/.codex/artifacts/reviews/repository-documentation-reconciliation.md +++ b/.codex/artifacts/reviews/repository-documentation-reconciliation.md @@ -16,39 +16,39 @@ checks alone are not treated as proof of technical accuracy. | Baseline document | Disposition | Source evidence | Result | |-------------------|-------------|-----------------|--------| -| `CHANGELOG.md` | `changed` | XCCY pricing/calibration source; `XccyNotionalMode_::Value_`; Python and Excel joint-result registrations | Current capability baseline now covers fixed, resettable, and MTM XCCY, immutable snapshots, staged/joint calibration, and the exact per-surface matrix split; dated history remains qualified. | -| `CONTRIBUTING.md` | `verified-current` | `build_linux.sh`; `CMakePresets.json`; `CalibrationTest`; `PublicApiTest`; `.github/scripts/check_benchmark_regressions.py` | Build, generation, targeted-test, and paired benchmark instructions match the current developer workflow. | -| `README.md` | `changed` | Registered XCCY examples; Python example 007; XCCY methodology | The overview, example navigation, and methodology shortlist now make cross-currency pricing and calibration discoverable without duplicating the detailed contract. | -| `dal-cpp/README.md` | `verified-current` | `dal-cpp/CMakeLists.txt`; `Platform.cmake`; `ThreadPool_`; generation targets | Target ownership, lazy/configurable concurrency, build options, and generated-code workflow match the core implementation. | -| `dal-excel/README.md` | `changed` | `__curveprotocol.cpp`; `__xccycalibration.cpp`; generated joint getter; `ExcelApiTest` | Snapshot duplicate/reciprocal rules and staged-versus-joint worksheet matrix visibility now match the registered Excel surface. | -| `dal-public/README.md` | `verified-current` | `dal-public/CMakeLists.txt`; installed consumer; public builders, XCCY helpers, and `PublicApiTest` | The facade, compatibility scope, installed target, header families, and tests are described accurately. | -| `dal-python/README.md` | `changed` | `dal-python/src/bindings/curve.cpp`; curve and XCCY pytest suites; Python example 007 | Staged diagnostics-only and joint options/matrix availability are separated, inverse scaling is warned, and the test inventory includes curve and XCCY coverage. | -| `dal-web/README.md` | `verified-current` | `app.native_runtime`; `DalGateway`; `valuation.py`; native valuation binding; start scripts | Native-only preflight, staged runtime, async thread handoff, GIL release, gateway serialization, and matrix support match the current web implementation. | -| `docs/README.md` | `changed` | Curve parameterization/factory source; XCCY and Jacobian methodology | Heading hierarchy is corrected, LOG_DISCOUNT exclusivity is removed, and XCCY snapshot/range/inverse topics are indexed. | -| `docs/architecture.md` | `changed` | `MarketFixingSnapshot_`; global fixing store; generic, staged-XCCY, and joint-XCCY calibration entry points | Mutable process state is distinguished from immutable operation snapshots, and calibration architecture now branches by implemented workflow. | -| `docs/experimental/aad-analytic-jacobian-curve-calibration.md` | `verified-current` | `calibration.cpp`; `jointcalibration.cpp`; `aadjacobian.cpp`; analytic-Jacobian tests | The former rollout note is a concise redirect to the current normative methodology and no longer labels shipped behavior experimental. | -| `docs/experimental/replicate-ptirds-single-currency-curve.md` | `changed` | `PTIRDSCurveTest`; `Holidays::IsBusinessDay`; calibration source | The every-day target is now distinguished from `Holidays::None()` plus `Unadjusted` DAL conventions, and comparison language is neutral. | -| `docs/installation.md` | `verified-current` | `build_linux.sh`; presets; exported DAL CMake packages; installed consumer; web preflight | Core/full/distribution builds, staged installs, downstream consumption, CPU tuning, tests, and native web startup are current. | -| `docs/methodology/aad.md` | `changed` | `Rewind`; `RewindToMark`; `MCSimulation`; tape backends | Full-recording rewind and per-path checkpoint rewind are now named and timed according to the tape implementation. | -| `docs/methodology/black_scholes.md` | `verified-current` | Black/Bachelier option and implied-volatility source; real-domain round-trip tests | Formula, units, intrinsic handling, and Bachelier real-forward/strike support match the corrected implementation. | -| `docs/methodology/dupire.md` | `verified-current` | `IVS_::Call`; `IVS_::LocalVol`; rate-aware local-vol and repricing tests | Discounting, carry, strike derivative, grid layout, and tail behavior match the current rate-aware implementation. | -| `docs/methodology/interpolation.md` | `changed` | `Cubic1_`; `NaturalCubicWeightGeometry_`; mixed interpolation source/tests | Linear error order, cubic curvature factor, endpoint third-derivative semantics, and linear-head/cubic-tail orientation are accurate. | -| `docs/methodology/log_discount_curve.md` | `verified-current` | log-DF and zero-rate curve implementations; parameterization and PTIRDS tests | Anchor layout, interpolation schemes, extrapolation, AAD propagation, persistence, and zero-rate mapping remain current. | -| `docs/methodology/matrix.md` | `changed` | tri-diagonal solve; `BandedCholesky_`; `CholeskyImpl`; asymmetric and Krylov tests | Thomas notation, factorization/solve complexity, strict-dominance qualification, and dense regularization now describe the algorithms exactly. | -| `docs/methodology/pde.md` | `verified-current` | PDE grid/operators/theta scheme source and tests; `european_fd` | Coordinate maps, nonuniform operators, prepared-state contract, boundary ownership, and example flow match the implementation. | -| `docs/methodology/quadrature.md` | `verified-current` | quadrature source; Hermite moment and Simpson refinement tests | Gauss-Hermite exactness, Simpson point semantics, and fourth-order convergence match source and independent tests. | -| `docs/methodology/random.md` | `changed` | Sobol and pseudo-random implementations; simulation batch driver; RNG tests | Seeking is implementation-specific; IRN clone/branch behavior and the bounded MRG32 uniform replay contract are exact, while normal-path replay equivalence is explicitly not promised. | -| `docs/methodology/script_engine.md` | `verified-current` | parser/preprocessor; tree, compiled, and fuzzy visitors; simulation parity tests | Language phases, eager booleans, time split, compilation, batching, RNG selection, and AAD execution remain source-accurate. | -| `docs/methodology/underdetermined_search.md` | `verified-current` | `BacktrackMinimum`; exact/approximate solver; inverse/Jacobian tests | Scaling, minimum-change solve, corrected quadratic minimizer, diagnostics, and controls match the optimizer. | -| `docs/methodology/xccy_calibration.md` | `changed` | fixing snapshot and XCCY pricing/calibration source/tests; bindings; examples; `xccy_perf` and CI wiring | Canonical/reverse FX rules, immutable dependency closure, staged/joint matrix semantics and availability, four examples, and the 24-row smoke surface are complete. | -| `docs/methodology/yield_curve.md` | `verified-current` | curve construction/calibration headers; analytic/inverse tests; registered yield-curve examples | Representation layouts, single/staged/generic-joint solves, Jacobian population, risk scaling, and examples remain current. | -| `docs/methodology/yield_curve_jacobian.md` | `changed` | joint XCCY result/solver source; `TestXccyJointJacobian`; Python and Excel registrations | Joint range placement, complementary dimensions, accepted-solution construction, tolerance scaling, population modes, and binding availability are explicit. | -| `docs/public-api.md` | `changed` | generated notional enum; core/public XCCY headers; Python bindings; Excel selectors | C++ constants are copy-safe, core versus facade staged entry points are distinguished, and staged/joint matrices are documented per surface without inventing getters. | -| `docs/superpowers/plans/2026-07-12-unified-yield-curve-interpolation-aad.md` | `moved-to-.codex` | Task 1 hash comparison and 100% rename | Preserved intact at `.codex/artifacts/plans/2026-07-12-unified-yield-curve-interpolation-aad.md`; removed from the current-state published set. | -| `docs/superpowers/plans/2026-07-12-ycinstrument-pricing-performance.md` | `moved-to-.codex` | Task 1 hash comparison and 100% rename | Preserved intact at `.codex/artifacts/plans/2026-07-12-ycinstrument-pricing-performance.md`; removed from the current-state published set. | -| `docs/superpowers/plans/2026-07-12-zero-rate-parameterization.md` | `moved-to-.codex` | Task 1 hash comparison and 100% rename | Preserved intact at `.codex/artifacts/plans/2026-07-12-zero-rate-parameterization.md`; removed from the current-state published set. | -| `docs/superpowers/specs/2026-07-12-ycinstrument-pricing-performance-design.md` | `moved-to-.codex` | Task 1 hash comparison and 100% rename | Preserved intact at `.codex/artifacts/specs/2026-07-12-ycinstrument-pricing-performance-design.md`; removed from the current-state published set. | -| `docs/superpowers/specs/2026-07-12-zero-rate-parameterization-design.md` | `moved-to-.codex` | Task 1 hash comparison and 100% rename | Preserved intact at `.codex/artifacts/specs/2026-07-12-zero-rate-parameterization-design.md`; removed from the current-state published set. | +| `CHANGELOG.md` | `changed` | `dal-cpp/dal/curve/xccypricing.cpp`; `dal-cpp/dal/curve/xccycalibration.cpp`; `dal-cpp/dal/curve/xccyjointcalibration.cpp`; `dal-cpp/dal/auto/MG_XccyNotionalMode_enum.hpp`; `dal-python/src/bindings/curve.cpp`; `dal-excel/src/__xccycalibration.cpp` | Current capability baseline now covers fixed, resettable, and MTM XCCY, immutable snapshots, staged/joint calibration, and the exact per-surface matrix split; dated history remains qualified. | +| `CONTRIBUTING.md` | `verified-current` | `build_linux.sh`; `CMakePresets.json`; `dal-cpp/tests/curve/test_calibration.cpp`; `dal-public/tests/test_public_api.cpp`; `.github/scripts/check_benchmark_regressions.py` | Build, generation, targeted-test, and paired benchmark instructions match the current developer workflow. | +| `README.md` | `changed` | `dal-cpp/examples/xccy_curve_calibration/xccy_curve_calibration.cpp`; `dal-cpp/examples/xccy_reset_pricing/xccy_reset_pricing.cpp`; `dal-cpp/examples/xccy_mtm_calibration/xccy_mtm_calibration.cpp`; `dal-python/examples/007.xccy_joint_calibration.py`; `docs/methodology/xccy_calibration.md` | The overview, example navigation, and methodology shortlist now make cross-currency pricing and calibration discoverable without duplicating the detailed contract. | +| `dal-cpp/README.md` | `verified-current` | `dal-cpp/CMakeLists.txt`; `dal-cpp/cmake/Platform.cmake`; `dal-cpp/dal/concurrency/threadpool.cpp`; `dal-cpp/tests/concurrency/test_threadpool.cpp` | Target ownership, lazy/configurable concurrency, build options, and generated-code workflow match the core implementation. | +| `dal-excel/README.md` | `changed` | `dal-excel/src/__curveprotocol.cpp`; `dal-excel/src/__xccycalibration.cpp`; `dal-excel/auto/MG_JointXccyCalibrationResult_Get_public.inc`; `dal-excel/tests/test_excel_api.cpp` | Snapshot duplicate/reciprocal rules and staged-versus-joint worksheet matrix visibility now match the registered Excel surface. | +| `dal-public/README.md` | `verified-current` | `dal-public/CMakeLists.txt`; `dal-public/src/curvedata.hpp`; `dal-public/src/curvespec.hpp`; `dal-public/src/xccycalibration.hpp`; `dal-public/tests/test_public_api.cpp`; `tests/installed-consumer/CMakeLists.txt` | The facade, compatibility scope, installed target, header families, and tests are described accurately. | +| `dal-python/README.md` | `changed` | `dal-python/src/bindings/curve.cpp`; `dal-python/tests/test_curve_calibration.py`; `dal-python/tests/test_xccy_calibration.py`; `dal-python/tests/test_xccy_resettable.py`; `dal-python/tests/test_xccy_joint.py`; `dal-python/examples/007.xccy_joint_calibration.py` | Staged diagnostics-only and joint options/matrix availability are separated, inverse scaling is warned, and the test inventory includes curve and XCCY coverage. | +| `dal-web/README.md` | `verified-current` | `dal-web/backend/app/native_runtime.py`; `dal-web/backend/app/services/dal_gateway.py`; `dal-web/backend/app/services/valuation.py`; `dal-python/src/bindings/value.cpp`; `dal-web/scripts/start.sh`; `dal-web/scripts/start.ps1` | Native-only preflight, staged runtime, async thread handoff, GIL release, gateway serialization, and matrix support match the current web implementation. | +| `docs/README.md` | `changed` | `dal-cpp/dal/curve/curveparameterization.cpp`; `dal-cpp/dal/curve/calibration.cpp`; `dal-cpp/dal/curve/jointcalibration.cpp`; `docs/methodology/xccy_calibration.md`; `docs/methodology/yield_curve_jacobian.md` | Heading hierarchy is corrected, LOG_DISCOUNT exclusivity is removed, and XCCY snapshot/range/inverse topics are indexed. | +| `docs/architecture.md` | `changed` | `dal-cpp/dal/storage/globals.cpp`; `dal-cpp/dal/indice/fixingsnapshot.cpp`; `dal-cpp/dal/curve/calibration.cpp`; `dal-cpp/dal/curve/xccycalibration.cpp`; `dal-cpp/dal/curve/xccyjointcalibration.cpp` | Mutable process state is distinguished from immutable operation snapshots, and calibration architecture now branches by implemented workflow. | +| `docs/experimental/aad-analytic-jacobian-curve-calibration.md` | `verified-current` | `dal-cpp/dal/curve/calibration.cpp`; `dal-cpp/dal/curve/jointcalibration.cpp`; `dal-cpp/dal/curve/aadjacobian.cpp`; `dal-cpp/tests/curve/test_analytic_jacobian.cpp`; `dal-cpp/tests/curve/test_joint_analytic_jacobian.cpp` | The former rollout note is a concise redirect to the current normative methodology and no longer labels shipped behavior experimental. | +| `docs/experimental/replicate-ptirds-single-currency-curve.md` | `changed` | `dal-cpp/tests/curve/test_ptirds_curve.cpp`; `dal-cpp/dal/time/holidays.cpp` (`Holidays::IsBusinessDay`, `Holidays::Adjust`); `dal-cpp/dal/curve/calibration.cpp` | The every-day target is now distinguished from `Holidays::None()` plus `Unadjusted` DAL conventions, and comparison language is neutral. | +| `docs/installation.md` | `verified-current` | `build_linux.sh`; `CMakePresets.json`; `dal-cpp/CMakeLists.txt`; `dal-public/CMakeLists.txt`; `tests/installed-consumer/CMakeLists.txt`; `dal-cpp/cmake/Platform.cmake`; `dal-web/backend/app/native_runtime.py` | Core/full/distribution builds, staged installs, downstream consumption, CPU tuning, tests, and native web startup are current. | +| `docs/methodology/aad.md` | `changed` | `dal-cpp/dal/math/aad/tape.hpp` (`Rewind`, `RewindToMark`); `dal-cpp/dal/math/aad/tape.cpp`; `dal-cpp/dal/script/simulation.hpp`; `dal-cpp/dal/curve/aadjacobian.cpp` | Full-recording rewind and per-path checkpoint rewind are now named and timed according to the tape implementation. | +| `docs/methodology/black_scholes.md` | `verified-current` | `dal-cpp/dal/math/distribution/black.hpp`; `dal-cpp/dal/math/distribution/black.cpp`; `dal-cpp/dal/math/analytics/vanilla.hpp`; `dal-cpp/tests/math/distribution/test_black.cpp` (`TestBachelierRealForwardStrikeRoundTrips`) | Formula, units, intrinsic handling, and Bachelier real-forward/strike support match the corrected implementation. | +| `docs/methodology/dupire.md` | `verified-current` | `dal-cpp/dal/model/ivs.hpp` (`IVS_::Call`, `IVS_::LocalVol`); `dal-cpp/tests/math/aad/models/test_ivs.cpp`; `dal-cpp/tests/math/aad/models/test_dupire.cpp`; `dal-cpp/tests/model/test_dupire.cpp` | Discounting, carry, strike derivative, grid layout, and tail behavior match the current rate-aware implementation. | +| `docs/methodology/interpolation.md` | `changed` | `dal-cpp/dal/math/interp/interpcubic.cpp` (`Cubic1_::operator()`); `dal-cpp/dal/math/interp/interpweights.cpp` (`NaturalCubicWeightGeometry_::At`); `dal-cpp/dal/math/interp/interpmixed.cpp`; `dal-cpp/tests/math/interp/test_interpmixed.cpp`; `dal-cpp/tests/math/interp/test_interpweights.cpp` | Linear error order, cubic curvature factor, endpoint third-derivative semantics, and linear-head/cubic-tail orientation are accurate. | +| `docs/methodology/log_discount_curve.md` | `verified-current` | `dal-cpp/dal/curve/logdfinterp.cpp`; `dal-cpp/dal/curve/yclogdf.cpp`; `dal-cpp/dal/curve/curveparameterization.cpp`; `dal-cpp/dal/curve/yczerorate.hpp`; `dal-cpp/tests/curve/test_logdfinterp.cpp`; `dal-cpp/tests/curve/test_curveparameterization.cpp` | Anchor layout, interpolation schemes, extrapolation, AAD propagation, persistence, and zero-rate mapping remain current. | +| `docs/methodology/matrix.md` | `changed` | `dal-cpp/dal/math/matrix/banded.cpp` (`TridagBetaInverse`, `TriSolve`, `BandedCholesky_`); `dal-cpp/dal/math/matrix/cholesky.cpp` (`CholeskyImpl`); `dal-cpp/tests/math/matrix/test_banded.cpp`; `dal-cpp/tests/math/matrix/test_bcg.cpp` | Thomas notation, factorization/solve complexity, strict-dominance qualification, and dense regularization now describe the algorithms exactly. | +| `docs/methodology/pde.md` | `verified-current` | `dal-cpp/dal/math/pde/pdegrid.cpp`; `dal-cpp/dal/math/pde/pdeoperators.cpp`; `dal-cpp/dal/math/pde/thetascheme.cpp`; `dal-cpp/tests/math/pde/test_pdegrid.cpp`; `dal-cpp/tests/math/pde/test_pdeoperators.cpp`; `dal-cpp/tests/math/pde/test_thetascheme.cpp`; `dal-cpp/examples/european_fd/european_fd.cpp` | Coordinate maps, nonuniform operators, prepared-state contract, boundary ownership, and example flow match the implementation. | +| `docs/methodology/quadrature.md` | `verified-current` | `dal-cpp/dal/math/integral/quadrature.cpp`; `dal-cpp/dal/math/integral/quadrature.hpp`; `dal-cpp/tests/math/integral/test_quadrature.cpp` (`TestHermiteThreeNodesIntegratesFourthMoment`, `TestSimpsonFourthOrderMeshRefinement`) | Gauss-Hermite exactness, Simpson point semantics, and fourth-order convergence match source and independent tests. | +| `docs/methodology/random.md` | `changed` | `dal-cpp/dal/math/random/base.hpp`; `dal-cpp/dal/math/random/sobol.cpp`; `dal-cpp/dal/math/random/pseudorandom.cpp` (`PseudoRandom_::FillNormal`, `MRG32k32a_::SkipTo`, `ShuffledIRN_::SkipTo`); `dal-cpp/dal/script/simulation.hpp`; `dal-cpp/tests/math/random/test_sobol.cpp`; `dal-cpp/tests/math/random/test_pseudorandom.cpp` | Seeking is implementation-specific; IRN clone/branch behavior and the bounded MRG32 uniform replay contract are exact, while normal-path replay equivalence is explicitly not promised. | +| `docs/methodology/script_engine.md` | `verified-current` | `dal-cpp/dal/script/event.cpp`; `dal-cpp/dal/script/parser.cpp`; `dal-cpp/dal/script/preprocessor.hpp`; `dal-cpp/dal/script/simulation.hpp`; `dal-cpp/dal/script/visitor/evaluator.hpp`; `dal-cpp/dal/script/visitor/compiler.hpp`; `dal-cpp/dal/script/visitor/fuzzy.hpp` | Language phases, eager booleans, time split, compilation, batching, RNG selection, and AAD execution remain source-accurate. | +| `docs/methodology/underdetermined_search.md` | `verified-current` | `dal-cpp/dal/math/optimization/underdetermined.cpp` (`BacktrackMinimum`); `dal-cpp/dal/math/optimization/underdetermined.hpp`; `dal-cpp/tests/math/optimization/test_underdetermined.cpp` (`TestFindUsesQuadraticBacktrackMinimum`) | Scaling, minimum-change solve, corrected quadratic minimizer, diagnostics, and controls match the optimizer. | +| `docs/methodology/xccy_calibration.md` | `changed` | `dal-cpp/dal/indice/fixingsnapshot.cpp`; `dal-cpp/dal/curve/xccypricing.cpp`; `dal-cpp/dal/curve/xccycalibration.hpp`; `dal-cpp/dal/curve/xccyjointcalibration.hpp`; `dal-cpp/tests/curve/test_xccybasisjacobian.cpp`; `dal-cpp/tests/curve/test_xccyjointjacobian.cpp`; `dal-python/src/bindings/curve.cpp`; `dal-excel/src/__xccycalibration.cpp`; `dal-cpp/benchmarks/xccy_perf/xccy_perf.cpp` | Canonical/reverse FX rules, immutable dependency closure, staged/joint matrix semantics and availability, four examples, and the 24-row smoke surface are complete. | +| `docs/methodology/yield_curve.md` | `verified-current` | `dal-cpp/dal/curve/curveparameterization.hpp`; `dal-cpp/dal/curve/calibration.hpp`; `dal-cpp/dal/curve/jointcalibration.hpp`; `dal-cpp/tests/curve/test_analytic_jacobian.cpp`; `dal-cpp/tests/curve/test_inverse_jacobian_risk.cpp`; `dal-cpp/examples/curve_calibration/curve_calibration.cpp`; `dal-cpp/examples/joint_multi_curve_calibration/joint_multi_curve_calibration.cpp` | Representation layouts, single/staged/generic-joint solves, Jacobian population, risk scaling, and examples remain current. | +| `docs/methodology/yield_curve_jacobian.md` | `changed` | `dal-cpp/dal/curve/xccyjointcalibration.hpp`; `dal-cpp/dal/curve/xccyjointcalibration.cpp`; `dal-cpp/dal/math/optimization/underdetermined.cpp`; `dal-cpp/tests/curve/test_xccyjointjacobian.cpp`; `dal-python/src/bindings/curve.cpp`; `dal-excel/src/__xccycalibration.cpp` | Joint range placement, complementary dimensions, accepted-solution construction, tolerance scaling, population modes, and binding availability are explicit. | +| `docs/public-api.md` | `changed` | `dal-cpp/dal/auto/MG_XccyNotionalMode_enum.hpp`; `dal-cpp/dal/curve/xccycalibration.hpp`; `dal-cpp/dal/curve/xccyjointcalibration.hpp`; `dal-public/src/xccycalibration.hpp`; `dal-public/tests/test_xccy_calibration.cpp`; `dal-python/src/bindings/curve.cpp`; `dal-excel/src/__xccycalibration.cpp` | C++ constants are copy-safe, core versus facade staged entry points are distinguished, and staged/joint matrices are documented per surface without inventing getters. | +| `docs/superpowers/plans/2026-07-12-unified-yield-curve-interpolation-aad.md` | `moved-to-.codex` | `.codex/artifacts/plans/2026-07-12-unified-yield-curve-interpolation-aad.md`; Task 1 commit `ca147d8f` records an `R100` move | Preserved intact at `.codex/artifacts/plans/2026-07-12-unified-yield-curve-interpolation-aad.md`; removed from the current-state published set. | +| `docs/superpowers/plans/2026-07-12-ycinstrument-pricing-performance.md` | `moved-to-.codex` | `.codex/artifacts/plans/2026-07-12-ycinstrument-pricing-performance.md`; Task 1 commit `ca147d8f` records an `R100` move | Preserved intact at `.codex/artifacts/plans/2026-07-12-ycinstrument-pricing-performance.md`; removed from the current-state published set. | +| `docs/superpowers/plans/2026-07-12-zero-rate-parameterization.md` | `moved-to-.codex` | `.codex/artifacts/plans/2026-07-12-zero-rate-parameterization.md`; Task 1 commit `ca147d8f` records an `R100` move | Preserved intact at `.codex/artifacts/plans/2026-07-12-zero-rate-parameterization.md`; removed from the current-state published set. | +| `docs/superpowers/specs/2026-07-12-ycinstrument-pricing-performance-design.md` | `moved-to-.codex` | `.codex/artifacts/specs/2026-07-12-ycinstrument-pricing-performance-design.md`; Task 1 commit `ca147d8f` records an `R100` move | Preserved intact at `.codex/artifacts/specs/2026-07-12-ycinstrument-pricing-performance-design.md`; removed from the current-state published set. | +| `docs/superpowers/specs/2026-07-12-zero-rate-parameterization-design.md` | `moved-to-.codex` | `.codex/artifacts/specs/2026-07-12-zero-rate-parameterization-design.md`; Task 1 commit `ca147d8f` records an `R100` move | Preserved intact at `.codex/artifacts/specs/2026-07-12-zero-rate-parameterization-design.md`; removed from the current-state published set. | ## Prior Audit Reconciliation From 31fa8efcf0fbdb5b1c5f0ae245f4483c683ffe63 Mon Sep 17 00:00:00 2001 From: wegamekinglc Date: Wed, 15 Jul 2026 15:43:54 +0800 Subject: [PATCH 13/19] docs: correct reconciliation verification commands --- .../2026-07-15-repository-documentation-reconciliation.md | 4 ++-- .../reviews/repository-documentation-reconciliation.md | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/.codex/artifacts/plans/2026-07-15-repository-documentation-reconciliation.md b/.codex/artifacts/plans/2026-07-15-repository-documentation-reconciliation.md index 35aa8ba8c..bbd53a66f 100644 --- a/.codex/artifacts/plans/2026-07-15-repository-documentation-reconciliation.md +++ b/.codex/artifacts/plans/2026-07-15-repository-documentation-reconciliation.md @@ -319,7 +319,7 @@ build/Release-linux/dal-cpp/examples/xccy_curve_calibration/xccy_curve_calibrati build/Release-linux/dal-cpp/examples/xccy_reset_pricing/xccy_reset_pricing build/Release-linux/dal-cpp/examples/xccy_mtm_calibration/xccy_mtm_calibration PYTHONPATH="$PWD/build/stage/Release-linux" dal-python/.venv/bin/python dal-python/examples/007.xccy_joint_calibration.py -DAL_NUM_THREADS=4 build/Release-linux/dal-cpp/benchmarks/xccy_perf/xccy_perf | python3 -c 'import re,sys; row=re.compile(r"^\\S(?:.*?\\S)?\\s+[0-9]+(?:\\.[0-9]+)?\\s+(?:ns|us|ms|s)\\s+[0-9]+(?:\\.[0-9]+)?\\s+(?:ns|us|ms|s)\\s+[0-9]+(?:\\.[0-9]+)?\\s+(?:ns|us|ms|s)\\s+[0-9]+\\s*$"); names=[line.rsplit(None,7)[0] for line in sys.stdin if row.match(line)]; assert len(names)==len(set(names))==24, (len(names),len(set(names)))' +DAL_NUM_THREADS=4 build/Release-linux/dal-cpp/benchmarks/xccy_perf/xccy_perf | python3 -c 'import re,sys; row=re.compile(r"^\S(?:.*?\S)?\s+[0-9]+(?:\.[0-9]+)?\s+(?:ns|us|ms|s)\s+[0-9]+(?:\.[0-9]+)?\s+(?:ns|us|ms|s)\s+[0-9]+(?:\.[0-9]+)?\s+(?:ns|us|ms|s)\s+[0-9]+\s*$"); names=[line.rsplit(None,7)[0] for line in sys.stdin if row.match(line)]; assert len(names)==len(set(names))==24, (len(names),len(set(names)))' python3 .github/scripts/check_docs.py git diff --check git add docs/methodology/xccy_calibration.md docs/methodology/yield_curve_jacobian.md @@ -497,7 +497,7 @@ Expected: exit 0 with 33 unique rows. ```bash python3 .github/scripts/check_docs.py git diff --check 1589089b..HEAD -git diff --name-only 1589089b..HEAD | while IFS= read -r path; do case "$path" in *.md) ;; *) printf '%s\n' "$path";; esac; done +git diff --name-only 1589089b..HEAD | while IFS= read -r doc_path; do case "$doc_path" in *.md) ;; *) printf '%s\n' "$doc_path";; esac; done git diff --name-only 1589089b..HEAD -- CLAUDE.md .claude AGENTS.md dal-cpp/dal/auto dal-excel/auto ``` diff --git a/.codex/artifacts/reviews/repository-documentation-reconciliation.md b/.codex/artifacts/reviews/repository-documentation-reconciliation.md index 796f9e5b5..7bd92136b 100644 --- a/.codex/artifacts/reviews/repository-documentation-reconciliation.md +++ b/.codex/artifacts/reviews/repository-documentation-reconciliation.md @@ -71,6 +71,6 @@ checks alone are not treated as proof of technical accuracy. | Branch whitespace and scope | `git diff --check 1589089b..HEAD` passed; the complete branch diff contains only Markdown, and protected `CLAUDE.md`, `.claude/`, `AGENTS.md`, `dal-cpp/dal/auto`, and `dal-excel/auto` paths are unchanged. | | C++ XCCY examples | `xccy_curve_calibration`, `xccy_reset_pricing`, and `xccy_mtm_calibration` each exited 0; outputs included the staged 15-instrument fit, four pricing cases, and the converged `25x25` joint solve with five named ranges. | | Python XCCY example | Installed-surface `dal-python/examples/007.xccy_joint_calibration.py` exited 0 with a converged `3x3` solve, named ranges, and FX forwards. | -| XCCY benchmark smoke | The executable output parsed with the corrected single-backslash whitespace regex to exactly 24 unique timing rows. The doubled raw-regex backslashes in the written task command are not executable evidence. | +| XCCY benchmark smoke | The executable output parsed to exactly 24 unique timing rows with the verification command retained in the reconciliation plan. | | Task reviews | Task 1 and Task 2 were accepted clean; Tasks 3, 4, and 5 were accepted after scoped follow-up commits and re-review. The accepted head before this artifact is `ab4ca651`. | | Whole-branch review | Not yet performed. A fresh final reviewer is intentionally scheduled after this audit commit. | From 8c3c5e5171d0f535e9d6796eb3f8ee2b6697a6f4 Mon Sep 17 00:00:00 2001 From: wegamekinglc Date: Wed, 15 Jul 2026 15:56:54 +0800 Subject: [PATCH 14/19] docs: correct calibration and matrix summaries --- .../plans/2026-07-15-repository-documentation-reconciliation.md | 2 +- docs/README.md | 2 +- docs/architecture.md | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/.codex/artifacts/plans/2026-07-15-repository-documentation-reconciliation.md b/.codex/artifacts/plans/2026-07-15-repository-documentation-reconciliation.md index bbd53a66f..2070a67bd 100644 --- a/.codex/artifacts/plans/2026-07-15-repository-documentation-reconciliation.md +++ b/.codex/artifacts/plans/2026-07-15-repository-documentation-reconciliation.md @@ -157,7 +157,7 @@ Replace the single calibration diagram with one common validation/residual prefi ```text instrument/convention builders -> validated spec, curve layout, and model-rate residuals - -> single/staged curves: CalibrateCurve / CalibrateMultiCurve + -> single/staged curves: CalibrateYieldCurve / CalibrateMultiCurve -> staged XCCY basis: CalibrateCrossCurrencyMarket -> supplied domestic/foreign blocks + basis parameters + one fixing snapshot -> joint XCCY: CalibrateJointXccyMarket diff --git a/docs/README.md b/docs/README.md index 34d45a7d1..052918621 100644 --- a/docs/README.md +++ b/docs/README.md @@ -75,7 +75,7 @@ Deep dives into the quantitative methods and algorithms implemented in DAL: - **[matrix.md](methodology/matrix.md)** — Matrix and Linear Algebra - Numerical-Recipes band-storage layout and the `Sparse::Square_` / decomposition interfaces - Tri-diagonal Thomas-algorithm solve (`TriDiagonal_`, `TriDecomp_`, `TriDecompSymm_`) - - Dense and band-Cholesky factorizations with diagonal regularization + - Dense Cholesky with reciprocal-diagonal regularization, plus band-Cholesky factorization - Preconditioned conjugate-gradient (CG) and bi-conjugate-gradient (BCG) Krylov solvers - **[log_discount_curve.md](methodology/log_discount_curve.md)** — Log-Discount Curve diff --git a/docs/architecture.md b/docs/architecture.md index ffca1e1f0..e6be4800a 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -164,7 +164,7 @@ for each worker's path batch. ```text instrument/convention builders -> validated spec, curve layout, and model-rate residuals - -> single/staged curves: CalibrateCurve / CalibrateMultiCurve + -> single/staged curves: CalibrateYieldCurve / CalibrateMultiCurve -> staged XCCY basis: CalibrateCrossCurrencyMarket -> supplied domestic/foreign blocks + basis parameters + one fixing snapshot -> joint XCCY: CalibrateJointXccyMarket From ab930cf0b8015807bd60f810e015e91ee1c26b3e Mon Sep 17 00:00:00 2001 From: wegamekinglc Date: Wed, 15 Jul 2026 16:08:52 +0800 Subject: [PATCH 15/19] docs: make reconciliation checks shell-safe --- .../2026-07-15-repository-documentation-reconciliation.md | 4 ++-- docs/methodology/interpolation.md | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/.codex/artifacts/plans/2026-07-15-repository-documentation-reconciliation.md b/.codex/artifacts/plans/2026-07-15-repository-documentation-reconciliation.md index 2070a67bd..d89b9409f 100644 --- a/.codex/artifacts/plans/2026-07-15-repository-documentation-reconciliation.md +++ b/.codex/artifacts/plans/2026-07-15-repository-documentation-reconciliation.md @@ -111,7 +111,7 @@ git commit -m "docs: move historical plans out of published docs" ```bash rg -n "XCCY|Cross-Currency" README.md -rg -n "Why `LOG_DISCOUNT` is the parameterization that supports the analytic Jacobian" docs/README.md +rg -n 'Why `LOG_DISCOUNT` is the parameterization that supports the analytic Jacobian' docs/README.md rg -n "MarketFixingSnapshot_|CalibrateJointXccyMarket|CalibrateCrossCurrencyMarket" docs/architecture.md ``` @@ -205,7 +205,7 @@ git commit -m "docs: refresh architecture and navigation" ```bash rg -n "all-days|matches calendar = all|does better" docs/experimental/replicate-ptirds-single-currency-curve.md rg -n "first-order accurate|not-a-knot|O\\(n m\\)" docs/methodology/interpolation.md docs/methodology/matrix.md -rg -n "full `SkipTo` support|re-seed via `Branch`/`Clone`|repositions the generator without replay" docs/methodology/random.md +rg -n 'full `SkipTo` support|re-seed via `Branch`/`Clone`|repositions the generator without replay' docs/methodology/random.md ``` Expected: each stale claim is present before editing. diff --git a/docs/methodology/interpolation.md b/docs/methodology/interpolation.md index 5cf8772c3..cd830dbab 100644 --- a/docs/methodology/interpolation.md +++ b/docs/methodology/interpolation.md @@ -91,7 +91,7 @@ default for the `LOG_DISCOUNT` curve parameterization — see ## Cubic Spline -A natural cubic spline through the knots: a piecewise-cubic polynomial on each +A cubic spline through the knots: a piecewise-cubic polynomial on each $[x_i, x_{i+1}]$ that is $C^2$ continuous, with second derivatives $f''_i$ at the knots solved once at construction by the standard tri-diagonal elimination (the routine is based on the *Numerical Recipes* `spline`/`splint` pair). Evaluation between knots uses the local From 84f7dccc643320df6170625c32ac6cbf723304aa Mon Sep 17 00:00:00 2001 From: wegamekinglc Date: Thu, 16 Jul 2026 02:00:50 +0800 Subject: [PATCH 16/19] docs: record final documentation review --- .../reviews/repository-documentation-reconciliation.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.codex/artifacts/reviews/repository-documentation-reconciliation.md b/.codex/artifacts/reviews/repository-documentation-reconciliation.md index 7bd92136b..686cb0b0a 100644 --- a/.codex/artifacts/reviews/repository-documentation-reconciliation.md +++ b/.codex/artifacts/reviews/repository-documentation-reconciliation.md @@ -73,4 +73,4 @@ checks alone are not treated as proof of technical accuracy. | Python XCCY example | Installed-surface `dal-python/examples/007.xccy_joint_calibration.py` exited 0 with a converged `3x3` solve, named ranges, and FX forwards. | | XCCY benchmark smoke | The executable output parsed to exactly 24 unique timing rows with the verification command retained in the reconciliation plan. | | Task reviews | Task 1 and Task 2 were accepted clean; Tasks 3, 4, and 5 were accepted after scoped follow-up commits and re-review. The accepted head before this artifact is `ab4ca651`. | -| Whole-branch review | Not yet performed. A fresh final reviewer is intentionally scheduled after this audit commit. | +| Whole-branch review | A fresh reviewer approved the full `1589089b..ab930cf0` range with no Critical, Important, or Minor findings after all review-driven corrections. | From 0e3c009cc48c834238765649f3e1647d19bb02ad Mon Sep 17 00:00:00 2001 From: wegamekinglc Date: Thu, 16 Jul 2026 02:26:07 +0800 Subject: [PATCH 17/19] docs: address PR documentation review --- ...repository-documentation-reconciliation.md | 21 +++++++++------- ...repository-documentation-reconciliation.md | 24 ++++++++++++------- docs/methodology/interpolation.md | 11 +++++---- docs/methodology/matrix.md | 5 ++++ docs/methodology/random.md | 12 ++++++---- docs/methodology/xccy_calibration.md | 13 ++++++---- 6 files changed, 55 insertions(+), 31 deletions(-) diff --git a/.codex/artifacts/plans/2026-07-15-repository-documentation-reconciliation.md b/.codex/artifacts/plans/2026-07-15-repository-documentation-reconciliation.md index d89b9409f..e79b65ddd 100644 --- a/.codex/artifacts/plans/2026-07-15-repository-documentation-reconciliation.md +++ b/.codex/artifacts/plans/2026-07-15-repository-documentation-reconciliation.md @@ -298,15 +298,18 @@ State staged shapes as `nInstruments x nBasisParameters` and `nBasisParameters x State: exact analytic may produce both matrices; exact bumped produces only the effective inverse; approximate produces neither; options suppress each independently. Both matrices live on the top-level joint result, not per-group diagnostics. -- [ ] **Step 4: Add a compact surface-availability table** - -Use these exact rows: - -| Surface | Staged XCCY | Joint XCCY | -|---------|-------------|------------| -| Core/public C++ | Full options, forward Jacobian, and effective inverse | Full options and both top-level matrices | -| Python | Default staged solve with market/FX-forward and fit diagnostics; no staged matrix bindings | Options, named ranges, forward Jacobian, and effective inverse | -| Excel | Basis handle and fit diagnostics; no staged matrix views | Options plus forward-Jacobian/range views; no effective-inverse worksheet getter | +- [ ] **Step 4: Add a compact surface-availability summary** + +Use these exact bullet entries: + +- **Core/public C++:** staged XCCY has full options, a forward Jacobian, and an + effective inverse; joint XCCY has full options and both top-level matrices. +- **Python:** staged XCCY has the default solve, market/FX-forward output, and + fit diagnostics, but no staged matrix bindings; joint XCCY has options, named + ranges, a forward Jacobian, and an effective inverse. +- **Excel:** staged XCCY has a basis handle and fit diagnostics, but no staged + matrix views; joint XCCY has options plus forward-Jacobian/range views, but no + effective-inverse worksheet getter. - [ ] **Step 5: Complete examples and performance smoke navigation** diff --git a/.codex/artifacts/specs/repository-documentation-reconciliation.md b/.codex/artifacts/specs/repository-documentation-reconciliation.md index ad0c7edfe..fa8fb35bb 100644 --- a/.codex/artifacts/specs/repository-documentation-reconciliation.md +++ b/.codex/artifacts/specs/repository-documentation-reconciliation.md @@ -109,14 +109,22 @@ of the methodology documents. ## Inputs and Outputs -| Input | Authority | Output | -|------------------------|------------------------------------------------------------------------|---------------------------------------------------------------------------| -| Published Markdown set | `.github/scripts/check_docs.py::DOCS` | One 33-row disposition matrix | -| C++ surface | Core/public headers and generated enums | Exact types, constants, ownership, and solver contracts | -| Python surface | `dal-python/src/bindings/`, `dal-python/src/dal/`, tests, examples | Callable names, result properties, and supported workflows | -| Excel surface | `dal-excel/src/`, interface markup, generated help, smoke tests | Worksheet functions, settings, selectors, and documented limitations | -| Build/runtime surface | CMake, presets, scripts, workflows, web source | Current setup, ownership, execution, and CI descriptions | -| Numerical behavior | Methodology implementation plus independent tests | Correct formulas, matrix semantics, tolerances, and applicability | +- **Published Markdown set:** `.github/scripts/check_docs.py::DOCS` is the + authority; the output is one 33-row disposition matrix. +- **C++ surface:** core/public headers and generated enums are the authority; + the outputs are exact types, constants, ownership, and solver contracts. +- **Python surface:** `dal-python/src/bindings/`, `dal-python/src/dal/`, tests, + and examples are the authorities; the outputs are callable names, result + properties, and supported workflows. +- **Excel surface:** `dal-excel/src/`, interface markup, generated help, and + smoke tests are the authorities; the outputs are worksheet functions, + settings, selectors, and documented limitations. +- **Build/runtime surface:** CMake, presets, scripts, workflows, and web source + are the authorities; the outputs are current setup, ownership, execution, + and CI descriptions. +- **Numerical behavior:** methodology implementations and independent tests are + the authorities; the outputs are correct formulas, matrix semantics, + tolerances, and applicability. ## Acceptance Criteria diff --git a/docs/methodology/interpolation.md b/docs/methodology/interpolation.md index cd830dbab..a5f430212 100644 --- a/docs/methodology/interpolation.md +++ b/docs/methodology/interpolation.md @@ -106,11 +106,12 @@ with $h = x_{i+1}-x_i$, $b = (x-x_i)/h$, $a = 1-b$. The two end conditions are supplied as a `Boundary_(order, value)` pair, where `order` selects which derivative is pinned at the boundary: -| `order_` | Boundary condition | -|----------|------------------------------------------------------------------------------------------| -| 1 | `lhs` pins $f'(x_1)$; `rhs` pins $f'(x_N)$ to its respective `value_` | -| 2 | `lhs` pins $f''(x_1)$; `rhs` pins $f''(x_N)$ to its respective `value_` | -| 3 | each endpoint segment's third derivative is pinned to the respective `lhs`/`rhs` `value_` | +- **Order 1:** `lhs` pins $f'(x_1)$ and `rhs` pins $f'(x_N)$ to its respective + `value_`. +- **Order 2:** `lhs` pins $f''(x_1)$ and `rhs` pins $f''(x_N)$ to its respective + `value_`. +- **Order 3:** each endpoint segment's third derivative is pinned to the + respective `lhs`/`rhs` `value_`. `Boundary_(2, 0.0)` on both ends gives the classic natural spline (zero end curvature). diff --git a/docs/methodology/matrix.md b/docs/methodology/matrix.md index aac2137b6..3f8f2ff11 100644 --- a/docs/methodology/matrix.md +++ b/docs/methodology/matrix.md @@ -90,6 +90,11 @@ x_i=\frac{b_i-c_{i-1}x_{i-1}}{\beta_i}, x_{i-1}\leftarrow x_{i-1}-\frac{a_{i-1}}{\beta_{i-1}}x_i $$ +These are the `SolveLeft` recurrences for $A x=b$. `TriDecomp_::XSolveLeft_af` +passes the stored `below_` diagonal first and `above_` second to the internal +`TriSolve`, so forward substitution uses $c$ and backward substitution uses +$a$. `SolveRight` reverses those diagonals to solve against $A^{\top}$. + This is the Thomas algorithm. It is $O(n)$ in time and $O(n)$ in memory. Pivoting is not used: the factorization is valid only when every $\beta_i$ is non-zero, which holds in particular for **strictly diagonally dominant** and for **symmetric diff --git a/docs/methodology/random.md b/docs/methodology/random.md index 7de43f0de..75b4ae143 100644 --- a/docs/methodology/random.md +++ b/docs/methodology/random.md @@ -235,10 +235,14 @@ The pseudo-random family (`PseudoRandom_` in `dal-cpp/dal/math/random/pseudorandom.cpp`) is selected through the `RNGType_` enum, whose alternatives are `IRN` and `MRG32`. -| `RNGType_` | Implementation | Period / structure | `SkipTo` | -|------------|-----------------------|----------------------------------------------|-----------------------------------------------------| -| `IRN` | `ShuffledIRN_<55,31,128>` | Knuth-style lagged additive (IRN55) with shuffling, modulus $2^{30}$ | No-op; `Clone` restarts from the original seed and `Branch` creates another seeded generator | -| `MRG32` | `MRG32k32a_` | L'Ecuyer combined multiple recursive (MRG32k32a), periods $\approx 2^{191}$ | Matrix jump; fresh-generator `FillUniform` replay at even path offsets only; no normal-path replay | +- **`IRN`:** implemented by `ShuffledIRN_<55,31,128>`, a Knuth-style lagged + additive IRN55 generator with shuffling and modulus $2^{30}$. `SkipTo` is a + no-op; `Clone` restarts from the original seed, and `Branch` creates another + seeded generator. +- **`MRG32`:** implemented by `MRG32k32a_`, L'Ecuyer's combined multiple + recursive MRG32k32a generator with period approximately $2^{191}$. `SkipTo` + uses a matrix jump; on a fresh generator it replays `FillUniform` only at even + path offsets, and it does not replay normal paths. `PseudoRandom_` adds antithetic variates on top of the underlying engine: `FillUniform` alternates between drawing $u_i$ and emitting the antithetic diff --git a/docs/methodology/xccy_calibration.md b/docs/methodology/xccy_calibration.md index 1f4994f89..5ef5722c1 100644 --- a/docs/methodology/xccy_calibration.md +++ b/docs/methodology/xccy_calibration.md @@ -189,11 +189,14 @@ valid residual system without the analytic eligibility requirement. ## Surface Availability -| Surface | Staged XCCY | Joint XCCY | -|---------|-------------|------------| -| Core/public C++ | Full options, forward Jacobian, and effective inverse | Full options and both top-level matrices | -| Python | Default staged solve with market/FX-forward and fit diagnostics; no staged matrix bindings | Options, named ranges, forward Jacobian, and effective inverse | -| Excel | Basis handle and fit diagnostics; no staged matrix views | Options plus forward-Jacobian/range views; no effective-inverse worksheet getter | +- **Core/public C++:** staged XCCY has full options, a forward Jacobian, and an + effective inverse; joint XCCY has full options and both top-level matrices. +- **Python:** staged XCCY has the default solve, market/FX-forward output, and + fit diagnostics, but no staged matrix bindings; joint XCCY has options, named + ranges, a forward Jacobian, and an effective inverse. +- **Excel:** staged XCCY has a basis handle and fit diagnostics, but no staged + matrix views; joint XCCY has options plus forward-Jacobian/range views, but no + effective-inverse worksheet getter. ## Runnable Examples From 4c7cc1974340b2f75b114e4489ae9e8c5d45ddc9 Mon Sep 17 00:00:00 2001 From: wegamekinglc Date: Thu, 16 Jul 2026 02:30:06 +0800 Subject: [PATCH 18/19] docs: record PR review follow-up --- .../reviews/repository-documentation-reconciliation.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.codex/artifacts/reviews/repository-documentation-reconciliation.md b/.codex/artifacts/reviews/repository-documentation-reconciliation.md index 686cb0b0a..30e4f5f46 100644 --- a/.codex/artifacts/reviews/repository-documentation-reconciliation.md +++ b/.codex/artifacts/reviews/repository-documentation-reconciliation.md @@ -73,4 +73,4 @@ checks alone are not treated as proof of technical accuracy. | Python XCCY example | Installed-surface `dal-python/examples/007.xccy_joint_calibration.py` exited 0 with a converged `3x3` solve, named ranges, and FX forwards. | | XCCY benchmark smoke | The executable output parsed to exactly 24 unique timing rows with the verification command retained in the reconciliation plan. | | Task reviews | Task 1 and Task 2 were accepted clean; Tasks 3, 4, and 5 were accepted after scoped follow-up commits and re-review. The accepted head before this artifact is `ab4ca651`. | -| Whole-branch review | A fresh reviewer approved the full `1589089b..ab930cf0` range with no Critical, Important, or Minor findings after all review-driven corrections. | +| Whole-branch review | A fresh reviewer approved `1589089b..ab930cf0` with no Critical, Important, or Minor findings; the audit closure and PR follow-up `84f7dccc..0e3c009c` were then independently approved without findings. | From 8d66d90cea8eeef5d7504aedf0a48723269058a3 Mon Sep 17 00:00:00 2001 From: wegamekinglc Date: Thu, 16 Jul 2026 03:23:31 +0800 Subject: [PATCH 19/19] docs: fix linear interp contract claims in interpolation.md The linear interpolator (dal-cpp/dal/math/interp/interplinear.cpp) does not enforce a minimum knot count -- unlike log-linear and cubic, which gate on size. It also checks monotonicity with std::less_equal, so it accepts equal (duplicate) abscissae. The prior text overstated both contracts: the Linear section claimed "Requires N >= 2" and the Common Interface stated universally that abscissae "must be strictly increasing". Drop the false minimum-size claim for linear and soften the shared monotonicity statement to non-decreasing, which is the true common floor (cubic and log-linear keep their stricter scheme-specific requirements). Co-Authored-By: Claude --- docs/methodology/interpolation.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/docs/methodology/interpolation.md b/docs/methodology/interpolation.md index a5f430212..a26fce9da 100644 --- a/docs/methodology/interpolation.md +++ b/docs/methodology/interpolation.md @@ -15,7 +15,7 @@ $$ \texttt{double operator()(double x) const}, \qquad \texttt{bool IsInBounds(double x) const}. $$ -The abscissae $x_1 < x_2 < \dots < x_N$ must be strictly increasing (`IsMonotonic` is +The abscissae $x_1 \le x_2 \le \dots \le x_N$ must be non-decreasing (`IsMonotonic` is checked at construction). Evaluation is by `LowerBound` on the knot vector followed by a scheme-specific local formula; values exactly at a knot return the knot's $f$ value without rounding error. The shared linear kernel used internally is @@ -70,7 +70,6 @@ $[x_1, x_N]$; outside that range the kernel clamps to the nearest endpoint value extrapolation via the `LowerBound` edge cases). Factory: `Interp::NewLinear(name, x, f)` (`dal-cpp/dal/math/interp/interplinear.hpp`). -Requires $N \ge 2$. ## Log-Linear