Skip to content

feat(ar): ConjugateAR node with structured MvNormalGamma (θ, γ) posterior#612

Merged
bvdmitri merged 11 commits into
mainfrom
add-ar-mvnormalgamma-rule
Jun 19, 2026
Merged

feat(ar): ConjugateAR node with structured MvNormalGamma (θ, γ) posterior#612
bvdmitri merged 11 commits into
mainfrom
add-ar-mvnormalgamma-rule

Conversation

@RetrospectiveRotations

Copy link
Copy Markdown
Contributor

Summary

Adds ConjugateAR, an autoregressive node that carries the AR coefficients θ and the
transition precision γ jointly on a single edge w as an MvNormalGamma, instead of the
mean-field q(θ)q(γ) split of the existing AR node. Because (θ, γ) are jointly conjugate
for the AR likelihood (a Bayesian linear regression with unknown noise precision), the
structured posterior q(θ, γ) is exactly MvNormalGamma — more accurate than mean-field and
still closed-form.

Also registers MvNormalGamma as a prior factor node so models can write
w ~ MvNormalGamma(μ, Λ, α, β).

⚠️ Dependency

This branch depends on ReactiveBayes/ExponentialFamily.jl#287 (the MvNormalGamma
distribution + prod rule). That PR must land first; this one should not be merged until the
ExponentialFamily release containing MvNormalGamma is available.

What's included

  • ConjugateAR node [y, x, w] reusing ARMeta.
  • ConjugateAR(:w) marginal rule — the structured BLR Normal-Gamma posterior, with
    expected sufficient statistics from q(y, x): C = E[xxᵀ], b = E[x·y₁], a = E[y₁²].
  • ConjugateAR(:w) message rule — the AR likelihood as a Normal-Gamma factor; its product
    with the prior at the equality node reproduces the :w marginal (natural parameters add).
  • ConjugateAR(:x), (:y), (:y_x) state/marginal rules and @average_energy,
    delegating to the tested AR rules via effective (q_θ, q_γ) derived from q(w) (no
    duplicated algebra).
  • MvNormalGamma prior node [out, μ, Λ, α, β] with its :out message rule and
    @average_energy (so free_energy = true works for ConjugateAR models).

Math references

All formulas were cross-checked against these references and for internal consistency
(message ↔ marginal ↔ prod, entropy ↔ self cross-entropy, univariate ↔ NormalGamma).

Test plan

  • ConjugateAR(:w) message params + prod(prior, message) == :w marginal
  • (:x), (:y), (:y_x) delegation matches AR with effective (q_θ, q_γ)
  • BLR reference (orders 1–3), univariate NormalGamma reduction, properness/PSD
  • MvNormalGamma(:out) pass-through; average energy (self cross-entropy == entropy)
  • End-to-end RxInfer AR(2): recovers transition precision (0.99 vs 1.0) and coefficients
  • CI green on this branch

Spec and derivation for the structured q(θ,γ) = MvNormalGamma update on the
Autoregressive node, replacing the current mean-field q(θ)q(γ) split.

- Derives the AR(:θ_γ) marginal rule from the Bayesian linear regression
  Normal-Gamma posterior, with expected sufficient statistics from q(y,x):
  C = E[xxᵀ], b = E[x·y₁], a = E[y₁²].
- Shows equivalence to prod(MvNormalGamma, MvNormalGamma) and reduction to
  scalar NormalGamma at order 1.
- Specifies supporting x/y/average-energy rules, file wiring, and a TDD test
  matrix. Implementation gated on review (Section 8 open questions).

References: statproofbook.github.io/P/ng-pdf and /P/blr-post
… update

New ConjugateAR node [y, x, w] carrying joint (θ,γ) as MvNormalGamma, and the
@marginalrule ConjugateAR(:w) implementing the structured conjugate posterior
(Bayesian linear regression Normal-Gamma update). Marginal rule + tests only;
x/y/average-energy and model wiring deferred.

Verified against an independent BLR reference (orders 1-3), prod(prior, likelihood)
equivalence, the univariate NormalGamma reduction, and properness checks.
…ma prior node

Completes the ConjugateAR inference path:
- ConjugateAR(:w) message rule: the AR likelihood as a Normal-Gamma factor; its product
  with the prior at the equality node yields the MvNormalGamma posterior.
- ConjugateAR(:y), (:x) state messages and (:y_x) joint marginal, delegating to the tested
  AR rules via the effective (q_θ, q_γ) derived from q(w) (no duplicated algebra).
- @average_energy ConjugateAR (also delegated).
- MvNormalGamma registered as a prior node [out, μ, Λ, α, β] with its :out message rule.

Verified end-to-end in RxInfer: a latent AR(2) model with an MvNormalGamma prior on (θ, τ)
recovers the transition precision (0.99 vs 1.0) and the coefficients. Free energy for this
model is not yet available (the MvNormalGamma prior node has no average energy yet).
- ConjugateAR(:w) message: likelihood-factor parameters and prod(prior, message) == the
  :w marginal posterior.
- ConjugateAR(:y), (:x), (:y_x): delegation matches AR with the effective (q_θ, q_γ).
- MvNormalGamma(:out): hyperparameter pass-through (PointMass and q_ variants).
- conjugatear_effective_marginals (E[θ]=μ, E[γ]=α/β, mγ·Vθ=Λ⁻¹) and the average energy.
…gelog

- @average_energy MvNormalGamma: cross-entropy of the posterior marginal against the
  prior factor, so free_energy=true works for ConjugateAR models. Monte-Carlo validated.
- Unit test for the MvNormalGamma average energy (self cross-entropy == entropy).
- Apply JuliaFormatter to the ConjugateAR / MvNormalGamma sources and tests.
- CHANGELOG: document the ConjugateAR node and the MvNormalGamma factor node.
The derivation/spec doc was a pre-coding review gate; the node is now implemented
and the math lives in the source comments and the PR description.
@github-actions

Copy link
Copy Markdown
Contributor

🤖 Code Formatting

Your PR still has some code formatting issues. I've updated PR #613 with the necessary formatting changes.

You can merge that PR into this branch to fix the code style check.

Alternatively, you can run make format locally and push the changes yourself.

@RetrospectiveRotations
RetrospectiveRotations marked this pull request as ready for review June 17, 2026 11:09

@wmkouw wmkouw left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great. Go for it.

My only comment is that the conjugate_autoregressive/y.jl message can already be cast as a Tdist. I realized after our call that the reason we don't have T-distributions in ExponentialFamily.jl is because T-distributions are not part of the exponential family. They do exist in Distributions.jl (https://juliastats.org/Distributions.jl/stable/univariate/#Distributions.TDist; although it may not be full location-scale parameterization).

@RetrospectiveRotations

Copy link
Copy Markdown
Contributor Author

Looks great. Go for it.

My only comment is that the conjugate_autoregressive/y.jl message can already be cast as a Tdist. I realized after our call that the reason we don't have T-distributions in ExponentialFamily.jl is because T-distributions are not part of the exponential family. They do exist in Distributions.jl (https://juliastats.org/Distributions.jl/stable/univariate/#Distributions.TDist; although it may not be full location-scale parameterization).

I asked our mutual friend for the reason of the current implementation. Since TDist isn't exp-family it has no closed-form product with the downstream Gaussian message, so as an internal message we'd just project back to Gaussian, right? Do you want me to change this?

@wmkouw

wmkouw commented Jun 17, 2026

Copy link
Copy Markdown
Member

Well, the y message is for output predictions which tend not to be re-used. And there are multiple ways of doing the Gaussian approximation (e.g., mean-field VI for accurate mean, moment matching for accurate variance).

But this is fine for now. If we need more, we can always open up a new branch with some options under the ARmeta.

@github-actions

Copy link
Copy Markdown
Contributor

🤖 Code Formatting

Your PR still has some code formatting issues. I've updated PR #613 with the necessary formatting changes.

You can merge that PR into this branch to fix the code style check.

Alternatively, you can run make format locally and push the changes yourself.

@codecov

codecov Bot commented Jun 18, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 98.33333% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 82.88%. Comparing base (b410342) to head (9176063).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
src/nodes/predefined/conjugate_autoregressive.jl 88.88% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #612      +/-   ##
==========================================
+ Coverage   82.64%   82.88%   +0.24%     
==========================================
  Files         214      220       +6     
  Lines        6660     6720      +60     
==========================================
+ Hits         5504     5570      +66     
+ Misses       1156     1150       -6     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

github-actions Bot and others added 3 commits June 18, 2026 15:00
Co-authored-by: bvdmitri <6557701+bvdmitri@users.noreply.github.com>
Documenter's strict checkdocs flagged `ConjugateAR` and
`conjugatear_effective_marginals` docstrings as not included in the
manual. Add a dedicated `lib/nodes/conjugate_ar.md` page (modeled on
`ar.md`) with a @docs block for both symbols and register it in the nav.

Also convert the `[AR](@ref)`/`[ARMeta](@ref)` cross-references in the
ConjugateAR docstring to plain backticks (neither has a docstring, so the
links are unresolvable once the docstring is surfaced) and drop the
dangling reference to the removed `design/ar_mvnormalgamma_conjugate_rule.md`.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add docstrings to the `AR` node and `ARMeta` metadata and surface them via
a @docs block in `ar.md`. With both now documented, restore the
`[AR](@ref)`/`[ARMeta](@ref)` cross-references in the `ConjugateAR` docs
page and docstring that were temporarily downgraded to plain backticks.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@bvdmitri
bvdmitri merged commit 0726ca2 into main Jun 19, 2026
9 checks passed
@bvdmitri
bvdmitri deleted the add-ar-mvnormalgamma-rule branch June 19, 2026 06:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants