feat(ar): ConjugateAR node with structured MvNormalGamma (θ, γ) posterior#612
Conversation
Spec and derivation for the structured q(θ,γ) = MvNormalGamma update on the Autoregressive node, replacing the current mean-field q(θ)q(γ) split. - Derives the AR(:θ_γ) marginal rule from the Bayesian linear regression Normal-Gamma posterior, with expected sufficient statistics from q(y,x): C = E[xxᵀ], b = E[x·y₁], a = E[y₁²]. - Shows equivalence to prod(MvNormalGamma, MvNormalGamma) and reduction to scalar NormalGamma at order 1. - Specifies supporting x/y/average-energy rules, file wiring, and a TDD test matrix. Implementation gated on review (Section 8 open questions). References: statproofbook.github.io/P/ng-pdf and /P/blr-post
… update New ConjugateAR node [y, x, w] carrying joint (θ,γ) as MvNormalGamma, and the @marginalrule ConjugateAR(:w) implementing the structured conjugate posterior (Bayesian linear regression Normal-Gamma update). Marginal rule + tests only; x/y/average-energy and model wiring deferred. Verified against an independent BLR reference (orders 1-3), prod(prior, likelihood) equivalence, the univariate NormalGamma reduction, and properness checks.
…ma prior node Completes the ConjugateAR inference path: - ConjugateAR(:w) message rule: the AR likelihood as a Normal-Gamma factor; its product with the prior at the equality node yields the MvNormalGamma posterior. - ConjugateAR(:y), (:x) state messages and (:y_x) joint marginal, delegating to the tested AR rules via the effective (q_θ, q_γ) derived from q(w) (no duplicated algebra). - @average_energy ConjugateAR (also delegated). - MvNormalGamma registered as a prior node [out, μ, Λ, α, β] with its :out message rule. Verified end-to-end in RxInfer: a latent AR(2) model with an MvNormalGamma prior on (θ, τ) recovers the transition precision (0.99 vs 1.0) and the coefficients. Free energy for this model is not yet available (the MvNormalGamma prior node has no average energy yet).
- ConjugateAR(:w) message: likelihood-factor parameters and prod(prior, message) == the :w marginal posterior. - ConjugateAR(:y), (:x), (:y_x): delegation matches AR with the effective (q_θ, q_γ). - MvNormalGamma(:out): hyperparameter pass-through (PointMass and q_ variants). - conjugatear_effective_marginals (E[θ]=μ, E[γ]=α/β, mγ·Vθ=Λ⁻¹) and the average energy.
…gelog - @average_energy MvNormalGamma: cross-entropy of the posterior marginal against the prior factor, so free_energy=true works for ConjugateAR models. Monte-Carlo validated. - Unit test for the MvNormalGamma average energy (self cross-entropy == entropy). - Apply JuliaFormatter to the ConjugateAR / MvNormalGamma sources and tests. - CHANGELOG: document the ConjugateAR node and the MvNormalGamma factor node.
The derivation/spec doc was a pre-coding review gate; the node is now implemented and the math lives in the source comments and the PR description.
🤖 Code FormattingYour PR still has some code formatting issues. I've updated PR #613 with the necessary formatting changes. You can merge that PR into this branch to fix the code style check. Alternatively, you can run |
wmkouw
left a comment
There was a problem hiding this comment.
Looks great. Go for it.
My only comment is that the conjugate_autoregressive/y.jl message can already be cast as a Tdist. I realized after our call that the reason we don't have T-distributions in ExponentialFamily.jl is because T-distributions are not part of the exponential family. They do exist in Distributions.jl (https://juliastats.org/Distributions.jl/stable/univariate/#Distributions.TDist; although it may not be full location-scale parameterization).
I asked our mutual friend for the reason of the current implementation. Since TDist isn't exp-family it has no closed-form product with the downstream Gaussian message, so as an internal message we'd just project back to Gaussian, right? Do you want me to change this? |
|
Well, the But this is fine for now. If we need more, we can always open up a new branch with some options under the ARmeta. |
🤖 Code FormattingYour PR still has some code formatting issues. I've updated PR #613 with the necessary formatting changes. You can merge that PR into this branch to fix the code style check. Alternatively, you can run |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #612 +/- ##
==========================================
+ Coverage 82.64% 82.88% +0.24%
==========================================
Files 214 220 +6
Lines 6660 6720 +60
==========================================
+ Hits 5504 5570 +66
+ Misses 1156 1150 -6 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Co-authored-by: bvdmitri <6557701+bvdmitri@users.noreply.github.com>
Documenter's strict checkdocs flagged `ConjugateAR` and `conjugatear_effective_marginals` docstrings as not included in the manual. Add a dedicated `lib/nodes/conjugate_ar.md` page (modeled on `ar.md`) with a @docs block for both symbols and register it in the nav. Also convert the `[AR](@ref)`/`[ARMeta](@ref)` cross-references in the ConjugateAR docstring to plain backticks (neither has a docstring, so the links are unresolvable once the docstring is surfaced) and drop the dangling reference to the removed `design/ar_mvnormalgamma_conjugate_rule.md`. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add docstrings to the `AR` node and `ARMeta` metadata and surface them via a @docs block in `ar.md`. With both now documented, restore the `[AR](@ref)`/`[ARMeta](@ref)` cross-references in the `ConjugateAR` docs page and docstring that were temporarily downgraded to plain backticks. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Summary
Adds
ConjugateAR, an autoregressive node that carries the AR coefficientsθand thetransition precision
γjointly on a single edgewas anMvNormalGamma, instead of themean-field
q(θ)q(γ)split of the existingARnode. Because(θ, γ)are jointly conjugatefor the AR likelihood (a Bayesian linear regression with unknown noise precision), the
structured posterior
q(θ, γ)is exactlyMvNormalGamma— more accurate than mean-field andstill closed-form.
Also registers
MvNormalGammaas a prior factor node so models can writew ~ MvNormalGamma(μ, Λ, α, β).This branch depends on ReactiveBayes/ExponentialFamily.jl#287 (the
MvNormalGammadistribution +
prodrule). That PR must land first; this one should not be merged until theExponentialFamily release containing
MvNormalGammais available.What's included
ConjugateARnode[y, x, w]reusingARMeta.ConjugateAR(:w)marginal rule — the structured BLR Normal-Gamma posterior, withexpected sufficient statistics from
q(y, x):C = E[xxᵀ],b = E[x·y₁],a = E[y₁²].ConjugateAR(:w)message rule — the AR likelihood as a Normal-Gamma factor; its productwith the prior at the equality node reproduces the
:wmarginal (natural parameters add).ConjugateAR(:x),(:y),(:y_x)state/marginal rules and@average_energy,delegating to the tested
ARrules via effective(q_θ, q_γ)derived fromq(w)(noduplicated algebra).
MvNormalGammaprior node[out, μ, Λ, α, β]with its:outmessage rule and@average_energy(sofree_energy = trueworks forConjugateARmodels).Math references
All formulas were cross-checked against these references and for internal consistency
(message ↔ marginal ↔
prod, entropy ↔ self cross-entropy, univariate ↔NormalGamma).Test plan
ConjugateAR(:w)message params +prod(prior, message) == :wmarginal(:x),(:y),(:y_x)delegation matchesARwith effective(q_θ, q_γ)NormalGammareduction, properness/PSDMvNormalGamma(:out)pass-through; average energy (self cross-entropy == entropy)