Default a kernel's parameter source to a linear map, and the audit it forces#51
Merged
Merged
Conversation
An inline distribution previously allowed at most one vector-typed parameter, and only in the final slot, because the parameter split was greedy: the single vector ate whatever columns remained in the stacked input. A family like MixtureNormal(weights, loc, scale), whose three per-component vectors are each shared across the response plate, could not be expressed. The split now walks the declared parameter spec and slices each vector at its own dim offset, so a family may declare any number of vector positions in any order. Stacking interprets each argument by its declared role rather than guessing from tensor rank: a rank-0 position is a per-row scalar, a rank->=1 position is a vector of its declared dim arriving either shared as ``(D,)`` or per-row as ``(N, D)``. Size-1 batches broadcast to the widest batch before the concat. VectorisedObserve forwards ``_param_spec`` alongside the event ranks it already forwarded, so an observe step sees the inner family's declared dims instead of falling back to rank-guessing.
A grouped marginalize resolves its ``via`` index from the runtime environment. The index is host data, supplied through the observations dict like any free covariate, so omitting it is an ordinary user error. It surfaced as a bare KeyError naming only the variable, which read as an internal fault rather than a missing input. The lookup now raises a CompileError that names the index, says it is neither bound nor supplied, and states what to pass.
The grouped-marginalize snippets declared their fibration index as ``sample idx : Resp <- HalfNormal(1.0)``, which scores a continuous prior over what is a long tensor of row-to-group indices. The term is spurious: the index is host data supplied through the observations dict, as lda.qvr already writes it. The declarations are gone and each ``[via=idx]`` now reads the free index. No expected value moves: every snippet touched asserts compilation, a specific CompileError, or finiteness, and the tests that pin densities exercise marginalize_grouped directly. Covers the clean-error contract for an index that is never supplied.
The mixture model grouped a per-row component assignment under a marginalize block keyed by a fibration index, which made a plain per-row mixture read as an item-grouped one and carried a discrete latent the guide could not biject. It now scores each row directly against the mixture through the MixtureNormal likelihood, whose three shared per-component vectors define the components and whose per-row marginal is the closed-form convex combination. No fibration index, no marginalize block, no discrete latent. The grouped form remains expressible, and lda.qvr demonstrates it on a model whose grouping is intrinsic. The lda.md cross-reference now describes what this page shows.
The gallery try-it sweep built a fresh namespace per block, so a block that used an import or binding from an earlier block on the same page raised NameError. The pages are written to be read and run top to bottom, so every multi-block example failed on its second block regardless of whether its code was correct. The blocks of a page now share one namespace and run in order, as a reader stepping through the page would. This is what the sweep meant to assert; it is deselected by default, which is why the pages rotted without anyone noticing.
The example composed four latent morphisms under the real algebra, where composition is matrix multiplication. With no pointwise nonlinearity the chain collapses to a single linear map, so the page titled "Bayesian Neural Network" built a linear model whose three hidden objects bought nothing but a rank bound. Its SVI block fit a learnable input morphism to pure noise and reported a residual worse than predicting zero, and its NUTS block referenced an x and an observations dict that no earlier block defined, over a composite carrying no log_joint and therefore no likelihood to sample. The walkthrough also placed MatrixNormal priors on the weights through a syntax that does not parse, and the page closed by stating that a nonlinear network was not expressible. It is: a morphism declared `~ Family` over continuous spaces draws its distribution parameters from a ParamSource, and the default for a continuous domain is already a two-hidden-layer tanh MLP. The example is now a Bayesian MLP for nonlinear regression. A conditional-Normal kernel selects the MLP source explicitly and emits both the mean and the log-scale, so the fit is heteroscedastic; the target is a sine wave that no linear model can represent. The page scores the fit against the best least-squares line under a matching Gaussian, which separates the two by several hundred nats and attributes the gap to the tanh layers. Weight priors come from lift_from_log_prob, which is what makes the NUTS block Bayesian rather than maximum likelihood, and it now runs. Documents ParamSource, whose eight sources and DSL option had no API page despite being the seam the nonlinearity lives in.
The default addopts deselect `slow`, and CI ran plain `pytest tests/`, so the gallery sweep and the recovery benchmarks never executed on any push or pull request. Every example page's Python was unverified, which is how one page came to reference undefined names and another came to describe a prover whose rules cannot fire. They run here in a separate job, keeping the main test job's runtime unchanged.
Morphism options were checked against the union of every role's keys, then each role read only the subset it understood. An option belonging to another role therefore passed validation and was dropped in silence. Every use of `[scale=]` in the repository sat on a family-backed kernel, where nothing reads it: 34 declarations across nine examples configured an init scale that never existed, and the prose around them described a prior scale the compiler had discarded. Each role now declares the keys its lowering consumes, and a key outside that set is rejected with an error naming the role that does read it. The no-op options are gone from the sources and the walkthroughs, which is behaviour-preserving: they had no effect to begin with. One example also claimed a Normal kernel's mean was a learned linear function of its input. The mean comes from the kernel's param_source, whose default is an MLP.
`param_source_from_option` parses parenthesised hidden widths, and the grammar has admitted the call form all along: `[param_source=mlp(64, 64)]` parses to an OptionCall. The compiler read the option through `get_option_name`, which accepts a bare identifier and raises on anything else, so the call form died at compile time and the width parser was unreachable from QVR. The `get_option_string` fallback underneath it was dead code, since the name getter raises rather than returning None on a shape mismatch. The option now decodes through `get_option_call_text`, which renders a name or a call back to surface text and hands it to the parser that defines the arguments. `[param_source=mlp(16, 8)]` builds those widths; `[param_source=mlp]` keeps the default. The keyword form the source parser also accepts, `attention(heads=4)`, stays out of reach: the grammar's option_call takes positional arguments only.
A conditional family reads its parameters from a network, so for a codomain of dimension d they arrive as (N, d). The natural way to write N scalar responses against a d=1 codomain is (N,), which is also the shape the compiler's own response placeholder implies. Subtracting that from an (N, 1) mean broadcast to (N, N), and since the observe step sums the score, the result was a finite log-joint that was silently wrong by a factor of N. Nothing raised. The observe step now restores the event axis, and rejects a response that cannot carry the codomain's event shape rather than broadcasting it into one. Both spellings of a scalar response now score identically and match the hand-rolled density. An inline family keeps the other layout: its parameters come from the program's environment as per-row values already aligned with the plate, so its response's last axis indexes the plate rather than an event, and is left untouched. `_param_spec` marks that case.
`binders Lam` alpha-renames a lambda's bound variable to a fresh canonical symbol per term construction, and the occurrences in the body are renamed with it. A rewrite that renames the binder but leaves the body's `Var(x)` behind still yields a well-formed tuple and a chart that parses: the lambda binds nothing, the body references a free variable, and every existing test stays green. Nothing downstream checks the two agree. This pins them together, so a change to the term encoding that pulls a binder apart from its occurrences fails here instead of silently producing terms whose bindings are gone.
A morphism declared `f : X -> Y ~ Normal` read its parameters out of a two-hidden-layer tanh MLP, because that was the factory's default. So a 1 -> 1 kernel silently carried 4418 weights and two nonlinearities, and whether a model was linear was decided three layers below the surface syntax rather than in it. The default is a linear map, which is how the arrow reads. A model that wants a nonlinearity between its input and its parameters asks for one with `[param_source=mlp]`, and the fact then appears in the source. The audit the flip forces finds the gallery was mostly wrong the other way. `linear_gaussian_ssm` was not linear: every kernel in it was an MLP. Each LSTM, GRU and vanilla-RNN gate was a hidden two-layer network under a cell whose only intended nonlinearity is the sigmoid its program body applies. Those models are now what they say they are. `deep_markov` and the VAE's encoder and decoder bodies do want the nonlinearity, and now request it: `deep_markov` called `trans_mlp_1 >> trans_mlp_2` a two-layer MLP while composing two kernels with no activation between them, which is linear whatever the names say. Also collapses a duplicate: `_NeuralSource` in `morphisms` was a second copy of `MLPSource`, reached only by the default path, so the factory that serves the DSL option and the factory that served the default were different code. `_make_source` now delegates to `make_param_source`, and the copy is gone. Closes #48.
Three recovery benchmarks failed on main, and the slow job that now runs them made that visible. Two were the tests' fault. Both random-effects tests read an `alpha_g` site that the formula compiler does not emit: it writes the non-centred parameterisation, so a group's effect is its standard-normal draw scaled by the group-level sigma. They now read that, and both models turn out to recover the per-group effects well, correlating with truth at 0.998 in the partial-pooling case. The partial-pooling test also asserted the intercept alone. The intercept and the random effects are identified only up to a shared shift: adding a constant to one and subtracting it from the others leaves the likelihood untouched, and with eight groups the prior pulls the split back only weakly. So the fit was penalised for a split the data cannot determine while recovering the level to within 0.004. The level is now asserted on the identified sum, and the effects through a correlation, which that shift does not move. The polynomial benchmark is a real defect and stays failing, as a strict xfail carrying the diagnosis: the SVI fit collapses on orthonormal poly() columns, and the program's own log_joint puts least squares 439 nats ahead of where the optimiser settles.
`fit("y ~ poly(x, 2)", method="svi")` returned a fit that had given up,
without saying so. The noise scale rose to the marginal spread of the
response and the coefficients never left the prior:
fitted b1 0.44 b2 0.94 sigma 1.35 (y sd 1.40)
truth b1 9.93 b2 21.44 sigma 0.30
The program was not at fault. Scored under its own log_joint, least
squares reaches -84.2 where the fit settled at -523.6, so the optimiser
was missing a solution 439 nats better, and it was not a budget: lr
1e-2 / 5e-2 / 1e-1 and 2000 / 6000 / 12000 steps all landed in the same
place.
The prior was. A column enters the linear predictor as `beta * column`,
so a prior on the coefficient alone is a statement about the
coefficient's contribution, and a fixed `Normal(0, 5)` therefore means
something different for every column. `poly` returns an orthonormal
basis, whose entries run about 1/sqrt(N), so the default asserted the
contribution was near zero and the fit agreed with the prior rather
than the data.
A default prior's scale is now divided by its column's RMS, which
states it in contribution space, where its nominal value is what it
reads as. The coefficients stay on their own column's scale, so nothing
is transformed back. This is the autoscaling rstanarm applies by
default. An explicit prior is the user's statement about that
coefficient and is emitted as written.
The polynomial benchmark now recovers least squares to within a few
percent and pins sigma to the data's noise rather than to the spread of
y, so a fit that gives up fails it.
Also derives `quivers.__version__` from the installed distribution's metadata. It was a literal, so it was a second place to state the version and a second place to forget it, and it had drifted: the package reported 0.15.0 while importing it reported 0.14.1.
`[param_source=...]` is a morphism option the compiler accepts for any family-backed kernel, and exactly one of the 28 conditional families could receive it. `_make_continuous_morphism` passed the keyword into a constructor that only `ConditionalNormal` declared, so the other 27 died on a `TypeError` raised inside `__init__`, with no line or column on it. Nothing in the surface said the option was Normal-only: the key sits in the kernel role's set, the option checker passes it for every family, and the module documents it generally. Each family already built its source through the same `_make_source` call, and that call already took an override, so the seam was there and unreached. `_make_source` now resolves all three ways of choosing a source, and every family whose parameters come from one takes `param_source` and `param_source_option` and hands them over. The four whose parameters do not, Horseshoe, GaussianProcess, Independent and Transformed, refuse the option at compile time, naming the family. `hidden_dim` becomes a sequence, one width per hidden layer, because a single number can say how wide an MLP's layers are but not how many of them there are. A bare number is the one-layer case. It was also dropped whenever `param_source` was given, so `[param_source=mlp, hidden_dim=32]` silently built the default width; it now reaches the source, and a width aimed at a source with no hidden layers is an error rather than a no-op. Collapses the resolution that `ConditionalNormal` and `_IndependentConditional` each did inline behind a function-local import, which existed to dodge a cycle that no longer exists.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
A morphism declared
f : X -> Y ~ Normalover continuous spaces read its distribution parameters from a two-hidden-layer tanh MLP, because that was the factory's default. So aReal 1 -> Real 1kernel silently carried 4418 weights, and whether a model was linear was decided three layers below the surface syntax rather than in it. The default is now a linear map, which is how the arrow reads, and a model that wants a nonlinearity asks for one.Most of the rest of this PR is the audit that flipping the default forces, plus the defects it turned up along the way. The theme is uniform: things the surface accepted and then quietly ignored, or quietly got wrong.
Motivation
Closes #48. Refs #49, #50.
The gallery was mostly wrong in the other direction, which is the argument for the flip:
linear_gaussian_ssmwas not linear. Every kernel in it was an MLP.deep_markovcalledtrans_mlp_1 >> trans_mlp_2a "two-layer MLP" while composing two kernels with no activation between them, which is linear whatever the names say.bnn.mdclosed by asserting that a nonlinear MLP "is not currently expressible as a pure latent-morphism composition", while every~ Normalmorphism in the gallery already was one.The models that genuinely want the nonlinearity now name it. Four sequence models NaN'd under the flip because their numerical stability had been resting on the default's tanh, which is itself the point: a vanilla RNN's defining tanh had nowhere to live but a default nobody wrote.
Changes
continuous/make_param_sourcedefaults toLinearSource._make_sourcedelegates to it, collapsing_NeuralSourceinmorphisms, a second copy ofMLPSourcereachable only by the default path, so the factory serving the DSL option and the one serving the default are no longer different code.MixtureNormal(weights, locations, scales)is expressible.(N,)response against ad = 1codomain, which is the shape the compiler's own placeholder implies, broadcast against an(N, 1)mean into(N, N); because the observe step sums, the result was a finite log-joint wrong by a factor ofN, with nothing raised.dsl/[scale=]in the repository sat on family-backed kernels where nothing reads it.[param_source=mlp(16, 8)]reaches the width parser. The grammar always parsed the call form andparam_source_from_optionalways implemented the widths; the compiler decoded the option throughget_option_name, which raises on anything but an identifier, leaving theget_option_stringfallback beneath it unreachable.viafibration index raises aCompileErrornaming itself rather than a bareKeyError.formulas/fit("y ~ poly(x, 2)", method="svi")returned a fit that had given up without saying so: sigma rose to the marginal spread ofyand the coefficients never left the prior. The program was not at fault, and neither was the optimiser budget. A column enters asbeta * column, so a fixedNormal(0, 5)states the scale of the contribution, andpolyreturns an orthonormal basis whose entries run about1/sqrt(N), so the default asserted the signal was near zero and the fit agreed with the prior instead of the data.Docs
bnn.mdis a Bayesian MLP for nonlinear regression, fit to a sine wave no linear model can represent and scored against the best least-squares line under a matching Gaussian. Its NUTS block runs; it previously named anxand anobservationsno earlier block defined, over a composite carrying nolog_jointand therefore no likelihood to sample.mixture-model.mdscores per row throughMixtureNormalinstead of grouping a per-row assignment under amarginalizekeyed by a fibration index, which read as an item-grouped mixture and carried a discrete latent no guide could biject.montague-nli.md's prover ran rules that could never fire: both required aClaim(Implies(...))premise nothing produced, and itsImplieswas disjoint from the grammar'simplies_t. It now runs classically valid syllogistic rules, with existential import supplied explicitly rather than assumed.ParamSource, which had none despite being where a kernel's dependence on its input is computed.CI
addopts = -m 'not slow'and a plainpytest tests/meant the gallery sweep never ran on any push, which is how these pages rotted.API impact
[param_source=mlp]. Symptom if missed: a model that no longer fits, or a recurrence that diverges where the default's tanh had been bounding it.[scale=]on a family-backed kernel; it never did anything, so removing it is behaviour-preserving.N-times-inflated log-joint gets a correct number now.Tests
pytest -xpasses: 2577 passed, 138 skipped.pyright src/quiversis clean: 0 errors.ruff checkandruff format --checkpass.New coverage pins each fix to the defect it closes: multi-vector inline splitting; the role-aware option error naming the role that reads the key;
param_sourcename and call forms; both spellings of a scalar response scoring identically and matching the hand-rolled density; prior autoscaling in contribution space; and a lambda's bound occurrence matching its binder.That last one is worth calling out. Refs #49: the deduction term algebra has two encodings for a nullary constant, so a rule mentioning one inside an LF compiles clean and never fires. I attempted the unification and it silently unbinds every lambda in the gallery, with the entire suite still green. That test now exists and fails on exactly that break, so the fix is safe to attempt next time.
Three recovery benchmarks were failing on
mainand now pass. Two were the tests' fault: they read analpha_gsite the compiler does not emit (it writes the non-centred parameterisation), and one asserted an intercept that is only identified jointly with the random effects, penalising a fit that recovers the level to within 0.004 and the effects at a correlation of 0.998.Documentation
docs/updated.docs/developer/changelog.mdhas an entry for this change (0.16.0).Checklist