Skip to content

Commit 35da29b

Browse files
Oren Bassikclaude
andcommitted
Add algebraic_multiplicity option + M-row truncation in analysis_utils
`EstimationOptions.algebraic_multiplicity::Union{Int, Nothing} = nothing`. When set to a positive integer M, the output of analyze_estimation_result truncates `cluster_reps` to `min(M, branch_top_k, length(cluster_reps))` rows. When `nothing` (default), behavior is byte-identical to before — slice to branch_top_k. `branch_top_k` retains its role as a safety cap (so if M is set incorrectly large, we still bound the output). The two knobs compose multiplicatively: M is the algebraic intent, branch_top_k is the numerical cap. Validator rejects non-positive Ints. fast_core 258/258 stays green (default-path is byte-identical). Calling convention for downstream: - Pass M from a per-system catalog (PEB's generate_scripts.py now injects from config/systems.json[*].algebraic_multiplicity). - When ODEPE has an auto-compute (SI gate + HC root count via upstream Pogudin patch — TODO local-claude), the caller can stop setting this field and ODEPE will compute M itself. See PEB `results/wallaby_analysis/multiplicity/M_INFERENCE_INVESTIGATION.md` for the full investigation including: - Why count(:locally) is NOT the multiplicity (it's the swap-subspace dimensionality; for our 4 mult-2 systems n_locally varies 4-6 but M=2). - Failed attempt to harvest check_primality_zerodim's dim (it's over the LEADER ring not parameter ring; gave M=1 for daisy_mamil4 instead of 2). - Where M IS computed but not exposed: SIAN-Julia SIAN.jl:265 has the right Gröbner basis whose quotient_basis dim is M. A small upstream patch (~5 lines) would expose it. - Catalog-fallback approach used today (PEB injects per-system M from config/systems.json). Impact on benchmark stats (estimated from existing wallaby data via PEB `results/wallaby_analysis/m_truncation_impact.md`): - Paper-headline M-bounded metric: 0pp change (definition). - K=20 oracle ceiling diagnostic: -2.6pp polish / -4.1pp nopolish @10% — that diagnostic disappears because result.csv no longer contains rows beyond M. - Top-1: 0pp change (row 0 doesn't move). - 10 cells / 2300 ODEPE cells (0.4%) lose accuracy under truncation, concentrated at high noise where polish put truth at rank > M. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent b623e5e commit 35da29b

2 files changed

Lines changed: 44 additions & 3 deletions

File tree

src/core/analysis_utils.jl

Lines changed: 19 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -934,8 +934,25 @@ function analyze_estimation_result(problem::ParameterEstimationProblem, result;
934934
else # :err_only — cluster_reps already err-sorted upstream
935935
cluster_reps
936936
end
937-
if opts.branch_top_k > 0 && length(sorted_reps) > opts.branch_top_k
938-
sorted_reps[1:opts.branch_top_k]
937+
# Truncation policy:
938+
# - `opts.algebraic_multiplicity = nothing` (default) → keep up to
939+
# `branch_top_k` rows. Legacy K=20 behavior.
940+
# - `opts.algebraic_multiplicity = M` (positive Int) → keep
941+
# `min(M, branch_top_k, length(sorted_reps))` rows. M is the
942+
# algebraic multiplicity of the problem; output should contain
943+
# exactly that many algebraic branches. `branch_top_k` remains
944+
# a safety cap in case M is set wrong.
945+
# See PEB `results/wallaby_analysis/multiplicity/M_INFERENCE_INVESTIGATION.md`
946+
# for the inference pipeline and the upstream-patch path.
947+
M_eff = if opts.algebraic_multiplicity === nothing
948+
opts.branch_top_k
949+
elseif opts.branch_top_k > 0
950+
min(opts.algebraic_multiplicity, opts.branch_top_k)
951+
else
952+
opts.algebraic_multiplicity
953+
end
954+
if M_eff > 0 && length(sorted_reps) > M_eff
955+
sorted_reps[1:M_eff]
939956
else
940957
sorted_reps
941958
end

src/types/estimation_options.jl

Lines changed: 25 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -317,7 +317,27 @@ Base.@kwdef struct EstimationOptions
317317
branch_err_factor::Float64 = 100.0
318318
branch_resid_factor::Float64 = 100.0
319319
branch_min_size::Int = 1
320-
branch_top_k::Int = 20 # Maximum cluster reps to return at the output stage. Sorted by `rank_strategy` (default S2 = (saturation_count, is_neg1, err)) before slicing. **Dropped 100→20 on 2026-05-17** after probe4c K-recall analysis on the full 2026-05-14 numbat benchmark (1136 cells): under S2 sort, K=20 already saturates the candidate-set ceiling at the ≤10% threshold (83.5% K-recall at K=20 = 83.5% set ceiling); K=100 buys nothing further. K=10 catches 99.4% of the ceiling; K=5 catches 98.8%. Earlier 2026-05-14 setting of 100 was based on the legacy "within 2× of 06" recovery metric (74% at K=20, 77% at K=100) which is more conservative than absolute K-recall. Set to 0 to disable (return all reps).
320+
branch_top_k::Int = 20 # Maximum cluster reps to return at the output stage. Sorted by `rank_strategy` (default S2 = (saturation_count, is_neg1, err)) before slicing. **Dropped 100→20 on 2026-05-17** after probe4c K-recall analysis on the full 2026-05-14 numbat benchmark (1136 cells): under S2 sort, K=20 already saturates the candidate-set ceiling at the ≤10% threshold (83.5% K-recall at K=20 = 83.5% set ceiling); K=100 buys nothing further. K=10 catches 99.4% of the ceiling; K=5 catches 98.8%. Earlier 2026-05-14 setting of 100 was based on the legacy "within 2× of 06" recovery metric (74% at K=20, 77% at K=100) which is more conservative than absolute K-recall. Set to 0 to disable (return all reps). **Acts as a safety cap** when `algebraic_multiplicity` is set: actual output is `min(algebraic_multiplicity, branch_top_k, length(cluster_reps))`.
321+
322+
# Algebraic multiplicity of the parameter-estimation problem — the number of
323+
# distinct (params, IC) tuples in the identifiable subspace that produce
324+
# identical observations. When set, the output is truncated to this many rows
325+
# (capped above by `branch_top_k` as a safety net). When `nothing`, output is
326+
# the full top-K candidate list (length up to `branch_top_k`).
327+
#
328+
# Computing M structurally requires running SI/HC machinery (see the open
329+
# investigation at PEB `results/wallaby_analysis/multiplicity/M_INFERENCE_INVESTIGATION.md`).
330+
# Today this field is set by the caller — typically PEB's per-cell template
331+
# injects the value from `config/systems.json[*].algebraic_multiplicity`,
332+
# a hand-curated catalog whose values were derived via HC root-counting per
333+
# `repro/multiplicity_complete_2026_05_19/MULTIPLICITY_COMPLETE.md`.
334+
#
335+
# A future patch will add an auto-compute fallback inside ODEPE (SI gate
336+
# `n_locally == 0 ⇒ M = 1`, then HC root count for the M>1 case via a
337+
# small upstream patch to SIAN-Julia or StructuralIdentifiability.jl).
338+
#
339+
# Type: positive Int (the multiplicity), or `nothing` (use full top-K).
340+
algebraic_multiplicity::Union{Int, Nothing} = nothing
321341

322342
# Ranking strategy for the top-K cluster reps returned in result.csv.
323343
#
@@ -1249,6 +1269,10 @@ function validate_options(opts::EstimationOptions)
12491269
@error "cluster_method must be :identifiable_subspace or :bit_identical (got $(opts.cluster_method))"
12501270
valid = false
12511271
end
1272+
if !isnothing(opts.algebraic_multiplicity) && opts.algebraic_multiplicity <= 0
1273+
@error "algebraic_multiplicity must be a positive integer or `nothing` (got $(opts.algebraic_multiplicity))"
1274+
valid = false
1275+
end
12521276
if opts.rough_cluster_eps <= 0
12531277
@error "rough_cluster_eps must be positive (got $(opts.rough_cluster_eps))"
12541278
valid = false

0 commit comments

Comments
 (0)