Speed up GP gradient and initial hyperparameter estimation - #96
Open
k-yoshimi wants to merge 2 commits into
Open
Speed up GP gradient and initial hyperparameter estimation#96k-yoshimi wants to merge 2 commits into
k-yoshimi wants to merge 2 commits into
Conversation
- exact.get_grad_marlik: reuse the existing Cholesky factor via scipy.linalg.cho_solve instead of recomputing a full LU-based inverse with scipy.linalg.inv. Numerically identical (max abs diff ~1e-18), ~14x faster for the inverse step. - cov._gauss.get_params_bound: vectorize the random pairwise-distance sampling loop with a single np.random.randint((M, 2)) draw. Produces the identical random sequence as the previous per-iteration loop, so results are unchanged for a given seed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Addresses AI review feedback (rounds 1-2): lock down the numerical/RNG equivalence claims behind the perf changes, exercising the production code. - test_get_grad_marlik_matches_finite_difference: checks the cho_solve-based gradient against a central finite-difference of eval_marlik (rtol=1e-6, atol=1e-7; observed error ~1e-9 across seeds). - test_get_cand_params_matches_old_loop_implementation: runs the real cov.get_cand_params and asserts it equals an in-test reimplementation of the old per-iteration loop under the same seed (covers the full path including the post-sort randint draw). - test_get_cand_params_is_reproducible_for_fixed_seed: guards seed stability. RNG global state is saved/restored around the seeded tests. Verified by mutation testing: a wrong inverse and a non-equivalent pair-sampling change each make the corresponding test fail.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two numerically-equivalent GP speedups:
exact.get_grad_marlik: reuse the existing Cholesky factor viascipy.linalg.cho_solveinstead of recomputing a full LU-based inverse withscipy.linalg.inv. Numerically identical (max abs diff ~1e-18), ~14× faster for the inverse step. This is on the hyperparameter-optimization hot path.cov._gauss.get_cand_params: vectorize the random pairwise-distance sampling loop into a singlenp.random.randint((M, 2))draw. Produces the identical random sequence as the previous per-iteration loop, so per-seed results are unchanged.Tests
test_get_grad_marlik_matches_finite_difference: checks thecho_solve-based gradient against a central finite-difference ofeval_marlik(rtol=1e-6,atol=1e-7; observed error ~1e-9 across seeds).test_get_cand_params_matches_old_loop_implementation: runs the realcov.get_cand_paramsand asserts it equals an in-test reimplementation of the old loop under the same seed.test_get_cand_params_is_reproducible_for_fixed_seed.Verified by mutation testing (wrong Cholesky
lowerflag and non-equivalent pair sampling each fail the corresponding test). Reviewed with Codex until convergence.🤖 Generated with Claude Code