Repository navigation
Test that fit_from_preprocessed output depends only on the batch - #1360
Merged
Merged
Conversation
A custom fine-tuning loop calls fit_from_preprocessed and then forward. Both regressions we hit there were estimator state from outside the batch: a correction left over from fit(), and a target frame only fit() sets. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
bejaeger
approved these changes
Oct 12, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Generated by Claude Code
Adds one test, for both estimators, that pins the contract of a custom fine-tuning loop. The loop calls
fit_from_preprocessedon a batch and thenforward. The test requires that the output does not depend on whether the estimator has already runfit()or an earlier batch on other data. Two regressions of this kind have already slipped through review: #1270 (a stale correction fromfit()) and #1198 (a target frame that onlyfit()sets).What the test does
fit()on other data, or a loop step on another batch, then runs the same step. Its output must equal the fresh one (per-estimator logits, and borders for the regressor).majority_downsample, sofit()also stores a prior-shift correction._forward_on_batchis the whole loop step. If a change needs more steps there, every user loop outside this repo breaks, and the change to the helper makes that visible in review.Checked against both known regressions
main(15f5e6b): 4 passed, 2 s on CPU.AttributeError: 'TabPFNRegressor' object has no attribute 'y_train_mean_'.mainwithout the two Correct the target prior shift of majority_downsample row subsampling #1270 resets infit_from_preprocessed: the classifierfitcase fails, with all 50 outputs different by up to 0.26.predict, notforward. A stale correction there never reaches the fine-tuning loss.test__classifier__fit_from_preprocessed_clears_stale_correctionand its regressor twin intests/test_downsample_correction.pyoverlap this test. I left them in place.🤖 Generated with Claude Code