Skip to content

Use predictor for leaf id cache updates - #12385

Merged
RAMitchell merged 4 commits into
dmlc:masterfrom
RAMitchell:codex/predict-from-leaf-ids
Jul 31, 2026
Merged

Use predictor for leaf id cache updates#12385
RAMitchell merged 4 commits into
dmlc:masterfrom
RAMitchell:codex/predict-from-leaf-ids

Conversation

@RAMitchell

Copy link
Copy Markdown
Member

Summary

Move training prediction-cache accumulation from tree updaters into the predictor layer.

With updater row positions standardized, cache updates no longer need updater-specific implementations. This adds Predictor::PredictFromLeafIds, which accumulates prediction contributions from known leaf ids, mirroring the existing PredictLeaf direction:

  • PredictLeaf: features to leaf ids
  • PredictFromLeafIds: leaf ids to prediction contributions

GBTree::DoBoost now uses updater-produced node positions and the selected predictor backend to update cached predictions after leaf values are finalized.

Changes

  • Add Predictor::PredictFromLeafIds.
  • Implement CPU and GPU predictor support.
  • Add SYCL predictor delegation to CPU for the new method.
  • Remove TreeUpdater::UpdatePredictionCache.
  • Remove updater-specific prediction-cache implementations from hist, approx, GPU hist, and SYCL hist updaters.
  • Update GBTree training cache updates to use predictor-side leaf-id prediction.
  • Extend predictor/cache tests for the new ownership boundary.

Validation

  • cmake -S . -B build -DUSE_CUDA=OFF -DGOOGLE_TEST=ON -DUSE_DMLC_GTEST=ON
  • cmake --build build --target objxgboost -j2
  • cmake --build build --target testxgboost -j2
  • ./build/testxgboost --gtest_filter=CpuPredictor.Basic
  • ./build/testxgboost --gtest_filter=CPUPredictor.*:CpuPredictor.TrainingPredictionCache:Updater.PredictionCache*:TestPredictionCache.*
  • cmake -S . -B build-cuda -DUSE_CUDA=ON -DGOOGLE_TEST=ON -DUSE_DMLC_GTEST=ON
  • cmake --build build-cuda --target objxgboost -j2
  • cmake --build build-cuda --target testxgboost -j2
  • ./build-cuda/testxgboost --gtest_filter=GPUPredictor.Basic:GPUPredictor.PredictLeafBasic:GpuHist.UniformSampling:GpuHist.GradientBasedSampling:TestPredictionCache.GpuHist:TestPredictionCache.GpuMultiHist:TestPredictionCache.GpuApprox

@RAMitchell
RAMitchell requested a review from Copilot July 30, 2026 10:04
@RAMitchell
RAMitchell marked this pull request as ready for review July 30, 2026 10:04

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR moves training prediction-cache accumulation out of individual tree updaters and into the predictor layer by introducing Predictor::PredictFromLeafIds, enabling cache updates from known leaf IDs (node positions) after leaf values are finalized.

Changes:

  • Added Predictor::PredictFromLeafIds API and implemented it for CPU and GPU predictors, with SYCL delegating to CPU.
  • Updated GBTree::DoBoost to update cached predictions via the selected predictor using updater-produced node positions.
  • Removed updater-specific prediction-cache update code paths and updated/extended related unit tests.

Reviewed changes

Copilot reviewed 19 out of 19 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
tests/cpp/tree/test_prediction_cache.h Switches cache-update testing from updater API to PredictFromLeafIds.
tests/cpp/tree/test_gpu_hist.cu Updates GPU hist tests to use predictor-based cache updates.
tests/cpp/predictor/test_predictor.cc Adds direct unit coverage for PredictFromLeafIds equivalence vs PredictBatch.
tests/cpp/predictor/test_cpu_predictor.cc Renames/aligns CPU training cache test naming with new ownership boundary.
src/tree/updater_quantile_hist.cc Removes quantile-hist updater’s cache-update implementation.
src/tree/updater_gpu_hist.cuh Removes multi-target GPU hist cache-update implementation.
src/tree/updater_gpu_hist.cu Removes GPU hist/approx updater cache-update implementations.
src/tree/updater_approx.cc Removes approx updater cache-update implementation and related include usage.
src/tree/hist/evaluate_splits.h Removes shared CPU cache-update helpers previously used by updaters.
src/predictor/gpu_predictor.cu Implements PredictFromLeafIds on GPU using leaf IDs to accumulate predictions.
src/predictor/cpu_predictor.cc Implements PredictFromLeafIds on CPU using leaf IDs to accumulate predictions.
src/gbm/gbtree.cc Updates training-time cache updates to use predictor-based leaf-id accumulation.
plugin/sycl/tree/updater_quantile_hist.h Removes SYCL updater cache-update API and related declarations.
plugin/sycl/tree/updater_quantile_hist.cc Removes SYCL updater cache-update implementation and registry formatting updates.
plugin/sycl/tree/hist_updater.h Removes SYCL hist-updater cache-update declaration.
plugin/sycl/tree/hist_updater.cc Removes SYCL hist-updater cache-update implementation.
plugin/sycl/predictor/predictor.cc Adds SYCL predictor delegation for PredictFromLeafIds (currently CPU fallback).
include/xgboost/tree_updater.h Removes TreeUpdater::UpdatePredictionCache from the public interface.
include/xgboost/predictor.h Adds Predictor::PredictFromLeafIds to the public predictor interface.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/predictor/cpu_predictor.cc
Comment thread plugin/sycl/predictor/predictor.cc Outdated
Comment on lines +223 to +224
LOG(WARNING) << "PredictFromLeafIds is not yet implemented for SYCL. CPU Predictor is used.";
cpu_predictor->PredictFromLeafIds(leaf_ids, trees, out_preds);
Move prediction-cache updates out of tree updaters and into predictors using exported leaf positions. Add CPU, CUDA, and SYCL implementations and prediction-cache coverage.
@RAMitchell
RAMitchell force-pushed the codex/predict-from-leaf-ids branch from d4fa239 to f0eb010 Compare July 31, 2026 15:01
The previous run failed before project compilation in the CPU and clang-CUDA jobs, while CUDA 12 death tests failed independently of this change.
The prior two runs failed in missing Protobuf, clang-CUDA configuration, and unrelated CUDA 12 span death tests. Current master at the same base SHA passed.
@RAMitchell
RAMitchell merged commit e787a44 into dmlc:master Jul 31, 2026
81 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants