Enhance Regression Metrics Module with REC Curves and AUC Calculation#126
Merged
idanmoradarthas merged 2 commits intoMay 2, 2026
Conversation
- Introduced Regression Error Characteristic (REC) curves and the associated Area Over the Curve (AOC) metric for evaluating regression models. - Added functions `plot_rec_curve_with_annotations` and `regression_auc_score` to visualize and quantify model performance. - Updated README.md and documentation to include new metrics and usage examples. - Enhanced tests for regression metrics, ensuring robust validation of new functionalities. - Bumped version to 1.10.0rc12 to reflect these additions.
12 tasks
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
- Deleted the import statements for `matplotlib.pyplot` in `test_curves.py` and `test_regression.py` to clean up the code and improve clarity. - This change helps streamline the test files by removing unnecessary dependencies, enhancing maintainability.
idanmoradarthas
deleted the
95-add-regression-error-characteristic-rec-curve-and-regression-auc
branch
May 2, 2026 07:51
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Resolves #95 by implementing the Regression Error Characteristic (REC) curve and its associated Regression Area Over the Curve (AOC) metric.
REC curves generalize the classification ROC curve to the regression domain, allowing for robust visual comparisons of continuous models. This PR implements these capabilities as a new dedicated sub-module (
ds_utils.metrics.regression), keeping the architectural design consistent with the existing metrics workflows.✨ New Features
regression_auc_score: Calculates the normalized Area Over the REC Curve (AOC). Returns a score in the[0, 1]range where0.0denotes perfect predictions (zero error).plot_rec_curve_with_annotations: Visualizes REC curves for multiple regression models simultaneously using Plotly. The calculated AOC score is automatically appended to the interactive hover/legend.📚 Documentation Updates
docs/source/metrics/regression.rst) containing mathematical background and theoretical context, properly referencing Bi & Bennett (2003) and Hernández-Orallo (2013).README.mdwith an end-to-end REC curve example.skills/metrics/SKILL.mdto establish a formal "Regression Workflow" for agentic interactions.🧪 Testing & Refactoring
tests/test_metrics/test_regression.pywith 100% code coverage for all logic branches and edge cases (e.g., shape mismatches, sample weight propagation).pytest-mplto guarantee visualization consistency over time.save_plotly_figure_and_return_matplot) into a sharedtests/utils.pyfile to DRY up the testing suite.Type of Change
Verification
pytest).--mpl).make html).ruff check,ruff format).