Skip to content

Update dependencies, add PCA explained variance plotting functionalit…#127

Merged
idanmoradarthas merged 3 commits into
masterfrom
85-add-plot_pca_explained_variance-function-to-preprocess-module
May 21, 2026
Merged

Update dependencies, add PCA explained variance plotting functionalit…#127
idanmoradarthas merged 3 commits into
masterfrom
85-add-plot_pca_explained_variance-function-to-preprocess-module

Conversation

@idanmoradarthas

@idanmoradarthas idanmoradarthas commented May 20, 2026

Copy link
Copy Markdown
Owner

Description

This PR resolves #85. It enhances the preprocess module by adding a new plotting function, plot_pca_explained_variance, which visualizes the cumulative explained variance ratio from Principal Component Analysis (PCA).

This visualization will help users quickly determine how many principal components are required to capture a desired proportion of variance in their datasets.

Key Changes

  • Source Code: Added plot_pca_explained_variance to ds_utils/preprocess/visualization.py. It accepts numerical data, optionally scales it (defaulting to StandardScaler), and draws a line plot for cumulative variance with horizontal reference lines at 70% and 80%.
  • Testing: Added tests/test_preprocess/test_plot_pca_explained_variance.py with comprehensive unit tests for different parameters, handling invalid inputs, and rendering the plot. Used sklearn.datasets.make_classification to generate a realistic test dataset.
  • Baseline Images: Generated new pytest-mpl baseline images to test plot rendering.
  • Documentation:
    • Updated docs/source/preprocess/visualization.rst with the function definition, code example, and the generated output image.
    • Added a new section for this method in README.md containing a code snippet and the example image.
    • Updated skills/preprocess/SKILL.md to document the new feature for agentic workflows.
  • Formatting: Linted and formatted all modifications using ruff.

Type of change

  • New feature (non-breaking change which adds functionality)
  • This change requires a documentation update

Checklist:

  • I have performed a self-review of my code
  • I have made corresponding changes to the documentation (Sphinx, README, SKILL.md)
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes (pytest & pytest-mpl)
  • Code is linted using ruff

…y, and enhance documentation

- Updated `pyarrow`, `kaleido`, `ruff`, and `build` dependencies in `pyproject.toml`.
- Introduced `plot_pca_explained_variance` function to visualize cumulative explained variance of PCA components, aiding in dimensionality reduction analysis.
- Added usage examples and detailed documentation for the new PCA plotting function in `README.md` and `visualization.rst`.
- Updated `SKILL.md` to include the new PCA function with parameters and common usage scenarios.
- Bumped version to 1.10.0rc13 to reflect these changes.
@idanmoradarthas idanmoradarthas self-assigned this May 20, 2026
@idanmoradarthas idanmoradarthas linked an issue May 20, 2026 that may be closed by this pull request
@codecov

codecov Bot commented May 20, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

…A parameters

- Updated the `plot_pca_explained_variance` function to allow passing additional keyword arguments (`pca_kwargs`) directly to the PCA initialization, enhancing flexibility in PCA configuration.
- Modified the scaler parameter to default to `None`, using `StandardScaler` if no scaler is provided.
- Enhanced documentation to reflect the new parameter options and usage examples in `SKILL.md`.
- Added tests to validate the new functionality, ensuring correct handling of PCA parameters and Axes object passing.
@idanmoradarthas
idanmoradarthas merged commit 6b24838 into master May 21, 2026
13 checks passed
@idanmoradarthas
idanmoradarthas deleted the 85-add-plot_pca_explained_variance-function-to-preprocess-module branch May 21, 2026 03:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add plot_pca_explained_variance function to preprocess module

1 participant