Summary
test_silhouette_score_batched intermittently produces results that do not match scikit-learn for the cosine and sqeuclidean metrics.
The observed failures occur with chunk_divider=1, where the full input is processed as one batch. The corresponding cases with chunk dividers 3 and 5 passed, as did the ordinary Euclidean cases.
Failing job:
https://github.com/rapidsai/cuml/actions/runs/30895344798/job/91946855208
Failures
FAILED test_metrics.py::test_silhouette_score_batched[labeled_clusters0-1-cosine]
ACTUAL: 0.011623237282037735
DESIRED: 0.028383221477270126
FAILED test_metrics.py::test_silhouette_score_batched[labeled_clusters0-1-sqeuclidean]
ACTUAL: 0.0030782711692154408
DESIRED: 0.02632964349662489
Both comparisons fail at two decimal places.
Environment
- Branch:
release/26.08
- OS: Ubuntu 22.04
- Python: 3.12
- CUDA: 12.2.2
- GPU: A100
- Architecture: amd64
- Dependencies: latest
libcuvs: 26.08.00a117, build 260731_2db820e7libraft: 26.08.00a61, build 260804_97d22c81librmm: 26.08.00a62, build 260804_1a39f9e8
The tests ran with eight pytest-xdist workers using work stealing.
Expected behavior
cuml.metrics.silhouette_score should match sklearn.metrics.silhouette_score within the test tolerance for every supported metric and batch size.
Related issues
Summary
test_silhouette_score_batchedintermittently produces results that do not match scikit-learn for thecosineandsqeuclideanmetrics.The observed failures occur with
chunk_divider=1, where the full input is processed as one batch. The corresponding cases with chunk dividers 3 and 5 passed, as did the ordinary Euclidean cases.Failing job:
https://github.com/rapidsai/cuml/actions/runs/30895344798/job/91946855208
Failures
Both comparisons fail at two decimal places.
Environment
release/26.08libcuvs:26.08.00a117, build260731_2db820e7libraft:26.08.00a61, build260804_97d22c81librmm:26.08.00a62, build260804_1a39f9e8The tests ran with eight pytest-xdist workers using work stealing.
Expected behavior
cuml.metrics.silhouette_scoreshould matchsklearn.metrics.silhouette_scorewithin the test tolerance for every supported metric and batch size.Related issues