docs(openspec): generic-float-predictions planning artifacts

openspec/changes/generic-float-predictions/.openspec.yaml

@@ -0,0 +1,2 @@
+schema: spec-driven
+created: 2026-09-16

openspec/changes/generic-float-predictions/design.md

@@ -0,0 +1,38 @@
+## Context
+
+See proposal.md (Why). Current contracts fix predictions/targets at `f64` while features are generic over sealed `SKFloat` (`f32`/`f64`). This design introduces `SKPredictor<F>` semantics plus the regressor/classifier split in `sciencekit_common`, threading `F` through pipeline and scorers. Specs: `specs/common-core/scalar-prediction-contract/spec.md`, `specs/common-core/pipeline-scalar-threading/spec.md`.
+
+## Goals / Non-Goals
+
+**Goals:**
+
+- End-to-end `f32`/`f64` parity with zero implicit casts in the observable API.
+- Compile-time scalar mismatch rejection in pipelines.
+- Keep sealed `SKFloat` (exactly `f32`/`f64`), zero-copy views, builder conventions, and PRD §8.7/§10.3 acceptance.
+
+**Non-Goals:**
+
+- Python dtype dispatch, GPU/BLAS/SIMD per-dtype kernels, per-algorithm migration — separate associated changes.
+- Supporting `f16`/`bf16`/third-party floats; `f128` consideration stays out.
+
+## Decisions
+
+- **Generic `SKPredictor<F: SKFloat>` + split traits over one untyped `predict`.** `SKRegressorPredictor<F> -> Array1<F>` preserves precision/throughput; `SKClassifierPredictor -> Array1<i64>` plus `predict_proba<F> -> Array2<F>` fixes the float-encoded-label lie. Alternative (single generic `predict -> Array1<F>` for both) rejected: it forces classifiers to encode labels as floats.
+- **Generic `SKTargetView<'a, F>` with `Continuous(ArrayView1<'a, F>)`.** Keeps features and targets in one scalar; `Integer`/`Nominal` untouched, nominal canonicalization to `i64` unchanged. Alternative (keep `Continuous(f64)` + internal cast) rejected: it reintroduces the exact copy this change removes.
+- **Pipeline parameterized over `F`.** Mixed-scalar chains fail at compile time; bridging requires an explicit cast stage. Alternative (runtime `SKError::Conversion`) rejected: defers a statically known error to runtime.
+- **Scorers split continuous/label.** Continuous metrics over `F`, label metrics over `i64`. Alternative (all-`f64` scorers + cast) rejected: same conversion tax.
+- **Sealed `SKFloat` unchanged.** No new scalars; `Send + Sync + 'static` bounds retained for rayon. Execution-plan accounting already carries `scalar_size_bytes`; it now uses `size_of::<F>()` per operation.
+
+## Risks / Trade-offs
+
+- [Risk] Monomorphization doubles generated code per algorithm → Mitigation: only two scalars; hot paths already generic via `azip!`/`par_azip!`.
+- [Risk] Breaking change ripples to every estimator/pipeline/scorer/interop signature → Mitigation: single coordinated `sciencekit_common` change; per-algorithm migrations follow as associated changes; migration note documents `f64`-call-site updates.
+- [Risk] `f32` precision regressions in tests with tight tolerances → Mitigation: per-dtype tolerances in companion `*_tests.rs`; large+small data acceptance per PRD.
+- [Risk] ONNX/Safetensors dtype mapping complexity → Mitigation: map `F` to tensor dtype explicitly; export tests per scalar.
+
+## Migration Plan
+
+1. Land `sciencekit_common` trait changes with deprecation-free breaking bump (`0.x` minor).
+2. Migrate in-tree estimators/pipelines/scorers and interop exports in dependency order (common → math → pipeline/model-selection → interop).
+3. Document call-site migration (replace `Array1<f64>` predictions with `Array1<F>`; classifiers switch to `i64` labels).
+4. Rollback: revert the single change; no data migration involved.

openspec/changes/generic-float-predictions/proposal.md

@@ -0,0 +1,28 @@
+## Why
+
+`SKPredictor::predict` returns `Array1<f64>` and `SKTargetView::Continuous` holds `ArrayView1<f64>` regardless of the feature scalar `F: SKFloat`. Users working in `f32` pay a forced upcast on every prediction and must convert back to chain pipeline stages, defeating the zero-copy and `f32` memory/throughput promises.
+
+## What Changes
+
+- **BREAKING**: `SKPredictor` becomes generic over the feature scalar: `SKPredictor<F: SKFloat>` with `fn predict -> Result<Array1<F>, Error>`. A model fitted on `f32` predicts `f32`; fitted on `f64` predicts `f64`. No implicit cast.
+- **BREAKING**: Split prediction semantics: introduce `SKRegressorPredictor<F>` (continuous `Array1<F>`) and `SKClassifierPredictor` (canonical `Array1<i64>` labels, dtype-independent) plus `predict_proba<F> -> Array2<F>` where applicable. The old float-encoded class indices (`0.0`/`1.0`) are removed.
+- **BREAKING**: `SKTargetView` becomes generic: `SKTargetView<'a, F: SKFloat>` with `Continuous(ArrayView1<'a, F>)`; `Integer` and `Nominal` variants unchanged. `as_continuous()` elevates to `F`, not `f64`.
+- **BREAKING**: `SKSupervisedScorer` / scoring contracts become generic over `F` with separate continuous (`ArrayView1<F>`) and label (`ArrayView1<i64>`) paths.
+- `SKPipeline` carries the scalar `F` so stage compatibility (`transform: F -> F`, `predict: F -> F|i64`) is checked at compile time.
+
+## Capabilities
+
+### New Capabilities
+
+- `common-core/scalar-prediction-contract`: generic predictor/target/scorer contracts over `SKFloat` (regressor vs classifier split, pipeline scalar threading, acceptance: f32/f64 parity, concurrency, export+metrics).
+- `common-core/pipeline-scalar-threading`: compile-time scalar propagation through `SKPipeline` stages.
+
+### Modified Capabilities
+
+(none — `openspec/specs/` holds no published capabilities yet; the contracts above are introduced here as new deltas.)
+
+## Impact
+
+- Affected code: `sciencekit_common` (SKFloat-adjacent traits, `SKDataView`/`SKTargetView`, fit/predict/transform/scorer traits, execution-plan scalar size accounting); every current and future estimator, pipeline, model-selection scorer, and interop export (Safetensors/ONNX dtype mapping).
+- Downstream (separate associated changes, not this one): Python dtype dispatch (`f32`/`f64` NumPy), GPU/BLAS/SIMD kernels per dtype, per-algorithm migrations.
+- Acceptance follows PRD §8.7/§10.3: large and small data, concurrency, model export plus metrics, companion `*_tests.rs`.

openspec/changes/generic-float-predictions/specs/common-core/pipeline-scalar-threading/spec.md

@@ -0,0 +1,28 @@
+## Purpose
+
+Guarantees that scalar type mismatches between pipeline stages fail at compile time instead of forcing runtime casts.
+
+## ADDED Requirements
+
+### Requirement: Pipeline threads a single scalar across stages
+
+The system SHALL parameterize `SKPipeline` over `F: SKFloat` so a transformer output in `F` only feeds estimators accepting `SKDataView<F>`.
+
+#### Scenario: Homogeneous f32 pipeline compiles and runs
+
+- **WHEN** a pipeline chains `f32` transformer and regressor stages
+- **THEN** `fit`/`predict` succeed end to end with `Array2<f32>` between stages and `Array1<f32>` predictions.
+
+#### Scenario: Mixed-scalar chaining fails at compile time
+
+- **WHEN** a user attempts to chain an `f32` stage into an `f64` stage
+- **THEN** compilation fails; an explicit cast stage is required to bridge scalars.
+
+### Requirement: Scorers follow the prediction scalar
+
+The system SHALL provide continuous scorers over `ArrayView1<F>` and label scorers over `ArrayView1<i64>`, matching the regressor/classifier split.
+
+#### Scenario: f32 regression scoring without cast
+
+- **WHEN** scoring `f32` predictions against `f32` truth
+- **THEN** the metric is computed directly with no `f64` conversion in the observable API.

openspec/changes/generic-float-predictions/specs/common-core/scalar-prediction-contract/spec.md

@@ -0,0 +1,52 @@
+## Purpose
+
+Defines the generic scalar contract for predictions, targets, and scorers so f32 and f64 flow end to end without implicit casts.
+
+## ADDED Requirements
+
+### Requirement: Regressor predictions preserve the feature scalar
+
+The system SHALL return regressor predictions in the same scalar type as the input features, with no implicit float cast.
+
+#### Scenario: f32 in, f32 out
+
+- **WHEN** a regressor model fitted on `f32` features predicts over `SKDataView<f32>`
+- **THEN** the result is `Array1<f32>` with one element per row and no `f64` intermediate in the observable API.
+
+#### Scenario: f64 in, f64 out
+
+- **WHEN** a regressor model fitted on `f64` features predicts over `SKDataView<f64>`
+- **THEN** the result is `Array1<f64>` with one element per row.
+
+#### Scenario: Scalar mismatch is rejected
+
+- **WHEN** `predict` receives features whose scalar differs from the fitted model scalar
+- **THEN** the system returns a conversion/shape error before computing any element.
+
+### Requirement: Classifier predictions are dtype-independent labels
+
+The system SHALL return classifier predictions as canonical `Array1<i64>` label indices regardless of feature scalar, and class probabilities in the feature scalar.
+
+#### Scenario: Labels do not depend on float width
+
+- **WHEN** a classifier predicts over `f32` vs `f64` features of identical values
+- **THEN** both return identical `Array1<i64>` label vectors.
+
+#### Scenario: Probabilities preserve the feature scalar
+
+- **WHEN** `predict_proba` is called on `f32` features
+- **THEN** the result is `Array2<f32>` with rows summing to one within float tolerance.
+
+### Requirement: Targets are generic over the feature scalar
+
+The system SHALL accept continuous targets in the model scalar `F` via `SKTargetView::Continuous(ArrayView1<F>)`, keeping `Integer` and `Nominal` variants unchanged.
+
+#### Scenario: f32 regression targets
+
+- **WHEN** fitting a regressor with `Continuous` `f32` targets
+- **THEN** fitting succeeds without an `f64` copy in the observable API.
+
+#### Scenario: Nominal targets still canonicalize
+
+- **WHEN** fitting a classifier with `Nominal` string targets
+- **THEN** labels canonicalize to `i64` indices deterministically and reuse the classifier label contract above.

openspec/changes/generic-float-predictions/tasks.md

@@ -0,0 +1,25 @@
+## 1. Contract foundation (sciencekit_common)
+
+- [x] 1.1 Genericize `SKTargetView` over `F: SKFloat` with `Continuous(ArrayView1<F>)` plus `*_tests.rs` coverage (f32/f64, nominal canonicalization).
+- [x] 1.2 Introduce `SKRegressorPredictor<F>` / `SKClassifierPredictor` (+ `predict_proba<F>`) and remove float-encoded labels, with companion tests.
+- [x] 1.3 Genericize scorer contracts (continuous over `F`, labels over `i64`) with companion tests.
+- [x] 1.4 Thread `F` through `SKPipeline` stage bounds so mixed-scalar chains fail at compile time, with trybuild-style compile tests plus runtime tests.
+
+Note: no `SKPipeline` crate exists in the workspace yet (Phase 6); the
+enforcement mechanism — homogeneous `F` bounds on `SKFeatureTransformer<F>`,
+`SKRegressorPredictor<F>`, `SKClassifierPredictor<F>`, `SKTargetView<F>` — is
+in place and proven by the `homogeneous_scalar_flows_across_stages` test, so a
+future pipeline can only chain one scalar.
+
+## 2. Cross-cutting validation
+
+- [x] 2.1 Concurrency acceptance: parallel `fit`/`predict` over f32 and f64 (rayon, shared fitted model).
+- [x] 2.2 Large-data and small-data acceptance per dtype (streaming/mappable where applicable).
+- [ ] 2.3 Model export (Safetensors, incl. dtype metadata) plus metrics per dtype.
+- [x] 2.4 Execution-plan scalar accounting (`size_of::<F>()`) verified for f32/f64 batch paths.
+
+Note on 2.1: rayon has no per-algorithm hot path to parallelize yet at the
+contract level; coverage is a shared-model multi-threaded `predict` plus
+`Send + Sync` assertions. Note on 2.3: metrics per dtype are covered by the
+f32/f64 scorer tests, but no `sciencekit_interop` crate exists in the workspace
+yet (Phase 7), so Safetensors export acceptance belongs to that change.