Simplify the staged Core pipeline

[sigilmakes]

Note

This PR is stacked on #50 and intentionally targets feat/qrf-kpoints-reland. After #50 merges, retarget this PR to main and rerun checks.

Summary

Goldilocks Core turns a structure and calculation intent into DFT recommendations and generated input files. This PR provides a staged, composable workflow for that process:

Load → Analyze → Advise → Kmesh → Select → Generate → Bundle

The built-in workflow currently generates Quantum ESPRESSO SCF input. Its stages and data contracts are reusable by future calculation types, including workflows that generate several linked files.

Pipeline API

CoreJobRequest is the serializable description of a job. Pipeline contains the Python callables used to process it.

The standard public functions are:

• recommend() for structure analysis, parameter advice, k-points, and resource selection;
• generate() for recommendations plus generated input files;
• write_bundle() for generated files plus a portable manifest directory.

Each stage is also a normal public Python function. Callers can run stages directly, inspect intermediate records, insert project-specific work, or compose a different sequence.

Any Pipeline field can be replaced with a compatible callable. For example, a calculation-specific generator can reuse the built-in scientific stages:

pipeline = Pipeline(generate=generate_magnetic_nscf)
result = generate(
    "Fe.cif",
    intent=CalculationIntent(task="magnetic_nscf"),
    pipeline=pipeline,
)

Generate returns a tuple of GeneratedFile records, allowing one task to produce files such as scf.in and nscf.in together.

Scientific recommendations

The workflow provides provenance-backed recommendations for:

• k-point sampling;
• smearing and occupations;
• spin polarization;
• spin-orbit coupling;
• pseudopotential family and relativistic treatment;
• convergence controls;
• van der Waals corrections.

Structure analysis reports facts separately from advice. Select resolves advice into concrete k-points, pseudopotentials, and cutoffs. Generate translates completed decisions into target-code syntax.

QRF k-point model

The default Kmesh backend loads the configured Quantile Random Forest lazily. Model and metallicity artifacts can come from Hugging Face or local paths, and an alternate TOML registry can replace the packaged configuration.

Explicit k_grid and k_spacing hints bypass model loading. Model-loading and prediction errors propagate directly to the caller. A model-free path is available explicitly through --heuristic-kpoints or Pipeline(kmesh=resolve_kpoints_from_advice).

Successful predictions record the model identity, confidence, predicted interval, and selected mesh in provenance.

Quantum ESPRESSO generation

The built-in generator supports Quantum ESPRESSO SCF single-point input. It renders:

• control, system, and electrons namelists;
• lattice vectors and atomic positions;
• atomic species and selected pseudopotential filenames;
• wavefunction and charge-density cutoffs;
• k-point grid and shift;
• magnetism, SOC, smearing, convergence, and vdW settings.

Pseudopotential metadata is treated as external input. Missing or invalid cutoffs remain unavailable and produce actionable warnings. Generation requires complete selections for every structure element and checks the values it places into QE syntax.

Bundle mode writes generated files and a deterministic manifest.json to a new directory. Paths are confined to the bundle root, and existing destinations, duplicate paths, and manifest collisions are rejected.

Test framework

Tests are organized into three layers:

• unit tests for individual policies, records, model adapters, and renderers;
• integration tests for complete recommendation, generation, and bundle workflows;
• physics tests for reciprocal-space sampling, smearing, SOC, pseudopotential selection, and QE output behavior.

CI reports branch coverage and runs focused mutation testing with mutmut against deterministic scientific and generation modules. The mutation gate currently requires a 74% score; the current suite scores 74.5% (919/1234 mutants killed).

CI also checks Ruff linting and formatting, builds the wheel and source distribution, and validates packaged data and importability.

Verification

• 237 tests pass.
• Ruff check and format pass.
• Pre-commit passes.
• Focused mutmut run passes at 74.5%.
• Wheel and source distributions build and pass package-content validation.

Closes #41.
Closes #43.
Closes #52.
Closes #54.
Closes #56.

---

Written by an agent on behalf of Willow.

[sigilmakes]

Blocked on PR #50

This PR is stacked on #50 and should remain targeted at feat/qrf-kpoints-reland while that work is under review.

Once #50 merges:

1. Retarget this PR to main.
2. Confirm the resulting diff contains only the pipeline hardening, testing, and package-health changes.
3. Rerun the required checks before marking it ready for review.

Until then, this PR is blocked and has been returned to draft status.

---

Written by an agent on behalf of Willow.

[junwen94]

I have merged PR #50 and retargeted PR #68.