Skip to content

fix(qe): reject unresolved cutoffs and close silent under-convergence paths - #14

Merged
k-yoshimi merged 9 commits into
mainfrom
feat/qe-cutoff-required
Jul 2, 2026
Merged

fix(qe): reject unresolved cutoffs and close silent under-convergence paths#14
k-yoshimi merged 9 commits into
mainfrom
feat/qe-cutoff-required

Conversation

@k-yoshimi

@k-yoshimi k-yoshimi commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Summary

Make the QE input generator handle unresolvable cutoffs (ecutwfc/ecutrho) explicitly instead of silently writing 0.0 into the generated input.

The first commits (ba4b6c2, 70bbec9, f0595f0) implement the rejection of unresolved cutoffs; the follow-up commits (0bfeeef, b5771ae) fix the gaps found by two code-review rounds tracked in #13.

Closes #13

Changes

Reject unresolved cutoffs (ba4b6c2 ...)

  • _find_elem_cutoff raises InputValidationError instead of falling back to 0.0 when neither the cutoff CSV nor the UPF header resolves a cutoff.
  • Only the fields the user did not supply via content: are resolved (need_wfc/need_rho).

First review round (0bfeeef, #13)

  • Normalize blank CSV cells (pandas NaN) and literal-0.0 UPF header attributes (common in ld1.x-generated files) to "not specified" (_normalize_cutoff). Previously they slipped past the is None checks and ecutrho = NaN / ecutwfc = 0.0 ended up in the generated input.
  • When the CSV provides only one field, fill just the missing one from the UPF header instead of overwriting both.
  • Fill ecutwfc also when the template omits the key entirely: it is mandatory for pw.x, and the input used to be written without it.
  • Omit ecutrho when it cannot be resolved and let pw.x apply its own default (4*ecutwfc). It is optional in QE and normally absent for norm-conserving sets, so only ecutwfc remains a hard error.
  • Point the error remedy at the real YAML path content.namelist.system.ecutwfc (content.system.* would be treated as a card).

Second review round (b5771ae, #13 comment)

  • Floor an explicitly written ecutrho at 4 * the ecutwfc actually emitted (the resolved maximum, or the user-supplied value), so it never falls below the default pw.x would have chosen; rounding-level differences are tolerated so CSVs storing a rounded 4*ecutwfc keep their value.
  • When the template carries no cutoff keys at all, resolve ecutrho too: filling only ecutwfc left ultrasoft sets running on the 4*ecutwfc default instead of the suggested ~8-12x.
  • Fill only header-missing fields from the pp_info "Suggested" lines; an unset "0.0 Ry" line could reset a valid header wfc_cutoff to None and turn a resolvable pseudopotential into a hard error.
  • Warn on unparseable cutoff values and on elements excluded from the ecutrho competition (both were silent).
  • Reject pw.x tasks without a &system namelist instead of writing an input pw.x cannot run.
  • Consult the UPF file only when a needed field is unresolved, and log an open failure as a warning: successful runs no longer emit ERROR noise.
  • Align utils/pp_cutoff.py with the runtime convention: normalize 0.0/unparseable header values and emit a blank CSV cell (instead of dropping the row) when only rho_cutoff is missing.

Verification

  • pytest tests/ → 261 passed (all fixes were implemented test-first).
  • End-to-end CLI runs on sample/cif2x/qe/NaCl:
    • default case: generated cutoffs match scf.in_ref
    • blank ecutrho CSV cell: the ecutrho line is omitted, ecutwfc keeps the CSV value
    • missing cutoff row: clean one-line error with the correct remedy, exit 1, no file written
    • template without ecutwfc:: filled from the CSV; with ecutwfc: 100 supplied, ecutrho = 400 (the 4*ecutwfc floor)
    • task without &system: rejected with exit 1

Note: the QE test modules skip silently when qe_tools is unavailable; running them locally requires qe-tools<2 (2.x removed qe_tools.parsers.qeinputparser; pyproject pins ^1.1).

🤖 Generated with Claude Code

Kazuyoshi Yoshimi and others added 6 commits July 1, 2026 12:43
…g to 0.0

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Kp3mfWC9io4qwRsWuXM1Ec
When the user supplies one of ecutwfc/ecutrho and leaves the other empty,
require only the empty field from the pseudopotential source instead of
both, so a valid input is no longer rejected.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Kp3mfWC9io4qwRsWuXM1Ec
…lling the other

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Kp3mfWC9io4qwRsWuXM1Ec
Review of the qe-cutoff-required work (issue #13) found paths that still
let unresolved cutoffs slip into generated inputs:

- treat blank cutoff-CSV cells (pandas NaN) and non-positive UPF header
  cutoffs (ld1.x writes wfc_cutoff="0.0") as "not specified"
- fill only the missing field from the UPF header instead of discarding
  a CSV value when the other field is missing
- fill ecutwfc also when the template omits the key entirely; pw.x
  requires it, so a template without it silently produced invalid input
- stop hard-erroring on unresolved ecutrho: omit the key and let pw.x
  default to 4*ecutwfc (it is optional in QE and normally absent for
  norm-conserving sets); in mixed sets the suggested values compete
  against 4*ecutwfc of the elements without a suggestion
- point the error remedy at content.namelist.system.ecutwfc, the real
  YAML path (content.system.* would be treated as a card)
- drop the tautological guard in _find_elem_cutoff and dedupe the test
  setup into _qe/_write_upf/_pw_mode helpers

Refs #13

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…nt paths

Second review round on #13 found paths where the cutoff rework traded
hard errors for silently under-converged inputs:

- floor an explicitly written ecutrho at 4 * the ecutwfc actually
  emitted (resolved max, or the user-supplied value), so it never falls
  below the default pw.x would have used; tolerate rounding-level
  differences so CSVs storing a rounded 4*ecutwfc keep their value
- when the template carries no cutoff keys at all, resolve ecutrho too:
  filling only ecutwfc left ultrasoft sets on the 4*ecutwfc default
  instead of the suggested ~8-12x
- fill only header-missing fields from the pp_info "Suggested" lines;
  an unset 0.0 line could reset a valid header wfc_cutoff to None and
  turn a resolvable pseudopotential into a hard error
- warn on unparseable cutoff values and on elements excluded from the
  ecutrho competition (both were silent)
- reject pw.x tasks without a &system namelist instead of writing an
  input that pw.x cannot run
- consult the UPF file only when a needed field is unresolved, and log
  an open failure as warning: successful runs no longer emit ERROR noise
- align utils/pp_cutoff.py with the runtime convention: normalize
  0.0/unparseable header values and emit a blank CSV cell (instead of
  dropping the row) when only rho_cutoff is missing

Refs #13

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@k-yoshimi

Copy link
Copy Markdown
Contributor Author

第2回レビュー(issue #13 のコメント参照)で確認された残課題を b5771ae で修正しました:

  • 明示出力する ecutrho に「実際に書かれる ecutwfc の 4 倍」のフロアを導入(QE デフォルトを下回る値を明示出力しない。CSV の丸め差は許容)
  • カットオフキーの無いテンプレートでは ecutrho も解決(US 擬ポテンシャルの推奨値を反映)
  • pp_info の Suggested 行はヘッダーで欠けたフィールドのみ補完(0.0 行による上書き防止)
  • パース不能値・競合から除外された元素に warning を追加
  • &system 無しの pw タスクを InputValidationError で拒否
  • UPF ファイル参照を「必要なフィールドが未解決のときのみ」に限定し、成功時の ERROR ログを解消
  • utils/pp_cutoff.py をランタイム規約(空欄セル=未指定)に整合

テスト: 261 passed。NaCl サンプルで実機確認済み(参照一致・ecutwfc=100 明示時に ecutrho=400・キー無しテンプレートで両方補完・&system 無しで exit 1)。

🤖 Generated with Claude Code

…lled wheel

src/utils/pp_cutoff.py is a standalone helper, not part of the installed
distribution: importing it as `utils.pp_cutoff` only worked when conftest
put src/ on sys.path, and broke CI's CIF2X_TEST_INSTALLED run.

Refs #13

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@k-yoshimi k-yoshimi changed the title fix(qe): 未解決カットオフを拒否し、すり抜け経路(NaN/0.0/キー欠落)を封鎖・ecutrho は省略方針に fix(qe): reject unresolved cutoffs and close silent under-convergence paths Jul 2, 2026
Third review round (Codex, strict research-software reviewer; 3 rounds
to convergence, must_fix now empty):

- reject a None or non-mapping &system section (a bare "system:" in
  YAML parses to None) with InputValidationError instead of crashing
  with TypeError in the update helpers; same for content.namelist=None,
  which Content() uses as its default
- validate a user-supplied ecutwfc before the 4*ecutwfc floor: reject
  non-numeric values with a clear error, accept numeric strings
  leniently and write the normalized float back so f90nml does not
  render a quoted Fortran string
- regression tests for all three paths, plus a pp_cutoff guard for an
  absent rho_cutoff attribute yielding a blank CSV cell

Refs #13

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@k-yoshimi

Copy link
Copy Markdown
Contributor Author

Third review pass, this time with Codex as an independent strict reviewer (different angle from the two multi-agent correctness reviews). Three rounds to convergence; fixes landed in 329a7e7:

  • Round 1 must_fix: "system" not in content.namelist raised TypeError when namelist is None (the Content() default). The guard now handles falsy namelists and raises the intended InputValidationError.
  • Round 2 must_fix: a system: section that parses to None (bare key in YAML) passed the membership check and crashed the update helpers. The guard now requires a mapping.
  • Round 1+2 should_fix: a user-supplied non-numeric ecutwfc reached 4.0 * ecutwfc and raised TypeError; numeric strings stayed strings and could render as quoted Fortran values. Non-numeric values are now rejected with a clear error, numeric strings are accepted and written back as floats.
  • Also added a pp_cutoff regression test for an absent rho_cutoff attribute (blank CSV cell), confirming the intended output-contract change flagged by the reviewer.

Round 3 reported no remaining must_fix. Tests: 266 passed.

@k-yoshimi
k-yoshimi merged commit 0ceeeb9 into main Jul 2, 2026
2 checks passed
@k-yoshimi
k-yoshimi deleted the feat/qe-cutoff-required branch July 2, 2026 07:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

QEカットオフ解決処理のバグ修正(NaN/0.0すり抜け・ecutwfc欠落・誤案内)と ecutrho 省略方針への変更

1 participant