feat(config): add strict config validation#654
Conversation
Signed-off-by: Yunfeng Zhang <yunzhang@nvidia.com>
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Enterprise Run ID: 📒 Files selected for processing (1)
📜 Recent review details⏰ Context from checks skipped due to timeout. (9)
🧰 Additional context used📓 Path-based instructions (7)**/*.{md,markdown,py}📄 CodeRabbit inference engine (.cursor/rules/agent-markdown-style.mdc)
Files:
**/*.py📄 CodeRabbit inference engine (AGENTS.md)
Files:
**/test_*.py📄 CodeRabbit inference engine (AGENTS.md)
Files:
tests/**📄 CodeRabbit inference engine (.cursor/rules/repo-navigation.mdc)
Files:
tests/**/*.py📄 CodeRabbit inference engine (tests/TESTING.md)
Files:
⚙️ CodeRabbit configuration file
Files:
**/*📄 CodeRabbit inference engine (STYLE_GUIDE.md)
Files:
⚙️ CodeRabbit configuration file
Files:
**/*.{py,sh,yaml,yml,md}📄 CodeRabbit inference engine (CONTRIBUTING.md)
Files:
🧠 Learnings (1)📚 Learning: 2026-05-27T22:20:37.354ZApplied to files:
🪛 ast-grep (0.44.1)tests/sdk/test_process_data.py[info] 529-529: use jsonify instead of json.dumps for JSON output (use-jsonify) 🔇 Additional comments (1)
WalkthroughAdds ChangesConfiguration strictness
Estimated code review effort: 4 (Complex) | ~45 minutes Suggested labels: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
Signed-off-by: Yunfeng Zhang <yunzhang@nvidia.com>
Signed-off-by: Yunfeng Zhang <yunzhang@nvidia.com>
There was a problem hiding this comment.
Actionable comments posted: 3
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 5ef03351-93c2-4be6-9a6f-950e36227939
📒 Files selected for processing (11)
docs/developer-guide/configuration_management.mddocs/user-guide/configuration.mdsrc/nemo_safe_synthesizer/config/job.pysrc/nemo_safe_synthesizer/config/parameters.pysrc/nemo_safe_synthesizer/configurator/parameters.pysrc/nemo_safe_synthesizer/sdk/config_builder.pysrc/nemo_safe_synthesizer/sdk/library_builder.pytests/cli/test_config.pytests/cli/test_utils.pytests/config/test_parameters.pytests/sdk/test_config_builder.py
📜 Review details
⏰ Context from checks skipped due to timeout. (1)
- GitHub Check: Greptile Review
🧰 Additional context used
📓 Path-based instructions (17)
**/*.{md,markdown,py}
📄 CodeRabbit inference engine (.cursor/rules/agent-markdown-style.mdc)
**/*.{md,markdown,py}: Avoid decorative bold (**text**) in list items, body text, and docstrings; use structural cues (headers, list markers, colons, backticks) for emphasis instead
Use backticks for code identifiers, paths, and CLI commands in markdown and docstrings
Files:
docs/developer-guide/configuration_management.mdtests/cli/test_config.pysrc/nemo_safe_synthesizer/sdk/library_builder.pydocs/user-guide/configuration.mdtests/sdk/test_config_builder.pysrc/nemo_safe_synthesizer/config/job.pytests/cli/test_utils.pysrc/nemo_safe_synthesizer/configurator/parameters.pysrc/nemo_safe_synthesizer/sdk/config_builder.pysrc/nemo_safe_synthesizer/config/parameters.pytests/config/test_parameters.py
**/*.{md,markdown}
📄 CodeRabbit inference engine (.cursor/rules/agent-markdown-style.mdc)
**/*.{md,markdown}: Bold is acceptable only in markdown tables where it's the conventional way to mark header-like cells in the body
Use##headers to segment markdown sections instead of bold text
Use--(em-dash) instead of-(hyphen) for asides in markdown
Files:
docs/developer-guide/configuration_management.mddocs/user-guide/configuration.md
docs/**/*.md
📄 CodeRabbit inference engine (.cursor/rules/writing-docs.mdc)
docs/**/*.md: Use MkDocs Material admonition syntax (!!! note, !!! warning, ??? tip) for highlighting important information and collapsible sections in documentation
Use MkDocs Material tabs syntax (=== "Label") to present alternative views or language-specific examples in documentation
Use code block syntax with title and highlight line parameters (title="filename", hl_lines="2 3") for code examples in documentation
Use Mermaid diagram syntax (```mermaid flowchart, etc.) for visualizations in documentationClassify documentation using Diátaxis and use MkDocs Material syntax for admonitions, tabs, and titled or highlighted code blocks.
Documentation pages must be Markdown files under
docs/, placed in the appropriate Diátaxis subdirectory, and added to thenav:section ofmkdocs.yml.
Files:
docs/developer-guide/configuration_management.mddocs/user-guide/configuration.md
**/*.md
📄 CodeRabbit inference engine (STYLE_GUIDE.md)
Do not use decorative bold in Markdown body text, list items, or docstrings; use single backticks for code identifiers, paths, and commands.
Use the repository's Markdown style and ensure Markdown source files have SPDX copyright headers unless excluded by
.copyrightignore.
Files:
docs/developer-guide/configuration_management.mddocs/user-guide/configuration.md
**/*
📄 CodeRabbit inference engine (STYLE_GUIDE.md)
**/*: Every source file requires the SPDX copyright and license header appropriate to its file format.
End files with a newline, remove trailing whitespace, use one space between sentences, and keep code, comments, and docstrings within 120 characters.
**/*: All merged commits must use Conventional Commits format:<type>(<scope>): <description>or<type>: <description>, with a lowercase valid type and a description no longer than 100 characters.
All contributions require a DCOSigned-off-bytrailer and a cryptographic commit signature.
Before submitting a pull request, run formatting, checks, and tests usingmise run format,mise run check, andmise run test.
Branches other thanmainmust use lowercase author-prefixed names in one of the documented forms, optionally including an issue ID and category.
Release tags must use avprefix and PEP 440 stable or release-candidate versions, such asv1.0.0orv0.1.0rc0; alpha versions and dashed-rcsuffixes are not used.
Do not move a published release tag; create and validate a new release candidate when code changes.
Usemise run <task>for project tasks; the Makefile only bootstraps mise and provides deprecated compatibility messages.
Files:
docs/developer-guide/configuration_management.mdtests/cli/test_config.pysrc/nemo_safe_synthesizer/sdk/library_builder.pydocs/user-guide/configuration.mdtests/sdk/test_config_builder.pysrc/nemo_safe_synthesizer/config/job.pytests/cli/test_utils.pysrc/nemo_safe_synthesizer/configurator/parameters.pysrc/nemo_safe_synthesizer/sdk/config_builder.pysrc/nemo_safe_synthesizer/config/parameters.pytests/config/test_parameters.py
⚙️ CodeRabbit configuration file
**/*: Review as a senior maintainer for NeMo Safe Synthesizer. Prioritize issues that can change behavior, break user workflows, weaken privacy guarantees, hide failures, make tests unreliable, or create maintenance risk. Avoid generic style commentary unless it points to a concrete project convention that automated tools will not catch.
Comment only when the finding is actionable and tied to changed code. For each finding, state the impact, the condition that triggers it, and the smallest practical fix. Prefer one precise comment over broad advice. Do not ask for refactors outside the PR scope unless the changed code creates the problem.
Review type guidance: - Potential issue: use for correctness bugs, data loss, privacy leaks,
security risks, broken public APIs, invalid config behavior, missing
validation, hidden failures, nondeterministic tests, or CI breakage.
- Refactor suggestion: use for local maintainability problems introduced
by the diff when they have clear future cost, such as duplicated setup,
unclear boundaries, over-mocking, avoidable complexity, or opaque test
helpers.- Nitpick: avoid in chill mode. Do not emit formatting, import-order,
wording, or style-only comments unless automated tools cannot catch the
issue and it affects maintainability.Severity guidance: - Critical: security/privacy leaks, data loss, training/test/holdout
contamination, or broken release/package/core pipeline execution.
- Major: incorrect generation/training/evaluation behavior, broken
CLI/SDK public API, invalid config defaults or validators, or GPU/vLLM
cleanup and process-isolation bugs likely to fail CI or production
runs.- Minor: localized bugs, missing focused tests for changed behavior, or
bad test patterns that weaken regression coverage.- Trivial: small cleanup with no behavior impact. Usually suppress in
chill mode.- Info: context only. Avoid unless it helps reviewers understand risk.
Safe-Synthesizer-specific review focus: - Data ...
Files:
docs/developer-guide/configuration_management.mdtests/cli/test_config.pysrc/nemo_safe_synthesizer/sdk/library_builder.pydocs/user-guide/configuration.mdtests/sdk/test_config_builder.pysrc/nemo_safe_synthesizer/config/job.pytests/cli/test_utils.pysrc/nemo_safe_synthesizer/configurator/parameters.pysrc/nemo_safe_synthesizer/sdk/config_builder.pysrc/nemo_safe_synthesizer/config/parameters.pytests/config/test_parameters.py
**/*.{py,sh,yaml,yml,md}
📄 CodeRabbit inference engine (CONTRIBUTING.md)
All source files with
.py,.sh,.yaml,.yml, or.mdextensions must include SPDX copyright headers, except files listed in.copyrightignore.
Files:
docs/developer-guide/configuration_management.mdtests/cli/test_config.pysrc/nemo_safe_synthesizer/sdk/library_builder.pydocs/user-guide/configuration.mdtests/sdk/test_config_builder.pysrc/nemo_safe_synthesizer/config/job.pytests/cli/test_utils.pysrc/nemo_safe_synthesizer/configurator/parameters.pysrc/nemo_safe_synthesizer/sdk/config_builder.pysrc/nemo_safe_synthesizer/config/parameters.pytests/config/test_parameters.py
docs/**
⚙️ CodeRabbit configuration file
Review documentation as MkDocs Material content. Check Diataxis fit, accurate commands, internal links, code fences, and markdown style from STYLE_GUIDE.md.
Files:
docs/developer-guide/configuration_management.mddocs/user-guide/configuration.md
**/*.py
📄 CodeRabbit inference engine (AGENTS.md)
**/*.py: Place durable implementation guidance in function and class docstrings for public contracts and source comments for local invariants
Target Python 3.11–3.13 with modern syntax (X | Y,list[str],Self). Python 3.14+ is not supported
**/*.py: Use American English spelling in Python code, documentation, and messages.
UseField(description=...)for every Pydantic model field.
Use assignment-styleField()by default; useAnnotatedonly for additional metadata such as validators, constrained aliases, or discriminated unions.
Use@dataclass(frozen=True)for immutable value objects and validators; use mutable dataclasses only for builders, accumulators, and pipeline state.
Usefield(default_factory=list)instead of mutable list defaults.
UseStrEnumfor string-valued configuration or serialization enums and plainEnumfor internal constants.
Obtain loggers withobservability.get_logger(__name__); do not calllogging.getLogger()orstructlog.get_logger()directly.
Use.runtime,.user, and.systemcategory loggers appropriately.
Do not useprint()for operational library output; use the approved logger,click.echo(), orsys.stdout.write()where appropriate.
Useextra={}for machine-queryable logging data and f-strings only for human-readable context.
Raise errors from the custom Safe Synthesizer error hierarchy, using the documented dual inheritance for user and internal errors.
Keep shared package code compatible with Python 3.11; do not use Python 3.12-only syntax such as PEP 695 type statements or bracketed generic parameters.
PreferX | Y, built-in collection generics, andSelfoverOptional,Union, and legacy typing collections.
Use collection ABCs for function arguments and concrete collection types for return values.
UseProtocolfor structural subtyping and avoidAnywhenobject, generics, or protocols are suitable.
UseTYPE_CHECKINGguards for heavy imports such as pandas, torch, and transformers.
...
Files:
tests/cli/test_config.pysrc/nemo_safe_synthesizer/sdk/library_builder.pytests/sdk/test_config_builder.pysrc/nemo_safe_synthesizer/config/job.pytests/cli/test_utils.pysrc/nemo_safe_synthesizer/configurator/parameters.pysrc/nemo_safe_synthesizer/sdk/config_builder.pysrc/nemo_safe_synthesizer/config/parameters.pytests/config/test_parameters.py
**/test_*.py
📄 CodeRabbit inference engine (AGENTS.md)
Use the
unitmarker instead of the deprecatedunit_testmarker for test identificationRun tests through the repository's mise tasks, such as
mise run test, rather than relying on unpinned standalone commands.
Files:
tests/cli/test_config.pytests/sdk/test_config_builder.pytests/cli/test_utils.pytests/config/test_parameters.py
tests/**
📄 CodeRabbit inference engine (.cursor/rules/repo-navigation.mdc)
tests/**: Mirrorsrc/directory structure intests/directory for test organization
Auto-mark tests by directory:tests/e2e/→e2e,tests/smoke/→smoke, otherwise default tounitMirror source code directory structure in tests directory (e.g.,
tests/training/,tests/generation/parallel to source structure)
Files:
tests/cli/test_config.pytests/sdk/test_config_builder.pytests/cli/test_utils.pytests/config/test_parameters.py
tests/**/*.py
📄 CodeRabbit inference engine (tests/TESTING.md)
tests/**/*.py: Auto-mark tests based on file path: tests under/e2e/gete2emarker, tests under/smoke/getsmokemarker, all others getunitmarker (only if no category marker already present)
Every test should have exactly one category marker:unit,smoke, ore2e
Usepytest.mark.requires_gpumodifier on tests that need CUDA hardware
Usepytest.mark.vllmon tests using vLLM generation backend and ensure each vLLM test file runs in its own process for GPU memory isolation
Usepytest.mark.slowon long-running tests
Usepytest.mark.smollm2for SmolLM2 Hub download tests to enable process isolation
Usepytest.mark.noautouseto skip autouse fixtures for specific tests
Useload_test_dataset(filename)helper to load test datasets fromtests/stub_datasets/as HuggingFaceDatasetobjects
Useload_test_dataframe(filename)helper to load test data files fromtests/stub_datasets/as pandas DataFrames
Convert pandas columns to nullable dtypes (pd.Int64Dtype(),pd.BooleanDtype()) before assigningnp.nanvalues
Usefake.seed_instance(seed)andrandom.seed(seed)together for Faker-based test data reproducibility
When sharing methods across multiple test files, define them inconftest.pyand import them using relative imports (e.g.,from .conftest import train_with_sdk); note that importing from other test files liketests/cli/helpers.pydoes not work
Usefixture_mock_processororfixture_mock_processor_without_valid_recordsfor mocking ParsedResponse objects withvalid_records,invalid_records,errors, andprompt_numberfields
Usepytest.importorskipto gate tests on optional dependencies that require specific extras (e.g.,sentence_transformers,vllm)
Run vLLM tests with separate pytest invocations (one per file) using-n 0(single process) for GPU memory isolation, or use staged mise tasks for CI visibility
Print statements are allowed in tests (ruffT201is suppressed fortests/directory) and should...
Files:
tests/cli/test_config.pytests/sdk/test_config_builder.pytests/cli/test_utils.pytests/config/test_parameters.py
⚙️ CodeRabbit configuration file
Review tests against tests/TESTING.md. Check marker usage, fixture naming, tmp_path usage, determinism, and GPU/vLLM process-isolation requirements. Flag slop tests that only check that code runs, assert result is not None when stronger invariants exist, over-mock internal implementation details, patch around the bug instead of reproducing it, or add broad snapshot/golden churn without a clear contract. Flag change detector tests that fail on harmless refactors, formatting, record ordering, incidental wording, or private implementation details without demonstrating a behavior regression. Prefer existing fixtures or focused new fixtures for repeated setup; keep tests DRY when reasonable without making the behavior under test opaque. print() is allowed in tests.
Files:
tests/cli/test_config.pytests/sdk/test_config_builder.pytests/cli/test_utils.pytests/config/test_parameters.py
tests/cli/**/*.py
📄 CodeRabbit inference engine (tests/TESTING.md)
Use
mock_workdir(tmp_path)helper in CLI tests to create temporary Workdir instances
Files:
tests/cli/test_config.pytests/cli/test_utils.py
src/**/*.py
📄 CodeRabbit inference engine (STYLE_GUIDE.md)
src/**/*.py: Use relative imports for package code undersrc/.
Do not useassertfor validation in library code; raise an appropriate exception instead.
Files:
src/nemo_safe_synthesizer/sdk/library_builder.pysrc/nemo_safe_synthesizer/config/job.pysrc/nemo_safe_synthesizer/configurator/parameters.pysrc/nemo_safe_synthesizer/sdk/config_builder.pysrc/nemo_safe_synthesizer/config/parameters.py
⚙️ CodeRabbit configuration file
Review library code against STYLE_GUIDE.md. Focus on behavior, API contracts, error handling, resource cleanup, typing, logging, and user-facing failures. Public APIs and nontrivial functions need Google-style docstrings.
Files:
src/nemo_safe_synthesizer/sdk/library_builder.pysrc/nemo_safe_synthesizer/config/job.pysrc/nemo_safe_synthesizer/configurator/parameters.pysrc/nemo_safe_synthesizer/sdk/config_builder.pysrc/nemo_safe_synthesizer/config/parameters.py
src/nemo_safe_synthesizer/**/*.py
📄 CodeRabbit inference engine (CONTRIBUTING.md)
Write durable module-level guidance in Python docstrings and source comments so it appears in the generated API reference.
Files:
src/nemo_safe_synthesizer/sdk/library_builder.pysrc/nemo_safe_synthesizer/config/job.pysrc/nemo_safe_synthesizer/configurator/parameters.pysrc/nemo_safe_synthesizer/sdk/config_builder.pysrc/nemo_safe_synthesizer/config/parameters.py
src/**/config/**/*.py
📄 CodeRabbit inference engine (STYLE_GUIDE.md)
Use
NSSBaseModelfor user-facing configuration and parameter models.
Files:
src/nemo_safe_synthesizer/config/job.pysrc/nemo_safe_synthesizer/config/parameters.py
src/nemo_safe_synthesizer/config/**/*.py
⚙️ CodeRabbit configuration file
Treat config changes as user-facing API changes. Check Pydantic field descriptions, defaults, validators, aliases, override behavior, CLI help text impact, YAML compatibility, and documented parameter semantics.
Files:
src/nemo_safe_synthesizer/config/job.pysrc/nemo_safe_synthesizer/config/parameters.py
src/nemo_safe_synthesizer/configurator/**/*.py
⚙️ CodeRabbit configuration file
Review Pydantic-to-Click mapping carefully. Check option names, type conversion, nullable sub-config behavior, validation errors, help text, and compatibility with parse_overrides().
Files:
src/nemo_safe_synthesizer/configurator/parameters.py
🧠 Learnings (1)
📚 Learning: 2026-05-27T22:20:37.354Z
Learnt from: kendrickb-nvidia
Repo: NVIDIA-NeMo/Safe-Synthesizer PR: 520
File: tests/generation/test_vllm_backend.py:556-587
Timestamp: 2026-05-27T22:20:37.354Z
Learning: In NVIDIA-NeMo/Safe-Synthesizer, `tests/conftest.py`’s `pytest_collection_modifyitems` hook applies pytest category markers automatically based on each test file’s path: tests under `/e2e/` get `pytest.mark.e2e`, tests under `/smoke/` get `pytest.mark.smoke`, and all other tests get `pytest.mark.unit`. Therefore, when reviewing pytest tests outside `tests/e2e/` and `tests/smoke/`, do not flag missing explicit `pytest.mark.unit` decorators on test classes/functions as an issue (the hook will add them during collection). If a new test directory/category is introduced, ensure the hook is updated so it’s categorized correctly.
Applied to files:
tests/cli/test_config.pytests/sdk/test_config_builder.pytests/cli/test_utils.pytests/config/test_parameters.py
🪛 Ruff (0.15.21)
src/nemo_safe_synthesizer/config/parameters.py
[warning] 138-138: Do not call setattr with a constant attribute value. It is not any safer than normal property access.
Replace setattr with assignment
(B010)
🔇 Additional comments (10)
src/nemo_safe_synthesizer/config/parameters.py (1)
8-10: LGTM!Also applies to: 40-42, 99-137, 140-163, 264-319
src/nemo_safe_synthesizer/config/job.py (1)
6-6: LGTM!Also applies to: 32-39
src/nemo_safe_synthesizer/configurator/parameters.py (1)
34-34: LGTM!Also applies to: 78-93, 116-116
src/nemo_safe_synthesizer/sdk/config_builder.py (1)
9-9: LGTM!Also applies to: 57-57, 68-68, 83-91, 114-114, 127-129, 142-144, 157-159, 174-176, 222-226, 243-245, 279-279
src/nemo_safe_synthesizer/sdk/library_builder.py (1)
300-300: LGTM!tests/config/test_parameters.py (1)
14-14: LGTM!Also applies to: 110-120, 131-135, 262-375
tests/sdk/test_config_builder.py (1)
89-107: LGTM!tests/cli/test_config.py (1)
127-144: LGTM!tests/cli/test_utils.py (1)
378-382: LGTM!docs/user-guide/configuration.md (1)
22-42: LGTM!Also applies to: 141-145
Greptile SummaryThis PR adds
Confidence Score: 4/5Safe to merge; the implementation is thorough and the previously reported resume-path limitation is correctly resolved. Two minor edge cases around API ordering and double-load semantics are worth noting but do not affect any documented workflow. The strict-config strictness is applied consistently across all input paths. The prior concern about legacy configs being unloadable on resume is addressed by injecting the override into the raw dict before validation. Two observations remain: the with_strict_config ordering constraint is undocumented, and _strict_config_explicit is set from model_fields_set post-load, which conflates serialized defaults with intentional overrides in repeated-load scenarios. Neither affects the tested workflows. library_builder.py lines 311-314 for the _strict_config_explicit post-load semantics; config_builder.py with_strict_config docstring for the ordering constraint. Important Files Changed
Reviews (3): Last reviewed commit: "test(config): cover persisted resume pol..." | Re-trigger Greptile |
Signed-off-by: Yunfeng Zhang <yunzhang@nvidia.com>
|
Addressed the Greptile resume-path concern in commit 842b9a1. An explicitly set runtime or builder strictness policy now applies before saved JSON validation, with regression tests covering default rejection and both non-strict opt-out paths. |
There was a problem hiding this comment.
Actionable comments posted: 1
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 28e305e4-fd5c-494c-96b8-11aa4cec7ecb
📒 Files selected for processing (7)
docs/developer-guide/configuration_management.mddocs/user-guide/configuration.mdsrc/nemo_safe_synthesizer/config/parameters.pysrc/nemo_safe_synthesizer/sdk/config_builder.pysrc/nemo_safe_synthesizer/sdk/library_builder.pytests/config/test_parameters.pytests/sdk/test_process_data.py
🚧 Files skipped from review as they are similar to previous changes (5)
- docs/developer-guide/configuration_management.md
- docs/user-guide/configuration.md
- tests/config/test_parameters.py
- src/nemo_safe_synthesizer/sdk/config_builder.py
- src/nemo_safe_synthesizer/config/parameters.py
📜 Review details
⏰ Context from checks skipped due to timeout. (7)
- GitHub Check: Unit Tests (3.11)
- GitHub Check: Unit Tests (3.12)
- GitHub Check: End-user Wheel Install
- GitHub Check: Unit Tests (3.13)
- GitHub Check: Smoke Tests
- GitHub Check: Greptile Review
- GitHub Check: Analyze (python)
🧰 Additional context used
📓 Path-based instructions (9)
**/*.{md,markdown,py}
📄 CodeRabbit inference engine (.cursor/rules/agent-markdown-style.mdc)
**/*.{md,markdown,py}: Avoid decorative bold (**text**) in list items, body text, and docstrings; use structural cues (headers, list markers, colons, backticks) for emphasis instead
Use backticks for code identifiers, paths, and CLI commands in markdown and docstrings
Files:
src/nemo_safe_synthesizer/sdk/library_builder.pytests/sdk/test_process_data.py
**/*.py
📄 CodeRabbit inference engine (AGENTS.md)
**/*.py: Place durable implementation guidance in function and class docstrings for public contracts and source comments for local invariants
Target Python 3.11–3.13 with modern syntax (X | Y,list[str],Self). Python 3.14+ is not supported
**/*.py: Use American English spelling in Python code, documentation, and messages.
UseField(description=...)for every Pydantic model field.
Use assignment-styleField()by default; useAnnotatedonly for additional metadata such as validators, constrained aliases, or discriminated unions.
Use@dataclass(frozen=True)for immutable value objects and validators; use mutable dataclasses only for builders, accumulators, and pipeline state.
Usefield(default_factory=list)instead of mutable list defaults.
UseStrEnumfor string-valued configuration or serialization enums and plainEnumfor internal constants.
Obtain loggers withobservability.get_logger(__name__); do not calllogging.getLogger()orstructlog.get_logger()directly.
Use.runtime,.user, and.systemcategory loggers appropriately.
Do not useprint()for operational library output; use the approved logger,click.echo(), orsys.stdout.write()where appropriate.
Useextra={}for machine-queryable logging data and f-strings only for human-readable context.
Raise errors from the custom Safe Synthesizer error hierarchy, using the documented dual inheritance for user and internal errors.
Keep shared package code compatible with Python 3.11; do not use Python 3.12-only syntax such as PEP 695 type statements or bracketed generic parameters.
PreferX | Y, built-in collection generics, andSelfoverOptional,Union, and legacy typing collections.
Use collection ABCs for function arguments and concrete collection types for return values.
UseProtocolfor structural subtyping and avoidAnywhenobject, generics, or protocols are suitable.
UseTYPE_CHECKINGguards for heavy imports such as pandas, torch, and transformers.
...
Files:
src/nemo_safe_synthesizer/sdk/library_builder.pytests/sdk/test_process_data.py
src/**/*.py
📄 CodeRabbit inference engine (STYLE_GUIDE.md)
src/**/*.py: Use relative imports for package code undersrc/.
Do not useassertfor validation in library code; raise an appropriate exception instead.
Files:
src/nemo_safe_synthesizer/sdk/library_builder.py
⚙️ CodeRabbit configuration file
Review library code against STYLE_GUIDE.md. Focus on behavior, API contracts, error handling, resource cleanup, typing, logging, and user-facing failures. Public APIs and nontrivial functions need Google-style docstrings.
Files:
src/nemo_safe_synthesizer/sdk/library_builder.py
**/*
📄 CodeRabbit inference engine (STYLE_GUIDE.md)
**/*: Every source file requires the SPDX copyright and license header appropriate to its file format.
End files with a newline, remove trailing whitespace, use one space between sentences, and keep code, comments, and docstrings within 120 characters.
**/*: All merged commits must use Conventional Commits format:<type>(<scope>): <description>or<type>: <description>, with a lowercase valid type and a description no longer than 100 characters.
All contributions require a DCOSigned-off-bytrailer and a cryptographic commit signature.
Before submitting a pull request, run formatting, checks, and tests usingmise run format,mise run check, andmise run test.
Branches other thanmainmust use lowercase author-prefixed names in one of the documented forms, optionally including an issue ID and category.
Release tags must use avprefix and PEP 440 stable or release-candidate versions, such asv1.0.0orv0.1.0rc0; alpha versions and dashed-rcsuffixes are not used.
Do not move a published release tag; create and validate a new release candidate when code changes.
Usemise run <task>for project tasks; the Makefile only bootstraps mise and provides deprecated compatibility messages.
Files:
src/nemo_safe_synthesizer/sdk/library_builder.pytests/sdk/test_process_data.py
⚙️ CodeRabbit configuration file
**/*: Review as a senior maintainer for NeMo Safe Synthesizer. Prioritize issues that can change behavior, break user workflows, weaken privacy guarantees, hide failures, make tests unreliable, or create maintenance risk. Avoid generic style commentary unless it points to a concrete project convention that automated tools will not catch.
Comment only when the finding is actionable and tied to changed code. For each finding, state the impact, the condition that triggers it, and the smallest practical fix. Prefer one precise comment over broad advice. Do not ask for refactors outside the PR scope unless the changed code creates the problem.
Review type guidance: - Potential issue: use for correctness bugs, data loss, privacy leaks,
security risks, broken public APIs, invalid config behavior, missing
validation, hidden failures, nondeterministic tests, or CI breakage.
- Refactor suggestion: use for local maintainability problems introduced
by the diff when they have clear future cost, such as duplicated setup,
unclear boundaries, over-mocking, avoidable complexity, or opaque test
helpers.- Nitpick: avoid in chill mode. Do not emit formatting, import-order,
wording, or style-only comments unless automated tools cannot catch the
issue and it affects maintainability.Severity guidance: - Critical: security/privacy leaks, data loss, training/test/holdout
contamination, or broken release/package/core pipeline execution.
- Major: incorrect generation/training/evaluation behavior, broken
CLI/SDK public API, invalid config defaults or validators, or GPU/vLLM
cleanup and process-isolation bugs likely to fail CI or production
runs.- Minor: localized bugs, missing focused tests for changed behavior, or
bad test patterns that weaken regression coverage.- Trivial: small cleanup with no behavior impact. Usually suppress in
chill mode.- Info: context only. Avoid unless it helps reviewers understand risk.
Safe-Synthesizer-specific review focus: - Data ...
Files:
src/nemo_safe_synthesizer/sdk/library_builder.pytests/sdk/test_process_data.py
**/*.{py,sh,yaml,yml,md}
📄 CodeRabbit inference engine (CONTRIBUTING.md)
All source files with
.py,.sh,.yaml,.yml, or.mdextensions must include SPDX copyright headers, except files listed in.copyrightignore.
Files:
src/nemo_safe_synthesizer/sdk/library_builder.pytests/sdk/test_process_data.py
src/nemo_safe_synthesizer/**/*.py
📄 CodeRabbit inference engine (CONTRIBUTING.md)
Write durable module-level guidance in Python docstrings and source comments so it appears in the generated API reference.
Files:
src/nemo_safe_synthesizer/sdk/library_builder.py
**/test_*.py
📄 CodeRabbit inference engine (AGENTS.md)
Use the
unitmarker instead of the deprecatedunit_testmarker for test identificationRun tests through the repository's mise tasks, such as
mise run test, rather than relying on unpinned standalone commands.
Files:
tests/sdk/test_process_data.py
tests/**
📄 CodeRabbit inference engine (.cursor/rules/repo-navigation.mdc)
tests/**: Mirrorsrc/directory structure intests/directory for test organization
Auto-mark tests by directory:tests/e2e/→e2e,tests/smoke/→smoke, otherwise default tounitMirror source code directory structure in tests directory (e.g.,
tests/training/,tests/generation/parallel to source structure)
Files:
tests/sdk/test_process_data.py
tests/**/*.py
📄 CodeRabbit inference engine (tests/TESTING.md)
tests/**/*.py: Auto-mark tests based on file path: tests under/e2e/gete2emarker, tests under/smoke/getsmokemarker, all others getunitmarker (only if no category marker already present)
Every test should have exactly one category marker:unit,smoke, ore2e
Usepytest.mark.requires_gpumodifier on tests that need CUDA hardware
Usepytest.mark.vllmon tests using vLLM generation backend and ensure each vLLM test file runs in its own process for GPU memory isolation
Usepytest.mark.slowon long-running tests
Usepytest.mark.smollm2for SmolLM2 Hub download tests to enable process isolation
Usepytest.mark.noautouseto skip autouse fixtures for specific tests
Useload_test_dataset(filename)helper to load test datasets fromtests/stub_datasets/as HuggingFaceDatasetobjects
Useload_test_dataframe(filename)helper to load test data files fromtests/stub_datasets/as pandas DataFrames
Convert pandas columns to nullable dtypes (pd.Int64Dtype(),pd.BooleanDtype()) before assigningnp.nanvalues
Usefake.seed_instance(seed)andrandom.seed(seed)together for Faker-based test data reproducibility
When sharing methods across multiple test files, define them inconftest.pyand import them using relative imports (e.g.,from .conftest import train_with_sdk); note that importing from other test files liketests/cli/helpers.pydoes not work
Usefixture_mock_processororfixture_mock_processor_without_valid_recordsfor mocking ParsedResponse objects withvalid_records,invalid_records,errors, andprompt_numberfields
Usepytest.importorskipto gate tests on optional dependencies that require specific extras (e.g.,sentence_transformers,vllm)
Run vLLM tests with separate pytest invocations (one per file) using-n 0(single process) for GPU memory isolation, or use staged mise tasks for CI visibility
Print statements are allowed in tests (ruffT201is suppressed fortests/directory) and should...
Files:
tests/sdk/test_process_data.py
⚙️ CodeRabbit configuration file
Review tests against tests/TESTING.md. Check marker usage, fixture naming, tmp_path usage, determinism, and GPU/vLLM process-isolation requirements. Flag slop tests that only check that code runs, assert result is not None when stronger invariants exist, over-mock internal implementation details, patch around the bug instead of reproducing it, or add broad snapshot/golden churn without a clear contract. Flag change detector tests that fail on harmless refactors, formatting, record ordering, incidental wording, or private implementation details without demonstrating a behavior regression. Prefer existing fixtures or focused new fixtures for repeated setup; keep tests DRY when reasonable without making the behavior under test opaque. print() is allowed in tests.
Files:
tests/sdk/test_process_data.py
🧠 Learnings (1)
📚 Learning: 2026-05-27T22:20:37.354Z
Learnt from: kendrickb-nvidia
Repo: NVIDIA-NeMo/Safe-Synthesizer PR: 520
File: tests/generation/test_vllm_backend.py:556-587
Timestamp: 2026-05-27T22:20:37.354Z
Learning: In NVIDIA-NeMo/Safe-Synthesizer, `tests/conftest.py`’s `pytest_collection_modifyitems` hook applies pytest category markers automatically based on each test file’s path: tests under `/e2e/` get `pytest.mark.e2e`, tests under `/smoke/` get `pytest.mark.smoke`, and all other tests get `pytest.mark.unit`. Therefore, when reviewing pytest tests outside `tests/e2e/` and `tests/smoke/`, do not flag missing explicit `pytest.mark.unit` decorators on test classes/functions as an issue (the hook will add them during collection). If a new test directory/category is introduced, ensure the hook is updated so it’s categorized correctly.
Applied to files:
tests/sdk/test_process_data.py
🪛 ast-grep (0.44.1)
tests/sdk/test_process_data.py
[info] 499-499: use jsonify instead of json.dumps for JSON output
Context: json.dumps(saved_config)
Note: [CWE-116] Improper Encoding or Escaping of Output.
(use-jsonify)
[info] 526-526: use jsonify instead of json.dumps for JSON output
Context: json.dumps(saved_config)
Note: [CWE-116] Improper Encoding or Escaping of Output.
(use-jsonify)
🔇 Additional comments (5)
src/nemo_safe_synthesizer/sdk/library_builder.py (3)
8-8: LGTM!
280-284: LGTM!
298-313: LGTM!tests/sdk/test_process_data.py (2)
15-21: LGTM!
483-507: LGTM!
Signed-off-by: Yunfeng Zhang <yunzhang@nvidia.com>
Summary
strict_config: falsecompatibility opt-outTest plan
mise run checkCloses #448
Summary by CodeRabbit
New Features
strict_configto control recursive unknown configuration key handling (reject vs ignore).with_strict_config(...), including propagation into resolved configs.Documentation
strict_config.Bug Fixes
Tests