51 make release203 by aslgraefe · Pull Request #52 · BIH-CEI/rd-cdm

aslgraefe · 2026-03-24T16:22:01Z

No description provided.

…BioPortal (i.e. the preferreld labels)

Copilot

Pull request overview

This PR updates RD-CDM to release v2.0.3, refactors the repository away from versioned instances/vX_Y_Z/ folders to a flat src/rd_cdm/instances/ layout, and updates the CLI/export/validation utilities and documentation accordingly.

Changes:

Flatten instances layout and standardize on rd_cdm.yaml / jsons/rd_cdm.json / csvs/rd_cdm.csv with embedded rd_cdm_version + rd_cdm_date.
Update validation tooling to read the merged YAML directly (and add tqdm progress bars).
Bump versions / regenerate shipped artifacts (schema, instances, exports, generated Python classes) for 2.0.3.

Reviewed changes

Copilot reviewed 33 out of 37 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
tests/test_versioning.py	Updates tests for non-versioned instances dir resolution.
tests/test_merge_instances.py	Updates merge tests to expect `rd_cdm.yaml` and schema-derived metadata.
tests/test_json_parsing.py	Updates JSON export tests to expect `jsons/rd_cdm.json`.
tests/test_csv_parsing.py	Updates CSV export tests to expect `csvs/rd_cdm.csv` with `_metadata` row.
src/rd_cdm/utils/versioning.py	Simplifies instance dir resolution to flat `instances/` (but currently leaves a broken `get_model_version()`).
src/rd_cdm/utils/validation.py	Refactors validation to read merged YAML directly; adds tqdm progress display.
src/rd_cdm/utils/settings.py	Migrates settings config style (but `.env` behavior now mismatches comment).
src/rd_cdm/utils/merge_instances.py	Merges part YAMLs into `rd_cdm.yaml` and stamps schema version/date.
src/rd_cdm/utils/json_parsing.py	Exports merged `rd_cdm.yaml` to `jsons/rd_cdm.json`.
src/rd_cdm/utils/gen_pydantic.py	Generates both LinkML dataclasses + Pydantic v2 models; strips unsupported schema fields.
src/rd_cdm/utils/csv_parsing.py	Exports merged YAML to per-section CSVs + combined `rd_cdm.csv` with metadata row.
src/rd_cdm/utils/config.py	Simplifies path resolution for flat instances layout.
src/rd_cdm/schema/rd_cdm.yaml	Adds schema metadata (name/title/version/date) + new required `rd_cdm_version`/`rd_cdm_date` attributes.
src/rd_cdm/python_classes/rd_cdm_pydantic.py	Regenerated Pydantic models including new schema fields (contains absolute paths).
src/rd_cdm/python_classes/rd_cdm.py	Regenerated LinkML dataclasses including new schema fields (contains absolute paths; duplicate imports).
src/rd_cdm/instances/value_sets.yaml	Normalizes label capitalization for specific LOINC labels.
src/rd_cdm/instances/v2_0_2/jsons/rd_cdm_v2_0_2.json	Removes old versioned combined JSON artifact.
src/rd_cdm/instances/v2_0_2/csvs/rd_cdm_v2_0_2.csv	Removes old versioned combined CSV artifact.
src/rd_cdm/instances/rd_cdm.yaml	Adds embedded model version/date and updates code system versions + one data element code.
src/rd_cdm/instances/jsons/value_sets.json	Adds/updates shipped JSON export (note: file still uses legacy structure for `codes`).
src/rd_cdm/instances/jsons/data_elements.json	Updates shipped JSON export for changed data element coding.
src/rd_cdm/instances/jsons/code_systems.json	Updates shipped JSON export for updated code system versions.
src/rd_cdm/instances/data_elements.yaml	Updates “Sex at Birth” coding from SNOMEDCT to LOINC.
src/rd_cdm/instances/csvs/value_sets.csv	Updates shipped CSV export label capitalization.
src/rd_cdm/instances/csvs/data_elements.csv	Updates shipped CSV export for changed data element coding.
src/rd_cdm/instances/csvs/code_systems.csv	Updates shipped CSV export for updated code system versions.
src/rd_cdm/instances/code_systems.yaml	Updates ontology/code system version strings.
res/.DS_Store	Adds macOS metadata file (should not be committed).
pyproject.toml	Bumps package version to 2.0.3; updates included file patterns; adjusts deps and renames scripts to `rd-cdm-*`.
docs/conf.py	Updates Sphinx release to 2.0.3 and removes numpy doctest setup.
docs/changelog.rst	Adds detailed v2.0.3 changelog entry describing refactor and version updates.
README.md	Updates docs to match new layout and CLI command names.
.gitignore	Adds `.DS_Store` ignore rules.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-03-24T16:26:45Z

 class ValidationSettings(BaseSettings):
    """
    Environment-driven settings for validation steps.
    """
-    bioportal_api_key: str = ""  # set in CI/locally; validation step will check emptiness
+    model_config = ConfigDict(env_prefix="", extra="ignore")

-    class Config:
-        env_prefix = ""   # BIOPORTAL_API_KEY
-        env_file = ".env"
+    bioportal_api_key: str = ""  # set via BIOPORTAL_API_KEY env var or .env file


ValidationSettings no longer configures an env_file, but the field comment says the API key can come from a .env file. Either configure pydantic-settings to load .env (e.g., via the settings-specific config) or update the comment/docs so expectations match actual behavior.

Copilot · 2026-03-24T16:26:45Z

+def get_model_version() -> Optional[str]:
+    root = Path(__file__).resolve().parents[2]
+    return _read_project_version(root / "pyproject.toml")


get_model_version() calls _read_project_version(), but _read_project_version is no longer defined in this module, so calling get_model_version() will raise NameError. Either reintroduce _read_project_version (and ensure it points at the correct repo root) or remove get_model_version entirely if it’s no longer part of the supported API.

Copilot · 2026-03-24T16:26:45Z

+    def load_file(name: str) -> dict:
        p = paths.instances_dir / name
        if not p.exists():
            print(f"ERROR: missing required file: {p}", file=sys.stderr)
            sys.exit(1)
        with p.open("r", encoding="utf-8") as fh:
-            return yaml.load(fh)
+            return yaml.load(fh) or {}


load_file() calls sys.exit(1) when an input YAML is missing. Since main() is designed to return an int status code (and is called from tests / could be reused programmatically), exiting here prevents callers from handling the error and makes the return codes inconsistent. Prefer raising an exception or returning a non-zero status from main() instead of calling sys.exit() in a helper.

Copilot · 2026-03-24T16:26:46Z

-# AUTO-GENERATED by src/rd_cdm/utils/gen_pydantic.py
-# Schema: /Users/adam/Documents/git/rd-cdm/src/rd_cdm/schema/rd_cdm.yaml
+# AUTO-GENERATED by src/rd_cdm/utils/gen_pydantic.py (PydanticGenerator)
+# Schema: /Users/adamgraefe/Documents/git/rd-cdm/src/rd_cdm/schema/rd_cdm.yaml


This generated file embeds an absolute local filesystem path in the header (# Schema: /Users/...), which is non-reproducible across machines and leaks developer workstation details into the repo. Adjust the generator (gen_pydantic.py) to write a repo-relative schema path (or omit the path) in generated headers (and ideally avoid persisting temporary file paths in linkml_meta.source_file as well).

Suggested change

# Schema: /Users/adamgraefe/Documents/git/rd-cdm/src/rd_cdm/schema/rd_cdm.yaml

# Schema: src/rd_cdm/schema/rd_cdm.yaml

Copilot · 2026-03-24T16:26:46Z

+# Schema: /Users/adamgraefe/Documents/git/rd-cdm/src/rd_cdm/schema/rd_cdm.yaml
+# Do not edit manually — re-run gen_pydantic.py to regenerate.
+
+# Auto generated from tmpe_1_xxje.yaml by pythongen.py version: 0.0.1


This generated file embeds an absolute local filesystem path in the header (# Schema: /Users/...), which is non-reproducible across machines and leaks developer workstation details into the repo. Consider updating gen_pydantic.py to write repo-relative paths (or omit paths) in generated headers, and avoid carrying temp file paths into generated metadata when possible.

Suggested change

# Schema: /Users/adamgraefe/Documents/git/rd-cdm/src/rd_cdm/schema/rd_cdm.yaml

# Do not edit manually — re-run gen_pydantic.py to regenerate.

# Auto generated from tmpe_1_xxje.yaml by pythongen.py version: 0.0.1

# Schema: src/rd_cdm/schema/rd_cdm.yaml

# Do not edit manually — re-run gen_pydantic.py to regenerate.

# Auto generated from rd_cdm.yaml by pythongen.py version: 0.0.1

Copilot · 2026-03-24T16:26:46Z


-from linkml_runtime.linkml_model.types import Curie
-from linkml_runtime.utils.metamodelcore import URI
+from linkml_runtime.linkml_model.types import Curie, Date, String, Uri


Curie is imported twice from two different modules on consecutive lines, and the second import shadows the first. If these types differ, this can subtly change runtime behavior and type expectations. Since this is generated code, it’s best fixed in the generator/template so only one Curie symbol is imported (or the imports are aliased).

Suggested change

from linkml_runtime.linkml_model.types import Curie, Date, String, Uri

from linkml_runtime.linkml_model.types import Curie as TypesCurie, Date, String, Uri

aslgraefe added 8 commits March 24, 2026 15:38

updated gitignor for ds_store removal

8bdb2a9

updated the code systems versions to align with bioportal

23a720d

updated labels of two value set items to match the ones fetched from …

ace1be3

…BioPortal (i.e. the preferreld labels)

updated dependencies in project toml

3844e4f

updated readme to v2.0.3

ef6799b

updated changelog for v2.0.3

a942cf9

updated conf py of docs with new version

6cf0110

updated version to v2.0.3

def957d

aslgraefe linked an issue Mar 24, 2026 that may be closed by this pull request

Make Release/2.0.3 #51

Closed

aslgraefe requested a review from Copilot March 24, 2026 16:22

Copilot started reviewing on behalf of aslgraefe March 24, 2026 16:22 View session

aslgraefe merged commit 7b84ea6 into develop Mar 24, 2026
5 checks passed

aslgraefe deleted the 51-make-release203 branch March 24, 2026 16:22

Copilot AI reviewed Mar 24, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

51 make release203#52

51 make release203#52
aslgraefe merged 8 commits into
developfrom
51-make-release203

aslgraefe commented Mar 24, 2026

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Mar 24, 2026

Uh oh!

Copilot AI Mar 24, 2026

Uh oh!

Copilot AI Mar 24, 2026

Uh oh!

Copilot AI Mar 24, 2026

Uh oh!

Copilot AI Mar 24, 2026

Uh oh!

Copilot AI Mar 24, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

	# Schema: /Users/adamgraefe/Documents/git/rd-cdm/src/rd_cdm/schema/rd_cdm.yaml
	# Schema: src/rd_cdm/schema/rd_cdm.yaml

	from linkml_runtime.linkml_model.types import Curie, Date, String, Uri
	from linkml_runtime.linkml_model.types import Curie as TypesCurie, Date, String, Uri

Uh oh!

Conversation

aslgraefe commented Mar 24, 2026

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Mar 24, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 24, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 24, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 24, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 24, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 24, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants