diff --git a/.DS_Store b/.DS_Store deleted file mode 100755 index a422ca80f..000000000 Binary files a/.DS_Store and /dev/null differ diff --git a/.gitignore b/.gitignore index bcd69d2da..940f0a9ec 100755 --- a/.gitignore +++ b/.gitignore @@ -170,13 +170,18 @@ templates/ # Ignore template files from Public folder Public/ +# ignore dev folders +dev/ + # Temporary REDCap test config *.rarelink_test_redcap_config.json # DS Store +# macOS metadata +.DS_Store +**/.DS_Store .DS_Store/ .DS_Store? -.DS_Store .src/.DS_Store? .src/rarelink/.DS_Store? .src/rarelink/utils/.DS_Store? diff --git a/README.md b/README.md index 16a600fe8..738531572 100644 --- a/README.md +++ b/README.md @@ -15,7 +15,7 @@ #### Packages & Compatibility [![REDCap](https://img.shields.io/badge/REDCap-API-darkred.svg)](https://www.project-redcap.org/) -[![RD-CDM](https://img.shields.io/badge/RD--CDM-v2.0.2-blue.svg)](https://github.com/BIH-CEI/rd-cdm) +[![RD-CDM](https://img.shields.io/badge/RD--CDM-v2.0.3-blue.svg)](https://github.com/BIH-CEI/rd-cdm) [![Phenopackets](https://img.shields.io/badge/Phenopackets-v2.0-purple.svg)](https://phenopacket-schema.readthedocs.io/en/latest/) [![LinkML](https://img.shields.io/badge/LinkML-1.9.0+-green.svg)](https://linkml.io/) [![HL7 FHIR](https://img.shields.io/badge/HL7%20FHIR-R4-orange.svg)](https://hl7.org/fhir/) @@ -116,37 +116,38 @@ Documentation on [Setting Up a REDCap Project](https://rarelink.readthedocs.io/e ## Installation -RareLink can be set up using various Python project management approaches. One - common method is to use a virtual environment. Below is an example where the - virtual environment is named `rarelink-venv`, but you can name it as you prefer: - +RareLink requires Python ≥3.10 and <3.13. ```bash python3.12 -m venv .venv source .venv/bin/activate pip install --upgrade pip ``` -> **Note**: You need Python ≥3.10 but <3.13 to be able to use RareLink! - -Next, install rarelink through pypi... - +### Core install ```bash pip install rarelink ``` -... or clone the RareLink repository, navigate to its root directory, and - install RareLink using: +This installs everything needed for REDCap interaction, Phenopacket export, +and the RareLink CLI. It does **not** install Docker, numpy/pandas, or +documentation/testing tools. -```bash -git clone https://github.com/BIH-CEI/rarelink.git -cd rarelink -pip install . -``` +### Optional extras -If you want to install development dependencies (e.g., `pytest`), use: +| Extra | Install command | What it adds | +|-------|----------------|--------------| +| `fhir` | `pip install rarelink[fhir]` | Docker SDK for the toFHIR FHIR export pipeline | +| `data` | `pip install rarelink[data]` | numpy + pandas for tabular data processing | +| `test` | `pip install rarelink[test]` | pytest and pytest-mock | +| `docs` | `pip install rarelink[docs]` | Sphinx and documentation themes | +| `dev` | `pip install rarelink[dev]` | All of the above plus build/twine/jupyter | +For development: ```bash -pip install .[dev] +git clone https://github.com/BIH-CEI/rarelink.git +cd rarelink +pip install -e .[dev] +pytest ``` ### Setting Up the `.env` File diff --git a/docs/.DS_Store b/docs/.DS_Store deleted file mode 100755 index 10f8d71be..000000000 Binary files a/docs/.DS_Store and /dev/null differ diff --git a/docs/2_rarelink_framework/2_1_rarelink_overview.rst b/docs/2_rarelink_framework/2_1_rarelink_overview.rst index bf6394f40..d849cce9b 100755 --- a/docs/2_rarelink_framework/2_1_rarelink_overview.rst +++ b/docs/2_rarelink_framework/2_1_rarelink_overview.rst @@ -16,17 +16,17 @@ GA4GH Phenopackets, without the need for additional coding or mapping. As a result, RareLink allows researchers and clinicians to collect and process RD data seamlessly within the REDCap environment. -.. figure:: ../_static/res/rarelink_overview.svg +.. figure:: ../_static/res/rarelink_overview.png Overview of the RareLink framework integrated with a local REDCap instance. - The framework comprises the RareLink instruments based on the RD-CDM, the - RareLink API, the toFHIR & CDIS module for export and import to HL7 FHIR IPS - and Genomic Reporting Resources, and the RareLink-Phenopackets module for + The framework comprises the REDCap instruments based on the `RD-CDM `_, + the RareLink API, the toFHIR & CDIS module for export and import to HL7 FHIR + IPS and Genomic Reporting Resources, and the RareLink-Phenopackets module for direct export to GA4GH Phenopackets. Additionally, the LinkML-based import mapper facilitates data import from tabular databases into REDCap, while the Manual Data Capture Guide assists with the manual data capture of - RD-CDM data in a local REDCap project. Click `here `_ - to see the figure in full size. + RD-CDM data in a local REDCap project. The framework is also configurable + for disease-specific extensions or custom data models. .. note:: You will need your own REDCap instance to deploy the RareLink framework. diff --git a/docs/2_rarelink_framework/2_2_rarelink_cdm.rst b/docs/2_rarelink_framework/2_2_rarelink_cdm.rst index a6d3d3f07..9fdc2b047 100755 --- a/docs/2_rarelink_framework/2_2_rarelink_cdm.rst +++ b/docs/2_rarelink_framework/2_2_rarelink_cdm.rst @@ -161,7 +161,7 @@ Example Field Annotation of 6.2.6 Temporal Pattern: - HP:0012829 | Profound - HP:0012828 | Severe Version(s): - - HPO Version 2025-05-06 + - HPO Version 2026-02-16 Mapping: - HL7 FHIR Expression v4.0.1: Observation.interpretation - GA4GH Phenopacket Schema v2.0 Element: PhenotypicFeature.severity @@ -315,7 +315,7 @@ ______________________________________________________________ - Cardinality: Required (1..1) - Validation: Date format (YYYY-MM-DD) -- 2.2 Sex at Birth (``snomedct_281053000``) +- 2.2 Sex at Birth (``loinc_76689_9``) - Cardinality: Optional - Validation: Dropdown (choices encoded according to :ref:`1_5`) diff --git a/docs/3_installation/3_1_setup_rarelink_framework.rst b/docs/3_installation/3_1_setup_rarelink_framework.rst index f8d8e1952..8e082d737 100755 --- a/docs/3_installation/3_1_setup_rarelink_framework.rst +++ b/docs/3_installation/3_1_setup_rarelink_framework.rst @@ -24,18 +24,34 @@ Follow these steps to set up the project locally and run tests. source .venv/bin/activate # On macOS/Linux .venv\Scripts\activate # On Windows -3. Install dependencies: +3. Install RareLink: .. code-block:: bash - pip install . + pip install rarelink + +This installs everything needed for REDCap interaction, Phenopacket export, +and the RareLink CLI. + +For optional features, install the relevant extras: + +.. code-block:: bash + + pip install rarelink[fhir] # adds Docker SDK for the toFHIR pipeline + pip install rarelink[data] # adds numpy and pandas for tabular data processing + +For development (includes testing, docs, and all optional extras): + +.. code-block:: bash + + pip install -e .[dev] 4. Configure all api keys necessary to use rarelink by running the following command: .. code-block:: bash - rarelink setup keys + rarelink setup keys This command will prompt you to enter the following keys: - BioPortal API key @@ -43,14 +59,14 @@ command: - REDCap URL - REDCap project ID -.. note:: +.. note:: You can create your free BioPortal account here: `BioPortal `_ 5. Run tests: - Use `pytest` to run the test suite. - + .. code-block:: bash + pip install rarelink[test] pytest _____________________________________________________________________________________ @@ -58,7 +74,7 @@ ________________________________________________________________________________ RareLink Framework CLI config ------------------------------------ -Use the following commands to update the framework and its components, view +Use the following commands to update the framework and its components, view its status or reset the framework. See :ref:`2_3` for more information. .. code-block:: bash @@ -91,32 +107,37 @@ ________________________________________________________________________________ Phenopacket Pipeline Configuration ___________________________________ -Via the RareLink CLI type: +The Phenopacket export pipeline requires a BioPortal API key and a +``CREATED_BY`` value, both configured via: .. code-block:: bash - to be implemented + rarelink setup keys -This command guides you through setting up the Phenopacket pipeline for RareLink. -You will be prompted to enter: -- Your location where to store the Phenopackets. +Once configured, export Phenopackets with: + +.. code-block:: bash + + rarelink phenopackets export --help _____________________________________________________________________________________ FHIR Pipeline Configuration ___________________________ +.. note:: + The FHIR pipeline requires the ``fhir`` extra: ``pip install rarelink[fhir]`` + .. code-block:: bash rarelink fhir setup -This command guides you through setting up the FHIR pipeline for RareLink. +This command guides you through setting up the FHIR pipeline for RareLink. You will be prompted to enter: - Your FHIR server URL. - If required, your FHIR server username & password. -.. note:: - All sensitive information will also be stored in the - hidden configuration file. - +.. note:: + All sensitive information will also be stored in the + hidden configuration file. \ No newline at end of file diff --git a/docs/4_user_guide/4_1_manual_data_capture.rst b/docs/4_user_guide/4_1_manual_data_capture.rst index 156f98968..828975288 100755 --- a/docs/4_user_guide/4_1_manual_data_capture.rst +++ b/docs/4_user_guide/4_1_manual_data_capture.rst @@ -354,7 +354,8 @@ Fields: 1. Validate mutations using `ClinVar `_ or `Varsome `_. - 2. Confirm the expression with the `HGVS Validator `_. + 2. Confirm the expression with the `Variant Validator `_ + or the `HGVS Validator `_. 3. If the validation fails, enter the details in the **6.1.6 Genetic Mutation String** field. 4. ``*``: **Are you sure the entered HGVS expression was validated using the variant validator?** - this field is mandatory to ensure the diff --git a/docs/4_user_guide/4_2_import_mapper.rst b/docs/4_user_guide/4_2_import_mapper.rst index 0dc70de95..6ce11f479 100755 --- a/docs/4_user_guide/4_2_import_mapper.rst +++ b/docs/4_user_guide/4_2_import_mapper.rst @@ -187,7 +187,7 @@ The template contains the following columns: RareLink-CDM schema, for example: - ``rarelink_cdm:snomedct_184099003`` for 2.1 Date of birth, or - - ``rarelink_cdm:snomedct_281053000`` for 2.2 Sex at birth. + - ``rarelink_cdm:loinc_76689_9`` for 2.2 Sex at birth. - **predicate_label**: The title of the target element from the schema, i.e. the *name* of the element @@ -198,7 +198,7 @@ The template contains the following columns: The target value in the RareLink-CDM. For literal fields, such as dates, this is the actual value (e.g., "1970-02-15"). For coded fields, such as sex, the object_id is the standardized code from the value set - (e.g., "SNOMEDCT:248152002" for female) or the code from the code system defined + (e.g., "LOINC:76689-9" for female) or the code from the code system defined by the data element (e.g., "HP:0001250" for a phenotype in element 6.2.1). - **object_label**: The human-readable label for the target value either from the @@ -255,7 +255,7 @@ Below you can find a **snippet of the RareLink-CDM SSSOM Template File:** # # === Personal Information (Singular) === "" rarelink_cdm:snomedct_184099003 "2.1 Date of birth" - lexical manual - "" rarelink_cdm:snomedct_281053000 "2.2 Sex at birth" lexical manual + "" rarelink_cdm:loinc_76689_9 "2.2 Sex at birth" lexical manual "" rarelink_cdm:snomedct_1296886006 "2.3 Karyotypic Sex" lexical manual "" rarelink_cdm:snomedct_263493000 "2.4 Gender Identity" lexical manual "" rarelink_cdm:snomedct_370159000 "2.5 Country of birth" lexical manual @@ -447,7 +447,7 @@ schema. subject_id subject_label predicate_id predicate_label object_id object_label match_type mapping_date mapping_tool ADR001 "Patient ADR001" rarelink_cdm:snomedct_422549004 "1.1 Pseudonym" ADR001 ADR001 lexical 2025-01-01 manual ADR001 "Patient ADR001" rarelink_cdm:snomedct_184099003 "2.1 Date of birth" 1970-02-15 - lexical 2025-01-01 manual - ADR001 "Patient ADR001" rarelink_cdm:snomedct_281053000 "2.2 Sex at birth" SNOMEDCT:248152002 Female lexical 2025-01-01 manual + ADR001 "Patient ADR001" rarelink_cdm:loinc_76689_9 "2.2 Sex at birth" LOINC:76689-9 Female lexical 2025-01-01 manual ADR001 "Patient ADR001" rarelink_cdm:snomedct_263493000 "2.4 Gender Identity" SNOMEDCT:446141000124107 Female gender identity lexical 2025-01-01 manual ADR001 "Patient ADR001" rarelink_cdm:snomedct_278844005 "3.1 Vital Status" SNOMEDCT:438949009 Alive lexical 2025-01-01 manual ADR001 "Patient ADR001" rarelink_cdm:snomedct_64572001_mondo "5.1 Disease [MONDO]" MONDO:0012345 Wilson Disease lexical 2025-01-01 manual @@ -475,7 +475,7 @@ schema. # Mappings for a second patient (ADR002) ADR002 "Patient ADR002" rarelink_cdm:snomedct_422549004 "1.1 Pseudonym" ADR002 ADR002 lexical 2025-01-01 manual ADR002 "Patient ADR002" rarelink_cdm:snomedct_184099003 "2.1 Date of birth" 1982-07-30 1982-07-30 lexical 2025-01-01 manual - ADR002 "Patient ADR002" rarelink_cdm:snomedct_281053000 "2.2 Sex at birth" SNOMEDCT:248153007 Male lexical 2025-01-01 manual + ADR002 "Patient ADR002" rarelink_cdm:loinc_76689_9 "2.2 Sex at birth" SNOMEDCT:248153007 Male lexical 2025-01-01 manual ADR002 "Patient ADR002" rarelink_cdm:snomedct_278844005 "3.1 Vital Status" SNOMEDCT:438949009 Alive lexical 2025-01-01 manual ADR002 "Patient ADR002" rarelink_cdm:snomedct_64572001_mondo "5.1 Disease [MONDO]" MONDO:0010526 Fabry Disease lexical 2025-01-01 manual ADR002 "Patient ADR002" rarelink_cdm:loinc_99498_8 "5.2 Verification status" HL7FHIR:differential Differential lexical 2025-01-01 manual diff --git a/docs/4_user_guide/4_3_phenopackets.rst b/docs/4_user_guide/4_3_phenopackets.rst index 6a1726b82..9283e9ed4 100755 --- a/docs/4_user_guide/4_3_phenopackets.rst +++ b/docs/4_user_guide/4_3_phenopackets.rst @@ -21,9 +21,12 @@ ________________________________________________________________________________ - :ref:`phenopacket-validation` - :ref:`rarelink-phenopacket-preconfigurations` - :ref:`phenopackets-other-redcap-data-models` - - :ref:`troubleshooting` + - :ref:`phenopacket-adapters` + - :ref:`multi-onset-adapter` + - :ref:`ontology-routing-adapter` + - :ref:`troubleshooting` _____________________________________________________________________________________ .. _get_started: @@ -686,55 +689,290 @@ placeholders with relevant codes and Phenopacket terms. _____________________________________________________________________________________ -.. _factory-approach: +.. _phenopacket-adapters: -Using the Factory Approach -~~~~~~~~~~~~~~~~~~~~~~~~~~ +Phenopacket Adapters +==================== -For more advanced use cases, you can use the PhenopacketMappingFactory to create -mappings for different data models: +Adapters are preprocessing functions that transform or split data **before** +any mapper runs. They are designed to solve structural challenges that arise +when a single REDCap instrument or data element does not map cleanly to a +single Phenopacket block. All adapters are: -.. code-block:: python +- **Opt-in** — activated by a configuration key; absent means no effect. +- **Mapper-agnostic** — they produce data shaped to the conventions that + existing mappers already understand, so no mapper code changes are needed. +- **Reusable** — they are designed for general use, not tied to any specific + data model. - from rarelink.phenopackets.factory import PhenopacketMappingFactory, get_phenopacket_mappings +The adapters live in ``src/rarelink/phenopackets/adapter/``. - # Get mappings for RareLink CDM (default) - rarelink_mappings = get_phenopacket_mappings() +----- - # Get mappings for a custom data model - custom_mappings = get_phenopacket_mappings(model_name="your_model") +.. _multi-onset-adapter: - # Creating phenopackets with specific model mappings - from rarelink.phenopackets import create_phenopacket +Multi-Onset Adapter +------------------- - # Create phenopacket using the specified mappings - phenopacket = create_phenopacket( - data=record_data, - created_by="Your Name", - mapping_configs=custom_mappings - ) +**Module:** ``rarelink.phenopackets.adapter.multi_onset`` -The factory approach provides additional helper functions for working with different -data models: +**When to use it** -1. **Converting to Multi-Instrument Format** - - .. code-block:: python - - updated_config = PhenopacketMappingFactory.convert_to_multi_instrument_format( - config=mappings, - block_name="phenotypicFeatures" - ) +Use the multi-onset adapter when a single clinical finding has been observed +on **multiple occasions**, each with its own date, and you want to represent +each observation as a separate ``PhenotypicFeature`` entry with its own +``onset`` value. -2. **Merging Configurations** - - .. code-block:: python - - merged_config = PhenopacketMappingFactory.merge_configurations( - base_config=base_mappings, - override_config=custom_overrides - ) +For example, a recurrent respiratory infection may have been recorded on +three separate dates. Without multi-onset, only the first date would be +captured. With multi-onset, three separate ``PhenotypicFeature`` messages +are produced — one per date — all sharing the same type, severity, and +modifiers. + +**Configuration** + +Enable multi-onset in the mapping block for the relevant phenotypicFeatures +instrument:: + + FEATURES_BLOCK = { + "redcap_repeat_instrument": "your_instrument_name", + "type_field": "your_type_field", + "severity_field": "your_severity_field", + + # Enable multi-onset + "multi_onset": True, + "onset_date_fields": [ + "onset_date_1", + "onset_date_2", + "onset_date_3", + # ... add as many as your instrument supports + ], + } + +**What it does** + +For each non-empty date in ``onset_date_fields``: + +1. Creates the base ``PhenotypicFeature`` using the normal mapping function. +2. Deep-copies it. +3. Replaces the ``onset`` field with an ``Age`` element computed from that + date and the subject's date of birth. +4. Appends the copy to the result list. + +If no valid dates are found, the base feature (with whatever onset was +already set) is returned as a single-element list. + +**Data shape expected** + +The adapter reads onset dates directly from the inner instrument dict of +each repeated element:: + + { + "redcap_repeat_instrument": "your_instrument_name", + "redcap_repeat_instance": 1, + "your_instrument_name": { + "your_type_field": "HP:0004469", + "severity_field": "HP:0012826", + "onset_date_1": "2022-02-01", + "onset_date_2": "2023-03-01", + "onset_date_3": "2023-12-01" + } + } + +Produces three separate ``PhenotypicFeature`` messages, each with +``onset.age.iso8601duration`` computed from the respective date and DOB. + +**Notes** + +- All copies share the same type, severity, and modifiers. +- If DOB is unavailable, no age calculation is possible and the base + feature is returned unchanged. +- Combine with :ref:`ontology-routing-adapter` when the same instrument + also mixes HP and MONDO codes. + +----- + +.. _ontology-routing-adapter: + +Ontology Routing Adapter +------------------------- + +**Module:** ``rarelink.phenopackets.adapter.ontology_routing_adapter`` + +**When to use it** + +Use the ontology routing adapter when a **single data element** (or a set +of mutually exclusive sub-fields within one instrument) may be populated +with codes from **different ontologies** depending on the record — and +each ontology should map to a different Phenopacket block. + +The canonical example is a "specific finding" field that stores either an +HPO term (a phenotypic abnormality) or a MONDO term (a disease entity). +Both are clinically valid answers to the same question, but they belong in +different Phenopacket blocks: +.. list-table:: + :widths: 20 20 60 + :header-rows: 1 + + * - Ontology prefix + - Phenopacket block + - Semantic basis + * - ``HP:`` + - ``phenotypicFeatures`` + - HPO terms describe phenotypic abnormalities + (``PhenotypicFeature.type``) + * - ``MONDO:`` + - ``diseases`` + - MONDO terms describe disease entities (``Disease.term``) + * - ``OMIM:`` + - ``diseases`` + - OMIM identifiers describe disease entities + * - ``ORDO:`` + - ``diseases`` + - Orphanet identifiers describe rare disease entities + +These defaults are grounded in the GA4GH Phenopacket v2 schema and are +always active. They can be extended or overridden via the ``rules`` key +(see below). + +**Configuration** + +Add an ``ontology_routing`` key to your ``mapping_configs``:: + + mapping_configs = { + ... + "ontology_routing": { + "enabled": True, + + # Instruments whose repeated elements should be inspected + "instruments": [ + "your_mixed_instrument", + "another_mixed_instrument", + ], + + # Per-instrument: which fields to scan for routable codes. + # These are the type_field_1…N names from your mapping block. + # If omitted, all string fields are scanned (slower, zero-config). + "scan_fields": { + "your_mixed_instrument": [ + "field_holding_hp_or_mondo_1", + "field_holding_hp_or_mondo_2", + ], + "another_mixed_instrument": [ + "field_a", + "field_b", + ], + }, + + # Per-instrument: onset date fields. + # The first non-empty value is used as Disease.onset when a + # MONDO-coded element is routed to the diseases block. + "onset_fields": { + "your_mixed_instrument": [ + "onset_date_1", + "onset_date_2", + ], + "another_mixed_instrument": [ + "condition_onset_date", + ], + }, + + # Optional: override or extend the built-in prefix→block rules. + # Useful if your data model uses a non-standard ontology. + # "rules": { + # "HP": "phenotypicFeatures", + # "MONDO": "diseases", + # "MYCUSTOM": "diseases", + # } + }, + ... + } + +When ``ontology_routing`` is absent from ``mapping_configs``, the adapter +is never called and pipeline behaviour is identical to before. + +**What it does** + +Before any mapper runs, the adapter: + +1. Iterates over ``data["repeated_elements"]``. +2. For each element from a configured instrument, scans the configured + (or all) string fields for a value whose ontology prefix is in the + routing rules. +3. Routes the element: + + - **HP-coded** → placed in ``data["__routed__phenotypicFeatures"]`` + *unchanged*. The ``PhenotypicFeatureMapper`` processes it normally, + including multi-onset if configured. + - **MONDO/OMIM/ORDO-coded** → *normalized* to the ``term_field_1`` / + ``onset_date_field`` convention and placed in + ``data["__routed__diseases"]``. The ``DiseaseMapper`` consumes it + via the same path it uses for all other disease data. + +4. Elements with no routable code (e.g. a SNOMED sub-field that holds only + SNOMED values) are left in the original ``repeated_elements`` stream and + processed normally by whatever mapper is configured for that instrument. + +The original ``repeated_elements`` list is **never mutated**. + +**Data shape expected** + +A typical mixed element:: + + { + "redcap_repeat_instrument": "infections_initial_form", + "redcap_repeat_instance": 2, + "infections_initial_form": { + "type_of_infection": "snomedct_127856007", ← SNOMED category, ignored + "snomedct_127856007": "mondo_0043653", ← MONDO → diseases + "infection_severity": "hp_0012826", + "infection_date": "2023-02-01" + } + } + +The adapter detects ``mondo_0043653`` in ``snomedct_127856007``, routes +the element to ``diseases``, and normalizes it to:: + + { + "term_field_1": "mondo_0043653", + "onset_date_field": "2023-02-01", + "onset_category_field": None, + "excluded_field": None, + "primary_site_field": None, + "__source_instrument": "infections_initial_form", + "__source_instance": 2 + } + +**Validation integration** + +The adapter also provides :func:`check_prefix_placement`, which is called +automatically by ``validate_phenopackets`` after each file is written. It +inspects the serialized Phenopacket JSON and emits **non-fatal soft warnings** +when: + +- An HP: term appears in ``diseases`` (likely a routing error) +- A non-HP: term appears in ``phenotypicFeatures`` (MONDO/OMIM in a + features block) + +These warnings appear in the validation output but do not cause the +phenopacket to fail validation. They are intended to help data curators +catch misconfigured routing rules. + +**Adding the adapter to your mapping config** + +If you are building a custom data model and have instruments with mixed +ontology codes, add the ``ontology_routing`` block alongside your existing +``phenotypicFeatures`` and ``diseases`` configurations. No changes to +the mapper classes or the rest of the pipeline are needed. + +.. tip:: + Combine this adapter with the :ref:`multi-onset-adapter` when MONDO-routed + elements are diseases observed multiple times. Configure ``multi_onset`` + in the base ``phenotypicFeatures`` block; the routing adapter handles the + HP/MONDO split first, and the multi-onset adapter then expands the HP + elements per date. + _____________________________________________________________________________________ .. _troubleshooting: diff --git a/docs/4_user_guide/4_5_develop_redcap_instruments.rst b/docs/4_user_guide/4_5_develop_redcap_instruments.rst index 85c160c66..013093860 100755 --- a/docs/4_user_guide/4_5_develop_redcap_instruments.rst +++ b/docs/4_user_guide/4_5_develop_redcap_instruments.rst @@ -106,7 +106,7 @@ but we recommend keeping these in there so that they are part your project's - HP:0012829 | Profound - HP:0012828 | Severe Version(s): - - HPO Version 2025-05-06 + - HPO Version 2026-02-16 Mapping: - HL7 FHIR Expression v4.0.1: Observation.interpretation - GA4GH Phenopacket Schema v2.0 Element: PhenotypicFeature.severity diff --git a/docs/6_changelog.rst b/docs/6_changelog.rst index 794ea4f77..d2d9f151b 100755 --- a/docs/6_changelog.rst +++ b/docs/6_changelog.rst @@ -3,7 +3,75 @@ Changelog =========== -v2.0.5 (2025-11-19) +v2.0.6 +--------------------- + +Phenopacket Export Pipeline: +""""""""""""""""""""""""""""" +- Fixed camelCase serialization in Phenopacket JSON output (``preserving_proto_field_name=False``), + producing valid GA4GH Phenopacket v2 JSON (`#205 `_) +- Integrated ``validate.py`` into the export pipeline with a two-tier approach: + Python-native structural checks (required fields, CURIE format, schema version) + plus optional ``phenopacket-tools`` CLI fallback when available on PATH +- Added ontology-prefix placement checks as soft validation warnings + (e.g. HP: terms outside ``phenotypicFeatures``, MONDO: terms outside ``diseases``) +- Improved CLI export with Rich progress bars: sequential two-phase display + (creation bar completes before validation bar appears) +- Added structured warnings system: mapper warnings (e.g. "No diagnosis ID found") + now use ``warnings.warn()`` consistently, are captured per-record with record ID + context, and displayed in the export summary +- Added ``warnings.json`` output file alongside ``failures.json`` for detailed + warning inspection +- Export summary now includes a Warnings column for both creation and validation + stages with deduplicated counts + +RD-CDM v2.0.3 Integration: +"""""""""""""""""""""""""""" +- Updated to `rd-cdm v2.0.3 `_ + with eleven updated ontology/code system versions and the replacement of the + deprecated SNOMED CT Sex at Birth concept with LOINC ``76689-9`` +- Updated the RareLink-CDM REDCap data dictionary and instruments to reflect + rd-cdm v2.0.3 changes +- Updated Phenopacket and FHIR export pipelines to use the revised code systems + and data element definitions + +Ontology Routing Adapter: +"""""""""""""""""""""""""" +- Designed ``ontology_routing_adapter.py`` for use cases where single repeated + elements contain mixed ontology prefixes (e.g. HP and MONDO codes in the same + instrument), enabling automatic routing of HP codes to ``phenotypicFeatures`` + and MONDO/OMIM/ORDO codes to ``diseases`` + (`#207 `_) + +Package & Dependencies: +"""""""""""""""""""""""" +- Reduced core dependencies from ~20 to ~10 by moving test, docs, Docker, numpy, + and pandas to optional extras (``[test]``, ``[docs]``, ``[fhir]``, ``[data]``, + ``[dev]``) +- Removed unused dependencies: ``node``, ``npm``, ``config``, ``schema-automator``, + ``tqdm`` +- Removed the ``rarelink_cdm/rd_cdm/`` codegen subfolder (unused at runtime); + future versions will import directly from the ``rd-cdm`` PyPI package + +Infrastructure: +"""""""""""""""" +- Fixed toFHIR Docker Compose: updated Kafka image references from deprecated + ``bitnami/kafka:latest`` to ``bitnamilegacy/kafka:3.7.0`` following Bitnami's + August 2025 image deprecation + (`#210 `_) +- Added ``.DS_Store`` to ``.gitignore`` + (`#202 `_) + +Documentation & Tests: +""""""""""""""""""""""" +- Updated installation docs to document optional extras + (``pip install rarelink[fhir]``, ``pip install rarelink[data]``, etc.) +- Updated tests to work with ``PipelineResult`` dataclass return type and + corrected mock patch paths +- Removed unused numpy/pandas references from Sphinx ``conf.py`` + + +v2.0.5 (2025-12-12) --------------------- - Fixed a few of bugs - enhanced versioning and refactored repo folders diff --git a/docs/_build/html/2_rarelink_framework/2_2_rarelink_cdm.html b/docs/_build/html/2_rarelink_framework/2_2_rarelink_cdm.html index c6a011ac0..5ba77d0b8 100644 --- a/docs/_build/html/2_rarelink_framework/2_2_rarelink_cdm.html +++ b/docs/_build/html/2_rarelink_framework/2_2_rarelink_cdm.html @@ -268,7 +268,7 @@

Contents

REDCap Data Dictionary

The RareLink CDM Data Dictionary is available for download as a CSV file:

-

Download: RareLink-CDM Data Dictionary (v2.0.5)

+

Download: RareLink-CDM Data Dictionary (v2.0.5)

… Next steps