fix stg_vocabulary__source_to_concept_map when seed_source is false#151
Open
odikia wants to merge 4 commits into
Open
fix stg_vocabulary__source_to_concept_map when seed_source is false#151odikia wants to merge 4 commits into
odikia wants to merge 4 commits into
Conversation
source_to_concept_map_seed is not part of the Athena vocabulary download and only exists in the seeds schema. When seed_source is false, the source resolves to the vocab schema where the table does not exist, causing a parser error. Add get_source wrapper macro that routes this table to ref() when seed_source is false. Closes OHDSI#95
Author
|
@katy-sadowski , hey there! It's Dan from Kheiron BTW, just in case you don't recognize my github handle ;) |
Collaborator
|
Hi Dan! Good catch! We actually did fix the issue in #95 but then I reverted that change later on and introduced a new bug. I think there might be a simpler solution than what's proposed here. I believe you can override the https://github.com/OHDSI/dbt-synthea/blob/main/models/staging/vocabulary/_vocabulary__sources.yml |
Add generate_schema_name macro and configure dbt_project.yml so that vocabulary models materialize into dbt_synthea_dev_vocab and clinical models into dbt_synthea_dev_cdm. Staging/intermediate remain in dbt_synthea_dev. The file restructure (moving models into vocabulary/ and clinical/ subdirectories) is a prerequisite for this to take effect.
Restructure models/omop/ so dbt applies the +schema config from dbt_project.yml. Vocabulary tables (concept, concept_ancestor, etc.) go to models/omop/vocabulary/ -> dbt_synthea_dev_vocab schema. Clinical tables (person, condition_occurrence, etc.) go to models/omop/clinical/ -> dbt_synthea_dev_cdm schema. No SQL or YML content changed — file moves only.
load_data_duckdb macro now skips strptime for DATE columns that parquet files already provide as native dates. Added pyproject.toml with dbt-core, dbt-duckdb, and duckdb dependencies for uv-based execution.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
source_to_concept_map_seedis not part of the Athena vocabulary download and only exists in the seeds schemaseed_source: false, the vocabulary source schema switches to_vocabwhere this table does not existget_filtered_columns_in_relation()returns an empty list, causingParser Error: SELECT clause without selection listget_sourcewrapper macro that routessource_to_concept_map_seedtoref()(the seed) whenseed_source: false, and passes all other tables through tosource()as normaldo return()to preserve the Relation object for introspection macros likeget_filtered_columns_in_relation()Verified on forked repo with full Athena vocabulary loaded and
seed_source: false.Note: #95 was closed as completed but no PR was merged — the bug remains on
main.Closes #95