Problem
The package contains two separate implementations of the same data-quality checks, with diverging logic and incompatible output:
- Orphaned / exported:
run_dqa_checks() + R/dqa_checks.R — documented and exported, but never called by the pipeline. Effectively dead code.
- Active / internal: the
dqa_* family in R/final_format_utils.R, invoked by run_final_formatting() (the path the state pipelines actually run).
Every check is duplicated, often with different thresholds and return structures:
| Check |
R/dqa_checks.R |
R/final_format_utils.R |
| Duplicates |
:28 |
:443 |
| Negative values |
:97 |
:506 |
| Exceeding enrollment |
:146 |
:537 |
| Coverage outliers |
:203 |
:569 |
| Enrollment deviation |
:279 |
:638 |
| Current/vacc deviation |
:346 |
:732 |
| Extreme outliers |
:405 |
:825 |
Additional issues:
- Incompatible report structures —
generate_dqa_summary() only understands the run_dqa_checks() output, not the dqa_report list that run_final_formatting() actually builds (final_format_utils.R:~1198).
- Duplicated thresholds with drift — e.g. coverage
1.05, deviation 0.5/0.75/1.0, extreme-outlier multiplier 10 appear in both systems, sometimes with different values at the call site.
- Filtering run twice — zero-enrollment / zero-current removal exists in both
clean_state_data() and final_format_utils.R (dqa_remove_zero_enrollment / dqa_remove_zero_current).
Suggested direction
- Pick one implementation as canonical (likely the wired-in
dqa_* family).
- Delete or formally deprecate the other (
run_dqa_checks() + R/dqa_checks.R).
- Unify on a single DQA report structure so
generate_dqa_summary() works against the real pipeline output.
- Centralize thresholds as documented parameters rather than scattered literals.
Scope
Investigation + design only for now — do not implement until the direction is confirmed.
Problem
The package contains two separate implementations of the same data-quality checks, with diverging logic and incompatible output:
run_dqa_checks()+R/dqa_checks.R— documented and exported, but never called by the pipeline. Effectively dead code.dqa_*family inR/final_format_utils.R, invoked byrun_final_formatting()(the path the state pipelines actually run).Every check is duplicated, often with different thresholds and return structures:
R/dqa_checks.RR/final_format_utils.R:28:443:97:506:146:537:203:569:279:638:346:732:405:825Additional issues:
generate_dqa_summary()only understands therun_dqa_checks()output, not thedqa_reportlist thatrun_final_formatting()actually builds (final_format_utils.R:~1198).1.05, deviation0.5/0.75/1.0, extreme-outlier multiplier10appear in both systems, sometimes with different values at the call site.clean_state_data()andfinal_format_utils.R(dqa_remove_zero_enrollment/dqa_remove_zero_current).Suggested direction
dqa_*family).run_dqa_checks()+R/dqa_checks.R).generate_dqa_summary()works against the real pipeline output.Scope
Investigation + design only for now — do not implement until the direction is confirmed.