Problem
Manual, per-school data corrections are currently hardcoded directly into cleaning code rather than supplied as data. This makes the cleaning logic state-specific, hard to review, and forces correction data into version control (a problem since these repos are intended to go public).
Locations:
inst/templates/03_state_cleaning.R:240-267 — hardcode_school_addresses() with one fix(...) call per school (address, city, zip, lat/lon, business status).- The same anti-pattern leaks into the package itself: state-specific corrections embedded in
R/preprocessing_utils.R — e.g. the "oosle" MD nonpublic-school pattern (:334) and the redaction strings (:244, :252).
Suggested direction
Two clean inputs -> one generic applier:
- Raw data file (already handled).
- Patch file — a state-specific table of corrections (e.g.
match_key, field, new_value, or a wide form mirroring the current fix() signature). Kept private / gitignored, the same way raw data is.
- A package function
apply_patches(data, patch_file) that applies the corrections programmatically. No school names or manual fixes in .R files.
Open design questions:
- Match key: the current approach matches on
school_name_orig, which is brittle and order-dependent. Decide on a stable key (school id?) before formalizing.
- Decide patch file format (CSV vs YAML) and where the template references it.
Context
Raised by Carl: be careful about hardcoding patch entries in cleaning; keep the patch file private; don't commit patches or raw data (repos will be public in future).
Scope
Design + abstraction only — do not implement until confirmed.
Problem
Manual, per-school data corrections are currently hardcoded directly into cleaning code rather than supplied as data. This makes the cleaning logic state-specific, hard to review, and forces correction data into version control (a problem since these repos are intended to go public).
Locations:
inst/templates/03_state_cleaning.R:240-267—hardcode_school_addresses()with onefix(...)call per school (address, city, zip, lat/lon, business status).R/preprocessing_utils.R— e.g. the"oosle"MD nonpublic-school pattern (:334) and the redaction strings (:244,:252).Suggested direction
Two clean inputs -> one generic applier:
match_key, field, new_value, or a wide form mirroring the currentfix()signature). Kept private / gitignored, the same way raw data is.apply_patches(data, patch_file)that applies the corrections programmatically. No school names or manual fixes in.Rfiles.Open design questions:
school_name_orig, which is brittle and order-dependent. Decide on a stable key (school id?) before formalizing.Context
Raised by Carl: be careful about hardcoding patch entries in cleaning; keep the patch file private; don't commit patches or raw data (repos will be public in future).
Scope
Design + abstraction only — do not implement until confirmed.