update_redcap_diffs currently compares two CSV files and then imports the changeset into REDCap. This is kind of bad because:
- It means the script is more complex than it needs to be
- There's no CLI to import arbitrary data even though that would be useful
- Stop and look at the JSON before importing things isn't a well-supported part of the import (yes you can --dry-run but that is not amazing)
The last point is a real issue; making and inspecting the changeset is critical to make sure your pipelines are working correctly.
This would split this CLI into two utilities -- one to write a JSON changeset (CSV changesets are not a good idea; we're updating different fieldsets per record), and a general "import records" utility. The import records one would take at least JSON and CSV; perhaps XML as well but that might not be worth the complexity
update_redcap_diffscurrently compares two CSV files and then imports the changeset into REDCap. This is kind of bad because:The last point is a real issue; making and inspecting the changeset is critical to make sure your pipelines are working correctly.
This would split this CLI into two utilities -- one to write a JSON changeset (CSV changesets are not a good idea; we're updating different fieldsets per record), and a general "import records" utility. The import records one would take at least JSON and CSV; perhaps XML as well but that might not be worth the complexity