Decode Dialog/Talk ExcelConfigData partially via msgspec Structs

[isundaylee]

What & why

Profiling the post-msgspec 17s CHS regen showed the largest chunks of the precompute region were full decodes of huge Excel files from which only a handful of fields are ever consumed. This switches the two biggest — DialogExcelConfigData.json (93MB) and TalkExcelConfigData_*.json (141MB) — to partial typed decodes: their item types are now msgspec.Structs with only the consumed fields, decoded directly with msgspec.json.decode(..., type=...). msgspec skips unparsed fields, so:

• Dialog chain (critical path of precompute_for_workers, ~3.9s): full decode + deobfuscation rename + mapping passes drop from ~3.8s to ~0.4s (~10x).
• Talk excel (on the build_talk_tracker critical path): decode drops from ~1.0s to ~0.2s.

Full CHS regen: 17.2s → 13.1s.

Notable details

• The rotating obfuscated dialog-id key (GFLDJMJKIKE) is now a public DIALOG_EXCEL_ID_KEY constant in deobfuscation.py, used both in the field-mapping literal and as the dialog Struct's rename target — one source of truth. If the key rotates in a future build and the constant goes stale, the decode fails loudly with a missing-required-field error.
• deobfuscate_dialog_excel_config_data is deleted (the Struct rename replaces it), along with the unused talkRole field and its DialogTalkRole TypedDict (no other references).
• The talk Structs stay strict for the current build only. first_seen.scan_snapshot scans historical AGD checkouts where the id key style varies by era (id/Id/_id), so it now goes through the same load_excel_raw + _row_id path it already uses for every other excel domain; the split-vs-legacy-single-file discovery is extracted into DataRepo.talk_excel_file_names() shared by both paths.
• agd_types now imports msgspec and deobfuscation; both are cycle-free.

Testing

• Isolated benchmarks: partial decodes produce values identical to the old paths (203,908 dialog entries; 42,001 talk entries on all 4 fields).
• Full CHS regen output is byte-identical to a pre-change baseline (diff -r, excluding metadata.json commit fields).
• first_seen.scan_snapshot smoke-tested against the current build (talk domain: 42,001 ids, matching the typed decode).
• pre-commit run (black, isort, mypy) and the full pytest suite pass (199 passed, 57 skipped).

🤖 Generated with Claude Code