Fix ZIP download 404 by building export archives deterministically - #59
Merged
Conversation
Both "Download Complete Package (ZIP)" and the SLZ package button served archives that were rebuilt on every Streamlit rerun with different bytes: zipfile.writestr stamps each member with the current local time, and the policy README embedded pd.Timestamp.now(). Streamlit serves download payloads from its MediaFileManager, keying each file by sha224(content + mimetype + filename) and deleting whatever file was previously registered at the same widget coordinates. New bytes every rerun therefore evicted the archive the browser was still pointing at, so the download 404'd: MediaFileHandler: Missing file f0338eff....zip GET /media/f0338eff....zip 404 Add utils.packaging.build_deterministic_zip, which pins member timestamps and writes members in sorted order, so identical inputs produce identical bytes and the media ID stays stable across reruns. The archive now changes only when the underlying data changes, matching how the other download buttons already behave. The README's "Generated on" now reports when the policy was generated rather than when the page last reran; policy_generated_at is persisted and restored with the rest of the workflow state. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Symptom
Downloading the initiative package failed. Frontend logs:
Intermittent 404s, which is why it looked flaky rather than broken.
Root cause
Streamlit serves
st.download_buttonpayloads from its in-memoryMediaFileManager:memory_media_file_storage._calculate_file_id=sha224(content + mimetype + filename)MediaFileManagertracks files per(session, widget coordinates)and, per its own docstring, marks a file for deletion "if it gets replaced by another file at the same coordinates"Both ZIP buttons rebuilt their archive on every rerun with different bytes:
zipfile.writestr(name, content)stamps each member with the current local timepd.Timestamp.now()So every rerun produced a new file ID at the same coordinates, evicting the archive the browser was still pointing at → 404.
The other download buttons (JSON, Bicep, PowerShell, CSV) pass stable strings, which is why only the ZIPs broke.
Fix
Add
utils/packaging.py::build_deterministic_zip, which pins member timestamps to a fixed value and writes members in sorted order. Identical inputs now produce byte-identical archives, so the media ID is stable and the file is reused across reruns.Applied to both affected buttons:
The archive still changes when the underlying data changes, so downloads never go stale.
The README's
Generated onnow reports when the policy was generated rather than when the page last reran.policy_generated_atis persisted and restored with the rest of the workflow state, guarded byif key in savedso existing saved sessions are unaffected.Tests
New
app/tests/test_export_package_zip.py(5 tests), including determinism across a clock tick, pinned member timestamps, order-independence, and a guard that changed content still yields new bytes.Proven to catch the bug: reverting the helper to the old
writestrbehaviour fails 3 of the 5, including the core determinism test.Full suite: 859 passed / 12 failed. The 12 are the documented pre-existing failures (
test_state_management.py×11,test_frontend_responsive_content.py::test_export_uses_south_africa_slz_defaults_and_token_guidance×1) — confirmed identical on a stashed clean tree. Zero regressions.Noted, not fixed
app/backend/requirements.txtpinsnumpy>=2.0.0whileapp/frontend/requirements.txtpinsnumpy==1.26.3— they cannot be co-installed. Harmless in production (separate images) but it breaks a single combined dev venv.