Skip to content

Fix ZIP download 404 by building export archives deterministically - #59

Merged
warrendt merged 1 commit into
mainfrom
fix-export-zip-download
Aug 14, 2026
Merged

Fix ZIP download 404 by building export archives deterministically#59
warrendt merged 1 commit into
mainfrom
fix-export-zip-download

Conversation

@warrendt

Copy link
Copy Markdown
Owner

Symptom

Downloading the initiative package failed. Frontend logs:

MediaFileHandler: Missing file f0338effeadac30f9ee70ea85f776da18c156e0ff82bb583e4100b87.zip
GET /media/f0338eff…zip HTTP/1.1" 404 69
GET /media/6d37eea2…zip HTTP/1.1" 200 163231   ← a later attempt worked

Intermittent 404s, which is why it looked flaky rather than broken.

Root cause

Streamlit serves st.download_button payloads from its in-memory MediaFileManager:

  • memory_media_file_storage._calculate_file_id = sha224(content + mimetype + filename)
  • MediaFileManager tracks files per (session, widget coordinates) and, per its own docstring, marks a file for deletion "if it gets replaced by another file at the same coordinates"

Both ZIP buttons rebuilt their archive on every rerun with different bytes:

  1. zipfile.writestr(name, content) stamps each member with the current local time
  2. the policy README additionally embedded pd.Timestamp.now()

So every rerun produced a new file ID at the same coordinates, evicting the archive the browser was still pointing at → 404.

The other download buttons (JSON, Bicep, PowerShell, CSV) pass stable strings, which is why only the ZIPs broke.

Fix

Add utils/packaging.py::build_deterministic_zip, which pins member timestamps to a fixed value and writes members in sorted order. Identical inputs now produce byte-identical archives, so the media ID is stable and the file is reused across reruns.

Applied to both affected buttons:

  • Download Complete Package (ZIP)
  • Download Complete SLZ Package (ZIP) — same defect via the mtime path

The archive still changes when the underlying data changes, so downloads never go stale.

The README's Generated on now reports when the policy was generated rather than when the page last reran. policy_generated_at is persisted and restored with the rest of the workflow state, guarded by if key in saved so existing saved sessions are unaffected.

Tests

New app/tests/test_export_package_zip.py (5 tests), including determinism across a clock tick, pinned member timestamps, order-independence, and a guard that changed content still yields new bytes.

Proven to catch the bug: reverting the helper to the old writestr behaviour fails 3 of the 5, including the core determinism test.

Full suite: 859 passed / 12 failed. The 12 are the documented pre-existing failures (test_state_management.py ×11, test_frontend_responsive_content.py::test_export_uses_south_africa_slz_defaults_and_token_guidance ×1) — confirmed identical on a stashed clean tree. Zero regressions.

Noted, not fixed

app/backend/requirements.txt pins numpy>=2.0.0 while app/frontend/requirements.txt pins numpy==1.26.3 — they cannot be co-installed. Harmless in production (separate images) but it breaks a single combined dev venv.

Both "Download Complete Package (ZIP)" and the SLZ package button served
archives that were rebuilt on every Streamlit rerun with different bytes:
zipfile.writestr stamps each member with the current local time, and the
policy README embedded pd.Timestamp.now().

Streamlit serves download payloads from its MediaFileManager, keying each
file by sha224(content + mimetype + filename) and deleting whatever file
was previously registered at the same widget coordinates. New bytes every
rerun therefore evicted the archive the browser was still pointing at, so
the download 404'd:

  MediaFileHandler: Missing file f0338eff....zip
  GET /media/f0338eff....zip 404

Add utils.packaging.build_deterministic_zip, which pins member timestamps
and writes members in sorted order, so identical inputs produce identical
bytes and the media ID stays stable across reruns. The archive now changes
only when the underlying data changes, matching how the other download
buttons already behave.

The README's "Generated on" now reports when the policy was generated
rather than when the page last reran; policy_generated_at is persisted and
restored with the rest of the workflow state.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@warrendt
warrendt merged commit d28b7a2 into main Aug 14, 2026
1 check passed
@warrendt
warrendt deleted the fix-export-zip-download branch August 14, 2026 15:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant