Skip to content

Widen File.file_size to 64-bit (bigint) via zero-downtime expand/contract #5974

@rtibbles

Description

@rtibbles

Overview

File.file_size is a 32-bit IntegerField (PostgreSQL integer, max ~2.1 GB); a larger file overflows it → "integer out of range" → upload_url 500s before any bytes transfer. Widen it to 64-bit bigint, zero-downtime on the ~100M-row File table, using the migration capability from studio#5973.

Complexity: Medium
Target branch: hotfixes

Context

upload_url constructs File(file_size=size, …); a multi-GB size overflows the column and Postgres raises "integer out of range". file_size is also read/aggregated for storage accounting, check_space, and resource-size queries. A naive AlterField to bigint would rewrite the table under an ACCESS EXCLUSIVE lock — hence expand/contract.

The Change

Widen File.file_size to bigint via expand/contract on studio#5973:

  • Add a nullable bigint shadow column with a declarative dual-write trigger.
  • Backfill existing rows online in batches; verify completion.
  • Cut reads/aggregations to the new column behind a flag.
  • Drop the old column and rename the shadow into place.

No full-table rewrite or ACCESS EXCLUSIVE lock at any step.

Out of Scope

  • The reusable migration capability itself (studio#5973).
  • Resumable uploads.
  • Other byte-size columns (audited and tracked separately if found).

Acceptance Criteria

  • File.file_size is a 64-bit bigint column.
  • A File larger than 2.1 GB persists without "integer out of range", and upload_url succeeds for it.
  • The change is applied with no full-table rewrite and no ACCESS EXCLUSIVE lock.
  • All file_size reads/aggregations (storage accounting, check_space, resource-size queries) use the widened column.
  • Backfilled values match the originals (no storage-accounting regression).

AI usage

I used Claude (Opus 4.8, via le-skills:writing-github-issues) to root-cause the overflow and draft this issue. The expand/contract approach was my decision; I edited for scope.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No fields configured for Task.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions