Context
The TimedBelief table (see e62ac5f519d7_create_table_for_timed_beliefs.py) defines both cumulative_probability and event_value as sa.Float(), which SQLAlchemy/Postgres maps to float8 (double precision, 8 bytes per value).
This is our largest and fastest-growing table, so column width has a direct effect on disk/storage costs and I/O.
Proposal
Change cumulative_probability and event_value from float8 (double precision) to float4 (single precision), via a new Alembic migration (ALTER COLUMN ... TYPE real).
Rough estimate: this could reduce the size of the timed_beliefs table by roughly 15-20%, since these are two of the core per-row numeric columns.
Things to check before implementing
- Precision loss:
float4 has ~7 significant decimal digits vs ~15-17 for float8. Need to confirm this is acceptable for:
event_value (sensor readings/schedules/forecasts - magnitudes and required precision vary by sensor/unit)
cumulative_probability (bounded in [0, 1], where lower precision may be more acceptable)
- Migration cost/downtime:
ALTER COLUMN TYPE on a large table rewrites it; needs a migration strategy (e.g. batched backfill, or accept the lock/downtime) given this is likely the largest table in most deployments.
- Any downstream code that assumes float64 precision (e.g. numpy/pandas dtype expectations when reading beliefs into a
BeliefsDataFrame, or serialization/rounding logic in the API).
- Whether this should be an opt-in migration or bundled with other maintenance (partitioning, etc.).
Context
The
TimedBelieftable (seee62ac5f519d7_create_table_for_timed_beliefs.py) defines bothcumulative_probabilityandevent_valueassa.Float(), which SQLAlchemy/Postgres maps tofloat8(double precision, 8 bytes per value).This is our largest and fastest-growing table, so column width has a direct effect on disk/storage costs and I/O.
Proposal
Change
cumulative_probabilityandevent_valuefromfloat8(double precision) tofloat4(single precision), via a new Alembic migration (ALTER COLUMN ... TYPE real).Rough estimate: this could reduce the size of the
timed_beliefstable by roughly 15-20%, since these are two of the core per-row numeric columns.Things to check before implementing
float4has ~7 significant decimal digits vs ~15-17 forfloat8. Need to confirm this is acceptable for:event_value(sensor readings/schedules/forecasts - magnitudes and required precision vary by sensor/unit)cumulative_probability(bounded in [0, 1], where lower precision may be more acceptable)ALTER COLUMN TYPEon a large table rewrites it; needs a migration strategy (e.g. batched backfill, or accept the lock/downtime) given this is likely the largest table in most deployments.BeliefsDataFrame, or serialization/rounding logic in the API).