Skip to content

Fix buffer overflow in parseTimeLineHistory when timeline > 1024 - #1139

Merged
dimitri merged 2 commits into
mainfrom
fix/timeline-history-overflow
Jul 10, 2026
Merged

Fix buffer overflow in parseTimeLineHistory when timeline > 1024#1139
dimitri merged 2 commits into
mainfrom
fix/timeline-history-overflow

Conversation

@dimitri

@dimitri dimitri commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Problem

When a pg_auto_failover cluster has undergone more than ~1024 failovers, its current PostgreSQL timeline number exceeds 1024. At that point the TIMELINE_HISTORY replication command returns more than 1024 lines, overflowing the fixed-size history[PG_AUTOCTL_MAX_TIMELINES] array inside TimeLineHistory.

parseTimeLineHistory writes a "tip" entry unconditionally after the loop at history[count], where count has just reached 1024 — one slot past the end of the array. Each additional timeline pushes that write further into memory. The array is the last field of IdentifySystem, which is the last field of ReplicationSource, which is embedded in LocalPostgresServer. The overflow therefore corrupts the fields that immediately follow in that struct: pgIsRunning, pgsrSyncState, currentLSN, pgFirstStartFailureTs, pgStartRetries, and pgKind.

The keeper then logs:

nodeKindToString: unknown node kind XX

and behaves incorrectly because pgKind now holds garbage bytes from the overwritten LSN data.

This bug is in the keeper (client code, src/bin/common/pgsql.c). The monitor is not involved.

Fix

Replace the fixed-size array in TimeLineHistory with a heap-allocated pointer:

typedef struct TimeLineHistory
{
    int count;
    int capacity;
    TimeLineHistoryEntry *history;   /* NULL until first parseTimeLineHistory() call */
} TimeLineHistory;

parseTimeLineHistory() allocates the buffer on first use and grows it with realloc() (doubling strategy) when needed. Because ReplicationSource is embedded in the long-lived Keeper struct, the buffer is reused across repeated pgctl_identify_system() calls and freed by the OS on process exit — no explicit cleanup path is needed.

All consumer sites (cli_do_misc.c, pgsql.c:3041) already use history[index] pointer-subscript syntax, which is syntactically identical whether history is a fixed array or a heap pointer — no changes to those callers are required.

Closes: #1107

TimeLineHistory.history was a fixed-size array of 1024 entries. When a
cluster's current timeline exceeded 1024, the TIMELINE_HISTORY replication
command returns more lines than the array can hold. parseTimeLineHistory
would then write a "tip" entry at history[1024] — one slot past the end —
corrupting fields that follow ReplicationSource in LocalPostgresServer,
including pgIsRunning, currentLSN, and pgKind.  The keeper then logged
"nodeKindToString: unknown node kind" and behaved incorrectly.

Replace the fixed array with a dynamically-allocated pointer. The buffer is
allocated on first use inside parseTimeLineHistory() and grown with realloc()
when needed (doubling strategy), so it handles any number of timelines without
a hard cap. Because ReplicationSource is embedded in the long-lived Keeper
struct, the buffer is reused across calls and freed by the OS on exit — no
explicit cleanup path is required.

Consumer code (cli_do_misc.c, pgsql.c) uses history[index] pointer subscript
syntax throughout, which is unchanged whether history is an array or a heap
pointer — no consumer edits needed.
@dimitri dimitri self-assigned this Jul 10, 2026
@dimitri dimitri added the bug Something isn't working label Jul 10, 2026
Timeline history cannot shrink (each new TLI's history file is the parent's
verbatim plus one appended line, and timelines are strictly increasing), so
stale entries above count are never reached by existing consumers.

Defensively zero the newly allocated region after realloc so that a future
consumer reading past count by mistake gets zeroes rather than garbage.
@dimitri
dimitri merged commit 1dd4c95 into main Jul 10, 2026
54 checks passed
@dimitri
dimitri deleted the fix/timeline-history-overflow branch July 10, 2026 18:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Frequent failover errors: nodeKindToString: unknown node kind xx

1 participant