Commit 0b2f5cb
committed
fix(cli/backup): stream pg_dump to disk via Node pipeline, drop gzip subprocess
Backup creation OOM-killed Node a few seconds into pg_dump on the
production-sized openmapx database:
FATAL ERROR: Ineffective mark-compacts near heap limit
Allocation failed - JavaScript heap out of memory
Cause: pgDumpToFile spawned `docker compose exec … pg_dump` via execa
and piped it into a sibling `gzip -c` execa with `input: sub.stdout`.
execa v9 buffers each child's stdout into the resolved result object
by default — so even though gzip was consuming the bytes via the
input stream, execa was simultaneously collecting the entire dump
into memory for the result.
Switch to a real Node stream pipeline:
pg_dump (execa, buffer.stdout=false)
→ createGzip()
→ createWriteStream(outFile)
via stream/promises pipeline().
Bonus: drops the separate gzip subprocess (zlib.createGzip handles it
in-process). Existing 117 CLI tests still pass.1 parent 57e1893 commit 0b2f5cb
1 file changed
Lines changed: 23 additions & 14 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
| 2 | + | |
2 | 3 | | |
3 | 4 | | |
4 | 5 | | |
| |||
8 | 9 | | |
9 | 10 | | |
10 | 11 | | |
| 12 | + | |
| 13 | + | |
11 | 14 | | |
12 | 15 | | |
13 | 16 | | |
| |||
476 | 479 | | |
477 | 480 | | |
478 | 481 | | |
479 | | - | |
480 | | - | |
481 | | - | |
| 482 | + | |
| 483 | + | |
| 484 | + | |
| 485 | + | |
| 486 | + | |
| 487 | + | |
| 488 | + | |
| 489 | + | |
| 490 | + | |
482 | 491 | | |
483 | 492 | | |
484 | 493 | | |
| |||
495 | 504 | | |
496 | 505 | | |
497 | 506 | | |
498 | | - | |
| 507 | + | |
499 | 508 | | |
500 | 509 | | |
501 | | - | |
502 | | - | |
503 | | - | |
504 | | - | |
505 | | - | |
506 | | - | |
| 510 | + | |
| 511 | + | |
| 512 | + | |
| 513 | + | |
| 514 | + | |
| 515 | + | |
507 | 516 | | |
508 | | - | |
| 517 | + | |
| 518 | + | |
| 519 | + | |
| 520 | + | |
509 | 521 | | |
510 | 522 | | |
511 | 523 | | |
512 | | - | |
513 | | - | |
514 | | - | |
515 | 524 | | |
516 | 525 | | |
517 | 526 | | |
| |||
0 commit comments