You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: AGENTS.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -11,7 +11,7 @@ The root directory contains `docker-compose.yml`, `.env.example`, and the long-f
11
11
## Build, Test, and Development Commands
12
12
`docker compose build postgres` rebuilds the custom image after modifying `postgres/Dockerfile` or config templates. `docker compose up -d` provisions the database stack using the active `.env`. Use `docker compose logs postgres` to inspect startup output and health checks. Run `./scripts/manage.sh psql -d app_main -U app_user` for an interactive shell, `./scripts/manage.sh backup --type=full` to seed the pgBackRest repository, `./scripts/manage.sh pgtune-config` to generate host-specific overrides in `postgresql.pgtune.conf`, `./scripts/manage.sh pgbadger-report --since "yesterday"` to produce HTML log analytics under `/backups`, and schedule `./scripts/manage.sh daily-maintenance --root ./backups/daily` nightly to capture dumps/logs and prune old runs. The CI workflow (`.github/workflows/ci.yml`) also runs `python -m pytest -k full_workflow`; mirror that smoke test before shipping meaningful changes.
13
13
14
-
After manual changes, run `./scripts/manage.sh config-check` to ensure rendered configs still match the templates. The audit helpers (`audit-roles`, `audit-security`, `audit-extensions`, `audit-autovacuum`, `audit-replication`, `audit-cron`, `audit-squeeze`) stream CSV/text reports to stdout or write directly under `/backups` with `--output` so you can diff posture over time. Capture an on-demand `pg_stat_statements` snapshot with `snapshot-pgstat --output /backups/pg_stat_manual.csv` and diff it later via `diff-pgstat --base A --compare B` when chasing query regressions, and add `--verify` to `backup` invocations to restore the latest backup into a throwaway directory for checksum validation. Use `exercise-extensions` / `pgtap-smoke` as pre-flight checks before shipping code that depends on pgvector, PostGIS, or Apache AGE features. When reclaiming space, reach for `compact --level 1/2` to review autovacuum & pg_squeeze output, `compact --level 3 --tables schema.table` for a targeted pg_repack run, and reserve `compact --level 4 --scope ... --yes` (VACUUM FULL) for scheduled downtime. Cron already handles nightly pg_squeeze refreshes, a `VACUUM (ANALYZE, SKIP_LOCKED, PARALLEL 4)` pass, and `pg_stat_statements_reset()` at 04:00 UTC. Set `DAILY_EMAIL_REPORT=true` plus `DAILY_REPORT_RECIPIENT` in `.env` if you want the HTML maintenance report mailed via `sendmail`.
14
+
After manual changes, run `./scripts/manage.sh config-check` to ensure rendered configs still match the templates. The audit helpers (`audit-roles`, `audit-security`, `audit-extensions`, `audit-autovacuum`, `audit-replication`, `audit-cron`, `audit-squeeze`, `audit-buffercache`) stream CSV/text reports to stdout or write directly under `/backups` with `--output` so you can diff posture over time. Capture an on-demand `pg_stat_statements` snapshot with `snapshot-pgstat --output /backups/pg_stat_manual.csv` (or grab the buffer cache distribution via `audit-buffercache`) and diff it later via `diff-pgstat --base A --compare B`. Add `--verify` to `backup` invocations to restore the latest backup into a throwaway directory for checksum validation. Use `exercise-extensions` / `pgtap-smoke` as pre-flight checks before shipping code that depends on the bundled extensions (vector, PostGIS, AGE, pgcrypto, citext, hstore, pg_partman, etc.). Reach for the dedicated pg_partman helpers when managing partition sets: `partman-show-config` enumerates tracked parents, `partman-maintenance` forces `run_maintenance_proc()`, and `partman-create-parent` wraps `create_parent()` with safe defaults. Need a lightweight job queue? `async-queue bootstrap` wires up the `asyncq` schema plus helper functions so workers can call `asyncq.dequeue` / `asyncq.complete` without extra tooling. Use `version-status` (or check the nightly `version_status.csv`) to spot drift between installed components and upstream releases. When reclaiming space, reach for `compact --level 1/2` to review autovacuum & pg_squeeze output, `compact --level 3 --tables schema.table` for a targeted pg_repack run, and reserve `compact --level 4 --scope ... --yes` (VACUUM FULL) for scheduled downtime. Cron already handles nightly pg_squeeze refreshes, hourly `pg_partman_bgw` maintenance (targeting the `postgres` database), a `VACUUM (ANALYZE, SKIP_LOCKED, PARALLEL 4)` pass, and `pg_stat_statements_reset()` at 04:00 UTC. Set `DAILY_EMAIL_REPORT=true` plus `DAILY_REPORT_RECIPIENT` in `.env` if you want the HTML maintenance report mailed via `sendmail`; adjust `DAILY_BUFFERCACHE_LIMIT` to control how many relations are captured per snapshot.
15
15
16
16
## Coding Style & Naming Conventions
17
17
Shell scripts should follow the Google Shell Style Guide: `#!/usr/bin/env bash`, `set -euo pipefail`, two-space indentation, and lowercase `snake_case` function names. Name init scripts with zero-padded prefixes (`01-init-db-user-creation.sh`) to enforce execution order. Keep environment variables upper snake case and explain new ones in `.env.example`. Template files should remain `.tpl` and consume `${VAR}` placeholders only.
The same bundle is installed into `template1` so freshly created databases inherit the tooling automatically.
39
+
40
+
`pg_partman_bgw` is preloaded with a one-hour interval targeting the `postgres` database under the `postgres` superuser. Adjust `pg_partman_bgw.dbname`/`role` in `postgresql.conf.tpl` (or via `postgresql.pgtune.conf`) if you manage partitions from a different control schema.
41
+
42
+
Use `./scripts/manage.sh partman-show-config` to inspect tracked parents, `partman-maintenance` to run `run_maintenance_proc()` on demand, and `partman-create-parent schema.table control_column '1 day'` to bootstrap new partition sets without hand-writing SQL.
43
+
44
+
Run `./scripts/manage.sh async-queue bootstrap` when you want a lightweight background-job queue. It provisions an `asyncq.jobs` table plus helpers (`enqueue`, `dequeue`, `complete`, `fail`, `extend_lease`) that rely on `FOR UPDATE SKIP LOCKED`, `pg_notify`, and UUID leasing. Point a worker at the queue with `SELECT * FROM asyncq.dequeue('default');` in a loop and call `asyncq.complete(...)` or `asyncq.fail(...)` as you process jobs.
45
+
27
46
## Quick Start
28
47
1. Copy the template: `cp .env.example .env` and customize credentials, host paths, and network settings (keep the generated `.env` local and untracked).
29
48
2. Build and start the stack:
@@ -73,17 +92,23 @@ Keep `data/` out of version control—it holds live cluster state and backup arc
73
92
|`audit-replication`| Summarise follower lag and sync state. |
The CLI sources modular helpers from `scripts/lib/` so each function can be imported by tests or future automation.
85
110
86
-
`daily-maintenance` now emits a richer bundle under `backups/daily/<YYYYMMDD>/`, including `pg_stat_statements` snapshots, role/extension/autovacuum/replication CSVs, pg_cron schedules, pg_squeeze activity, and a security checklist alongside logs, dumps, pgBadger HTML, and pgaudit summaries. Pair those reports with `config-check` to keep the rendered configs aligned with the templates. Tune the thresholds via `DAILY_PG_STAT_LIMIT`, `DAILY_DEAD_TUPLE_THRESHOLD`, `DAILY_DEAD_TUPLE_RATIO`, and `DAILY_REPLICATION_LAG_THRESHOLD` as needed.
111
+
`daily-maintenance` now emits a richer bundle under `backups/daily/<YYYYMMDD>/`, including `pg_stat_statements` snapshots, `pg_buffercache` heatmaps, role/extension/autovacuum/replication CSVs, pg_cron schedules, pg_squeeze activity, and a security checklist alongside logs, dumps, pgBadger HTML, and pgaudit summaries. The workflow also runs `partman.run_maintenance_proc()` across each database so freshly created partitions land even if the background worker interval has not elapsed, and it records any version drift in `version_status.csv` (focusing on out-of-date components). Pair those reports with `config-check` to keep the rendered configs aligned with the templates. Tune the thresholds via `DAILY_PG_STAT_LIMIT`, `DAILY_BUFFERCACHE_LIMIT`, `DAILY_DEAD_TUPLE_THRESHOLD`, `DAILY_DEAD_TUPLE_RATIO`, and `DAILY_REPLICATION_LAG_THRESHOLD` as needed.
87
112
88
113
Nightly cron jobs also refresh pg_squeeze targets, reset `pg_stat_statements`, and run a safe `VACUUM (ANALYZE, SKIP_LOCKED, PARALLEL 4)` so statistics stay current without blocking hot tables.
89
114
@@ -105,7 +130,7 @@ All runs write logs under `backups/` for auditing (`pg_repack-*.log`, `vacuum-fu
105
130
-**CI Workflow:**`.github/workflows/ci.yml` builds the image, runs `python -m pytest -k full_workflow`, and uploads generated backups for inspection.
106
131
-**Smoke Test:**`tests/test_manage.py` spins up a disposable environment, exercises key CLI commands (including `daily-maintenance`, pgBackRest, and `upgrade`), and tears everything down. Run locally with `python -m pytest -k full_workflow` (Docker required).
107
132
-**Fast Tests:**`tests/test_lightweight.py` validates offline flows like help output and the vendored tooling without needing Docker.
108
-
-**Extension Smoke:**`./scripts/manage.sh exercise-extensions --db <name>` plus `pgtap-smoke` provide quick feedback that pgvector, PostGIS, Apache AGE, and pgTap are ready for use.
133
+
-**Extension Smoke:**`./scripts/manage.sh exercise-extensions --db <name>` plus `pgtap-smoke` provide quick feedback that the entire core extension bundle (vector/PostGIS/AGE/hstore/citext/pgcrypto/pg_partman/etc.) is ready for use.
109
134
-**Documentation:**`AGENTS.md` offers contributor runbooks and on-call notes.
110
135
111
136
## Credits
@@ -118,9 +143,12 @@ Thank you to the maintainers and communities behind the components that make cor
0 commit comments