Skip to content

Commit cc28add

Browse files
committed
deploy(controlplane): drop snapshot ZIPs to the volume by default
Customers expect Signals to write out the snapshot ZIPs, not only serve them on demand via GET /export. Enable the native scheduled auto-export (#350) in the catalog template: after each collection cycle Signals drops one ZIP per database into export.dest (default /data/exports on the persistent volumeset), named <instance>-t<targetID>-<UTC>.zip. The directory is auto-created. - values.yaml: export.onCollect (default true) + export.dest (/data/exports). - workload-signals.yaml: emit SIGNALS_EXPORT_ON_COLLECT/SIGNALS_EXPORT_DEST when enabled. - _helpers.tpl: require export.dest when onCollect is true. - README.md: pull-vs-drop, filename pattern, retrieval (exec/cp or a volume-sharing sidecar), and the accumulation caveat (files are timestamped, never pruned — sync out and delete, or size storage.capacity). Verified live in the elevarq GVC against staging RDS: 'scheduled export written files=1 dest=/data/exports' and the ZIP present on the volume. closes #381
1 parent 265ba41 commit cc28add

4 files changed

Lines changed: 45 additions & 0 deletions

File tree

deploy/controlplane/signals/versions/1.0.0/README.md

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -59,6 +59,8 @@ The image is pinned by immutable digest
5959
| `collection.retentionDays` | `30` | Snapshot retention window |
6060
| `logLevel` | `info` | `debug` \| `info` \| `warn` \| `error` |
6161
| `metrics.enabled` | `false` | Prometheus `/metrics` (behind the API token) |
62+
| `export.onCollect` | `true` | Drop one ZIP per database into `export.dest` after each cycle (see Snapshot exports) |
63+
| `export.dest` | `/data/exports` | Directory for dropped ZIPs — keep it on the volume (`/data...`) |
6264
| `resources.cpu` / `resources.memory` | `100m` / `128Mi` | |
6365
| `storage.capacity` | `10` | Volume size in GiB (Control Plane minimum is 10) |
6466

@@ -73,6 +75,27 @@ The image is pinned by immutable digest
7375
- **Download a snapshot:** `GET /export` (bearer auth) → a `signals-snapshot.v1` ZIP.
7476
- **Metrics:** `GET /metrics` (bearer auth) when `metrics.enabled=true`.
7577

78+
## Snapshot exports
79+
80+
Snapshots are always available two ways:
81+
82+
- **Pull (always on):** `GET /export` (bearer) streams a `signals-snapshot.v1` ZIP.
83+
- **Drop to disk (`export.onCollect`, on by default):** after every collection
84+
cycle Signals writes one ZIP per database into `export.dest` (default
85+
`/data/exports` on the persistent volume), named
86+
`<instance>-t<targetID>-<UTC>.zip`. The directory is created automatically.
87+
88+
Retrieving dropped files: list/copy them with
89+
`cpln workload exec <name> --gvc <gvc> -- ls -la /data/exports`, or run a
90+
sidecar/second workload that mounts the same volume and syncs the directory to
91+
object storage.
92+
93+
> **Files accumulate.** Dropped ZIPs are timestamped and never overwritten, and
94+
> Signals does not prune them (only the SQLite store honours
95+
> `collection.retentionDays`). Sync them out and delete, or size
96+
> `storage.capacity` for your cadence and retention. Set `export.onCollect=false`
97+
> to disable dropping and rely on `GET /export` only.
98+
7699
## Security model
77100

78101
- Read-only enforced in three layers: static SQL linting, session

deploy/controlplane/signals/versions/1.0.0/templates/_helpers.tpl

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -72,6 +72,9 @@ Validation — fail fast on missing/invalid inputs before any resource renders.
7272
{{- if not .Values.postgres.caCert -}}
7373
{{- fail (printf "postgres.caCert (PEM CA bundle) is required for sslMode '%s'. For managed PostgreSQL use the provider bundle (e.g. Amazon RDS: https://truststore.pki.rds.amazonaws.com/global/global-bundle.pem)." .Values.postgres.sslMode) -}}
7474
{{- end -}}
75+
{{- if and .Values.export.onCollect (not .Values.export.dest) -}}
76+
{{- fail "export.dest is required when export.onCollect is true (use a path on the persistent volume, e.g. /data/exports)" -}}
77+
{{- end -}}
7578
{{- end -}}
7679

7780
{{/*

deploy/controlplane/signals/versions/1.0.0/templates/workload-signals.yaml

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,13 @@ spec:
3737
value: {{ .Values.collection.retentionDays | quote }}
3838
- name: SIGNALS_METRICS_ENABLED
3939
value: {{ .Values.metrics.enabled | quote }}
40+
{{- if .Values.export.onCollect }}
41+
# Drop one ZIP per database into dest after each collection cycle.
42+
- name: SIGNALS_EXPORT_ON_COLLECT
43+
value: "true"
44+
- name: SIGNALS_EXPORT_DEST
45+
value: {{ .Values.export.dest | quote }}
46+
{{- end }}
4047
# --- PostgreSQL target (customer-supplied external database) ---
4148
- name: SIGNALS_TARGET_NAME
4249
value: {{ .Values.postgres.name | default "default" | quote }}

deploy/controlplane/signals/versions/1.0.0/values.yaml

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -63,6 +63,18 @@ collection:
6363
pollInterval: "5m" # time between collection cycles (e.g. 5m, 1h)
6464
retentionDays: 30 # days of snapshots retained in the local SQLite store
6565

66+
# ---------------------------------------------------------------------------
67+
# Snapshot export. Signals always serves snapshots on demand via GET /export.
68+
# With onCollect enabled it ALSO drops one ZIP per database into `dest` after
69+
# every collection cycle (files are timestamped and accumulate — see README:
70+
# they are not pruned, so sync them out and/or size storage.capacity).
71+
# `dest` must be on the persistent volume (/data...) to survive restarts; the
72+
# directory is created automatically.
73+
# ---------------------------------------------------------------------------
74+
export:
75+
onCollect: true
76+
dest: "/data/exports"
77+
6678
# debug | info | warn | error
6779
logLevel: "info"
6880

0 commit comments

Comments
 (0)