|
| 1 | +# Cassandra |
| 2 | + |
| 3 | +This app deploys a Cassandra 5.0 cluster in a single location. Each node runs as a stateful replica with its own persistent volume, forming a peer-to-peer cluster that distributes and replicates data across nodes according to the configured replication factor. The template includes optional scheduled backups (logical or physical) and periodic anti-entropy repair. |
| 4 | + |
| 5 | +## Architecture |
| 6 | + |
| 7 | +- **Cassandra cluster**: Multi-node cluster deployed in a single location where each node owns a slice of the token ring and replicates data to peers |
| 8 | +- **Per-node volumes**: Each node gets its own persistent volume so SSTable data survives restarts |
| 9 | +- **Repair** (optional): Scheduled cron job that runs `nodetool repair` across all nodes to keep data consistent |
| 10 | +- **Backup** (optional): Logical (`cqlsh COPY TO`) or physical (`nodetool snapshot`) backup to S3 or GCS |
| 11 | + |
| 12 | +## Configuration |
| 13 | + |
| 14 | +### Core Settings |
| 15 | + |
| 16 | +```yaml |
| 17 | +replicas: 3 # Number of Cassandra nodes |
| 18 | +replicationFactor: 3 # Copies of each partition stored across the cluster |
| 19 | + # Must not exceed replicas |
| 20 | + |
| 21 | +superuserPassword: supersecretpassword # Built-in cassandra superuser password |
| 22 | +username: username # Application user |
| 23 | +password: password # Application user password |
| 24 | +keyspaceName: mydatabase # Keyspace created on startup |
| 25 | + |
| 26 | +image: cassandra:5.0 |
| 27 | +cpu: 1 |
| 28 | +memory: 4Gi |
| 29 | +jvmHeapSize: 2G # Set to ~50% of memory — Cassandra needs the rest for off-heap cache |
| 30 | +clusterName: my-cassandra |
| 31 | +``` |
| 32 | +
|
| 33 | +**Volume** — set the initial storage capacity and optionally enable autoscaling: |
| 34 | +
|
| 35 | +```yaml |
| 36 | +volumes: |
| 37 | + data: |
| 38 | + initialCapacity: 10 # GiB |
| 39 | + autoscaling: |
| 40 | + maxCapacity: 100 |
| 41 | + minFreePercentage: 20 |
| 42 | + scalingFactor: 1.5 |
| 43 | +``` |
| 44 | +
|
| 45 | +Configure which workloads can reach Cassandra: |
| 46 | +
|
| 47 | +```yaml |
| 48 | +internal_access: |
| 49 | + type: same-gvc # Options: same-gvc, same-org, workload-list |
| 50 | + workloads: |
| 51 | + # Uncomment and specify workloads if using workload-list |
| 52 | + #- //gvc/GVC_NAME/workload/WORKLOAD_NAME |
| 53 | +``` |
| 54 | + |
| 55 | +- `same-gvc`: Allow access from all workloads in the same GVC |
| 56 | +- `same-org`: Allow access from all workloads in the org |
| 57 | +- `workload-list`: Allow access only from specified workloads |
| 58 | + |
| 59 | +## Connecting |
| 60 | + |
| 61 | +Each Cassandra replica is reachable via its own DNS name: |
| 62 | + |
| 63 | +``` |
| 64 | +Host: {release-name}-cassandra-{n}.{gvc}.cpln.local |
| 65 | +Port: 9042 (CQL, native transport) |
| 66 | +Username: {username} |
| 67 | +Password: {password} |
| 68 | +Keyspace: {keyspaceName} |
| 69 | +``` |
| 70 | + |
| 71 | +Provide multiple node hostnames as contact points in your application so it can discover the full cluster topology. |
| 72 | + |
| 73 | +## Replicas vs Replication Factor |
| 74 | + |
| 75 | +These are two separate settings that work together: |
| 76 | + |
| 77 | +- **`replicas`** — how many Cassandra nodes are deployed. More nodes means more capacity and better throughput, as the token ring is split across more nodes. |
| 78 | +- **`replicationFactor`** — how many copies of each partition are stored across the cluster. A replication factor of 3 means every row exists on 3 different nodes, so the cluster can survive 2 node failures without data loss (with `QUORUM` consistency). |
| 79 | + |
| 80 | +`replicationFactor` must not exceed `replicas` — you cannot store 3 copies of data across only 2 nodes. |
| 81 | + |
| 82 | +## Multi-Zone |
| 83 | + |
| 84 | +When `multiZone.enabled: true`, Control Plane spreads replicas across availability zones within the location: |
| 85 | + |
| 86 | +```yaml |
| 87 | +multiZone: |
| 88 | + enabled: true |
| 89 | +``` |
| 90 | +
|
| 91 | +With a replication factor of 3 across 3 zones, each zone holds one copy of every partition. The cluster survives a complete zone outage with no data loss, provided your client uses `LOCAL_QUORUM` consistency (reads and writes succeed with responses from the surviving 2 zones). |
| 92 | + |
| 93 | +Verify your selected location supports multi-zone before enabling this option. |
| 94 | + |
| 95 | +## Repair |
| 96 | + |
| 97 | +Cassandra uses eventual consistency — when nodes miss writes during downtime, data can drift out of sync. `nodetool repair` runs an anti-entropy process that compares and reconciles data across all replicas. Repair must complete across all nodes at least once within `gc_grace_seconds` (default: 10 days) to prevent deleted data from reappearing. |
| 98 | + |
| 99 | +The template includes a scheduled repair cron job: |
| 100 | + |
| 101 | +```yaml |
| 102 | +repair: |
| 103 | + enabled: true |
| 104 | + schedule: "0 2 * * 0" # Weekly, Sunday at 2am UTC |
| 105 | +``` |
| 106 | + |
| 107 | +The default weekly schedule satisfies the 10-day `gc_grace_seconds` requirement with margin. Do not disable repair in production or increase the interval beyond 10 days. |
| 108 | + |
| 109 | +Repair can be resource-intensive on large datasets. If it impacts query performance, consider running it during low-traffic windows or increasing node resources. |
| 110 | + |
| 111 | +## Backing Up |
| 112 | + |
| 113 | +Two backup modes are available: |
| 114 | + |
| 115 | +- **Logical** — exports keyspace tables as CSVs using `cqlsh COPY TO`, then uploads to cloud storage. Runs as a standalone cron workload on schedule. Suitable for smaller datasets or when portability matters. |
| 116 | +- **Physical** — creates SSTable snapshots using `nodetool snapshot` and syncs them to cloud storage. Runs as a sidecar container on each Cassandra replica. Faster and more space-efficient for large datasets, but backups are per-node and must be restored node-by-node. |
| 117 | + |
| 118 | +Set `backup.enabled: true`, choose a `type`, set `backup.provider`, and fill in the corresponding cloud block: |
| 119 | + |
| 120 | +```yaml |
| 121 | +backup: |
| 122 | + enabled: true |
| 123 | + type: logical # logical or physical |
| 124 | + image: ghcr.io/controlplane-com/backup-images/cassandra-backup:5.0 |
| 125 | + schedule: "0 2 * * *" # daily at 2am UTC |
| 126 | +
|
| 127 | + resources: |
| 128 | + cpu: 250m |
| 129 | + memory: 256Mi |
| 130 | +
|
| 131 | + provider: aws # aws or gcp |
| 132 | +
|
| 133 | + aws: |
| 134 | + bucket: my-backup-bucket |
| 135 | + region: us-east-1 |
| 136 | + cloudAccountName: my-backup-cloudaccount |
| 137 | + policyName: my-s3-policy |
| 138 | + prefix: cassandra/backups |
| 139 | +
|
| 140 | + gcp: |
| 141 | + bucket: my-backup-bucket |
| 142 | + cloudAccountName: my-cloud-account |
| 143 | + prefix: cassandra/backups |
| 144 | +``` |
| 145 | + |
| 146 | +### AWS S3 |
| 147 | + |
| 148 | +1. Create your S3 bucket. Set `aws.bucket` and `aws.region` to match. |
| 149 | + |
| 150 | +2. If you do not have a Cloud Account set up, refer to the docs to [Create a Cloud Account](https://docs.controlplane.com/guides/create-cloud-account). Set `aws.cloudAccountName` to match. |
| 151 | + |
| 152 | +3. Create an AWS IAM policy with the following JSON (replace `YOUR_BUCKET_NAME`): |
| 153 | + |
| 154 | +```json |
| 155 | +{ |
| 156 | + "Version": "2012-10-17", |
| 157 | + "Statement": [ |
| 158 | + { |
| 159 | + "Effect": "Allow", |
| 160 | + "Action": [ |
| 161 | + "s3:GetObject", |
| 162 | + "s3:PutObject", |
| 163 | + "s3:DeleteObject", |
| 164 | + "s3:ListBucket", |
| 165 | + "s3:GetObjectVersion", |
| 166 | + "s3:DeleteObjectVersion" |
| 167 | + ], |
| 168 | + "Resource": [ |
| 169 | + "arn:aws:s3:::YOUR_BUCKET_NAME", |
| 170 | + "arn:aws:s3:::YOUR_BUCKET_NAME/*" |
| 171 | + ] |
| 172 | + } |
| 173 | + ] |
| 174 | +} |
| 175 | +``` |
| 176 | + |
| 177 | +4. Set `aws.policyName` to the name of the policy created in step 3. |
| 178 | + |
| 179 | +### GCS |
| 180 | + |
| 181 | +1. Create your GCS bucket. Set `gcp.bucket` to match. |
| 182 | + |
| 183 | +2. If you do not have a Cloud Account set up, refer to the docs to [Create a Cloud Account](https://docs.controlplane.com/guides/create-cloud-account). Set `gcp.cloudAccountName` to match. |
| 184 | + |
| 185 | +**Important**: Add the `Storage Admin` role to the GCP service account created for the Cloud Account. |
| 186 | + |
| 187 | +## Restoring a Backup |
| 188 | + |
| 189 | +### Logical Restore |
| 190 | + |
| 191 | +Exec into the backup cron workload and run `restore.sh` with the timestamp of the backup you want to restore: |
| 192 | + |
| 193 | +```bash |
| 194 | +RESTORE_TIMESTAMP=2026-05-15T02-00-00Z /usr/local/bin/restore.sh |
| 195 | +``` |
| 196 | + |
| 197 | +The timestamp format matches the backup filename in your bucket (e.g. `cassandra/backups/2026-05-15T02-00-00Z/`). |
| 198 | + |
| 199 | +The script downloads the CSVs for the configured keyspace and replays them into Cassandra using `cqlsh COPY FROM`. Existing rows with matching primary keys are overwritten; rows not in the backup are left in place. |
| 200 | + |
| 201 | +### Physical Restore |
| 202 | + |
| 203 | +Physical backups are per-node — each replica backed up its own SSTable slice. To restore, exec into the **backup sidecar container** (not the cassandra container) on each replica that needs to be restored and run: |
| 204 | + |
| 205 | +```bash |
| 206 | +RESTORE_TIMESTAMP=2026-05-15T02-00-00Z /usr/local/bin/restore.sh |
| 207 | +``` |
| 208 | + |
| 209 | +The script downloads the snapshot files for that replica from `{prefix}/{timestamp}/{hostname}/`, writes them to the shared volume, then calls `nodetool import` to load the SSTables into the live Cassandra instance without a restart. |
| 210 | + |
| 211 | +**Important**: Repeat this on every replica. Because each node owns a different token range, restoring only one replica leaves the cluster with incomplete data. |
| 212 | + |
| 213 | +## Important Notes |
| 214 | + |
| 215 | +- **Minimum replicas for production**: Use at least 3 replicas with a replication factor of 3 so the cluster can survive a node failure while still achieving quorum |
| 216 | +- **JVM heap**: Set `jvmHeapSize` to approximately 50% of `memory` — Cassandra relies heavily on off-heap memory for bloom filters, row cache, and OS page cache |
| 217 | +- **gc_grace_seconds**: The default is 10 days. Ensure repair runs at least once within this window on all nodes, or deleted data may reappear after a node recovers from downtime |
| 218 | +- **Scaling up**: Adding replicas after initial deployment does not automatically rebalance data. Run `nodetool rebuild` on new nodes and then `nodetool cleanup` on existing nodes after scaling |
| 219 | +- **Multi-zone**: Verify your selected location supports multi-zone before enabling |
| 220 | + |
| 221 | +## Supported External Services |
| 222 | + |
| 223 | +- [Cassandra Documentation](https://cassandra.apache.org/doc/latest/) |
| 224 | +- [Cassandra Driver Documentation](https://docs.datastax.com/en/developer/driver-matrix/doc/common/driverMatrix.html) |
0 commit comments