feat(funkwhale): optional nightly pg_dump → S3-compatible storage#32
Open
0cwa wants to merge 3 commits into
Open
feat(funkwhale): optional nightly pg_dump → S3-compatible storage#320cwa wants to merge 3 commits into
0cwa wants to merge 3 commits into
Conversation
…ugin Adds roles/docker, a generic role that installs the upstream Docker apt repository and the v2 'docker compose' plugin. No playbook references it yet -- it is consumed by the upcoming funkwhale role and is intentionally generic so any future containerised service can reuse it. Highlights: - pulls Docker's official GPG key into /etc/apt/keyrings - installs docker-ce + cli + containerd.io + buildx + compose plugin - exposes docker_group_users so the funkwhale service user can run docker without sudo Verified with: yamllint -c .yamllint.yaml .
Adds a Funkwhale role and playbook for music.bonfire.link.
Architecture:
client → host nginx (TLS, big bodies)
→ front container (in compose stack)
→ api / celery / postgres / redis (in compose stack)
audio → S3-compatible object storage (Storj gateway by default),
streamed directly to clients via signed URLs
Highlights:
- public + federated by default; e-mail verification on
- 5G upload ceiling (4h FLAC/WAV sets); proxy_request_buffering off
- direct streaming from Storj (PROXY_MEDIA=false +
AWS_QUERYSTRING_AUTH=true); the bucket can stay private
- storage backend toggle (s3 | local) for Vagrant/dev
- assert.yml refuses placeholder secrets; warns when registration
is on without real SMTP
- depends on roles/docker (added in feat/docker-role); reuses
common, fail2ban, certbot and nginx roles untouched
- public-repo-safe: no real secrets committed; vault layout
documented in roles/funkwhale/README.md
Two add-ons land in follow-up PRs and are explicitly out of scope
here:
- feat/funkwhale-anubis — Anubis bot protection
- feat/funkwhale-backup — nightly pg_dump → S3
Verified with:
yamllint -c .yamllint.yaml .
ansible-playbook --syntax-check -i hosts/prod playbooks/funkwhale.yml
jinja render of every template (compose YAML re-parses cleanly)
Adds an opt-in backup pipeline gated behind funkwhale_backup_enabled
(default false). When enabled, the role:
- installs awscli
- drops a backup script + 0600-mode env-file with credentials
- installs a systemd service + timer (default 03:30 UTC nightly,
randomised by 15 min)
- extends assert.yml to require S3 credentials/bucket when the
feature is on
The script streams 'pg_dump | gzip | aws s3 cp -' so the dump never
touches the host disk in plaintext. After the upload it lists the
host's prefix and deletes objects older than
funkwhale_backup_retention_days (default 30).
Defaults reuse the media bucket under backups/postgres/<host>/, so
operators don't need a second Storj bucket; override
funkwhale_backup_bucket if they want isolation.
Verified with:
yamllint -c .yamllint.yaml .
ansible-lint --profile safety roles/funkwhale (passes)
ansible-playbook --syntax-check -i hosts/prod playbooks/funkwhale.yml
jinja-render of all four backup templates
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds opt-in nightly
pg_dump → S3-compatible storagefor the Funkwhale database. Default: off.What
When
funkwhale_backup_enabled: true, the role:awscli0600-mode env file with credentialsservice+timer(default:*-*-* 03:30:00UTC, randomised by 15 min)tasks/assert.ymlto require S3 credentials/bucket when the feature is onHow it streams
The dump never touches the host disk in plaintext. After upload, the script lists the host's prefix and prunes objects older than
funkwhale_backup_retention_days(default30).Defaults
Reuses the media bucket under
backups/postgres/<host>/so operators don't need a second Storj bucket. Overridefunkwhale_backup_bucketfor isolation.Restore command lives in
roles/funkwhale/README.md.Verified with
Stacking
Depends on
feat/funkwhale-core. After that merges, rebase this ontomaster.Parallel with
feat/funkwhale-anubis— they touch disjoint code paths.