refactor(core): compose feature requests into shared host capabilities

.github/workflows/ci.yml

@@ -35,24 +35,91 @@ jobs:
           umask 177
           printf 'ci-test-only\n' > .vault_pass
 
+      - name: Stage matching provider collection branches
+        if: github.event_name == 'pull_request'
+        env:
+          GITHUB_TOKEN: ${{ github.token }}
+        run: |
+          set -euo pipefail
+
+          clone_matching_branch() {
+            local repo="$1"
+            local dest="$2"
+            local remote="https://x-access-token:${GITHUB_TOKEN}@github.com/CrownOpsEng/${repo}.git"
+
+            if git ls-remote --exit-code --heads "${remote}" "${GITHUB_HEAD_REF}" >/dev/null 2>&1; then
+              git clone --depth 1 --branch "${GITHUB_HEAD_REF}" "${remote}" "${dest}"
+            else
+              echo "No matching ${GITHUB_HEAD_REF} branch in ${repo}; using the default collection source."
+            fi
+          }
+
+          mkdir -p .deps
+          clone_matching_branch crownops-deploy-base .deps/crownops-deploy-base
+          clone_matching_branch crownops-deploy-services .deps/crownops-deploy-services
+
       - name: Install collections
-        run: ./scripts/install-collections.sh
+        run: |
+          set -euo pipefail
+
+          if [[ -f "${GITHUB_WORKSPACE}/.deps/crownops-deploy-base/galaxy.yml" ]]; then
+            export CROWNOPS_BASE_COLLECTION_SOURCE="${GITHUB_WORKSPACE}/.deps/crownops-deploy-base"
+          fi
+
+          if [[ -f "${GITHUB_WORKSPACE}/.deps/crownops-deploy-services/galaxy.yml" ]]; then
+            export CROWNOPS_SERVICES_COLLECTION_SOURCE="${GITHUB_WORKSPACE}/.deps/crownops-deploy-services"
+          fi
+
+          ./scripts/install-collections.sh
 
       - name: Syntax check preflight
         run: ansible-playbook --syntax-check -i inventories/prod/hosts.yml playbooks/preflight.yml
 
       - name: Smoke test preflight placeholder validation
         run: bash tests/smoke/test_preflight_placeholder_scan.sh
 
+      - name: Smoke test preflight missing nested contract roots
+        run: bash tests/smoke/test_preflight_missing_contract_roots.sh
+
       - name: Smoke test Tailscale tag validation
         run: bash tests/smoke/test_preflight_tailscale_tag_validation.sh
 
+      - name: Smoke test platform bindings public HTTPS composition
+        run: bash tests/smoke/test_platform_bindings_public_https.sh
+
+      - name: Smoke test platform bindings private mesh composition
+        run: bash tests/smoke/test_platform_bindings_private_mesh.sh
+
+      - name: Smoke test platform bindings preserve host extensions
+        run: bash tests/smoke/test_platform_bindings_host_extensions.sh
+
+      - name: Smoke test host Traefik preflight contract validation
+        run: bash tests/smoke/test_preflight_host_traefik_contract_validation.sh
+
       - name: Smoke test builder restic target normalization
         run: bash tests/smoke/test_builder_restic_target_name_normalization.sh
 
+      - name: Smoke test builder obsidian setup URI generation
+        run: bash tests/smoke/test_builder_obsidian_setup_uri_generation.sh
+
+      - name: Smoke test builder restic target keypair consistency
+        run: bash tests/smoke/test_builder_restic_target_keypair_consistency.sh
+
       - name: Smoke test restic target name preflight validation
         run: bash tests/smoke/test_preflight_restic_target_name_validation.sh
 
+      - name: Smoke test restic cache valid time preflight validation
+        run: bash tests/smoke/test_preflight_restic_cache_valid_time_validation.sh
+
+      - name: Smoke test broad backup dataset warning
+        run: bash tests/smoke/test_preflight_restic_broad_path_warning.sh
+
+      - name: Smoke test unsupported feature-owned jobs validation
+        run: bash tests/smoke/test_preflight_feature_owned_jobs_rejected.sh
+
+      - name: Smoke test Obsidian route name validation
+        run: bash tests/smoke/test_preflight_obsidian_route_name_validation.sh
+
       - name: Syntax check bootstrap
         run: ansible-playbook --syntax-check -i inventories/prod/hosts.yml playbooks/bootstrap.yml
 

README.md

@@ -22,7 +22,10 @@ Configuration model:
 - `inventories/prod/group_vars/all/main.yml` is the primary non-secret configuration surface
 - `inventories/prod/group_vars/all/vault.yml` holds secret values only
 - `.vault_pass` is the repo-local default Ansible Vault password file; keep it local, `0600`, and untracked
-- backup policy is expressed as `restic_targets`, `restic_backup_jobs`, and `restic_backup_contributions`
+- shared host capabilities live under `host.*`
+- feature contracts live under `features.*`
+- site-local composition derives shared ingress, backup datasets, and firewall requests before host roles reconcile them
+- host-owned `host.traefik.routes`, `host.restic.datasets`, and `host.ufw.requests` remain additive extension points and are preserved when the site layer composes shared inputs
 
 Read first:
 

docs/ARCHITECTURE.md

@@ -26,14 +26,34 @@ Current feature set:
 
 - Obsidian via the `crownops.deploy_services.obsidian_livesync` role
 
-`playbooks/backup.yml` uses the `crownops.deploy_services.restic_host_backups` role.
+Shared host capabilities:
 
-That backup layer is modeled as:
+- `crownops.deploy_services.host_traefik`
+- `crownops.deploy_services.host_restic`
+- `crownops.deploy_base.host_ufw`
+
+`playbooks/backup.yml` uses the `crownops.deploy_services.host_restic` role.
+
+This repo owns one site-local composition step through `roles/platform_bindings`.
+
+That composition layer derives:
+
+- `platform_ingress_routes`
+- `platform_backup_datasets`
+- `platform_ufw_requests`
+
+Then it builds the effective host contracts consumed by the shared roles:
+
+- `platform_host_traefik` appends composed routes to any host-owned `host.traefik.routes`
+- `platform_host_restic` appends composed datasets to any host-owned `host.restic.datasets`
+- `platform_host_ufw` appends composed firewall requests to any host-owned `host.ufw.requests`
+
+The backup layer is modeled as:
 
 - targets: backup destinations and transport credentials
-- jobs: logical host backup policies with schedule and retention
-- contributions: feature-specific paths and consistency hooks merged into named jobs
-- converge-time performance policy: no fact gathering for the dedicated backup play, SSH pipelining enabled in the repo Ansible config, and package cache reuse controlled through `restic_apt_cache_valid_time`
+- datasets: composed durable backup scopes owned by the site layer and feature/host boundaries
+- jobs: logical host-owned backup policy with schedule and retention
+- converge-time performance policy: no fact gathering for the dedicated backup play, SSH pipelining enabled in the repo Ansible config, and package cache reuse controlled through `host.restic.apt_cache_valid_time`
 - restore-first scope policy: back up durable state such as host identity, local markdown workspaces, CouchDB data, and Traefik ACME state, not broad service roots that can be rebuilt from Ansible
 
 `playbooks/lockdown.yml` consumes the reusable `crownops.deploy_base.network_lockdown` role so SSH lockdown policy stays consistent across site repos.

docs/CONFIG_WIZARD_SPEC.md

@@ -174,7 +174,7 @@ label: CouchDB admin password
 type: secret
 stage: features
 required: true
-when: feature_obsidian_enabled == true
+when: features.obsidian_livesync.enabled == true
 default: null
 target:
   file: inventories/prod/group_vars/all/vault.yml

docs/IMPLEMENTATION_STATUS.md

@@ -49,6 +49,7 @@ This file states exactly what is done, what is not done, and what gaps remain.
 5. Verify whether port 443 should be publicly reachable or source-restricted in your exact model.
 6. Join the host to Tailscale with the real auth approach you want.
 7. Prepare the backup targets and confirm the job/contribution model matches the host role.
+7. Prepare the backup targets and confirm the host job plus composed dataset model matches the host role.
 8. Decide how you will handle LiveSync passphrase ownership and recovery expectations.
 9. Run and validate the package in a controlled sequence, including the staged SSH lockdown.
 

docs/PREFLIGHT.md

@@ -21,15 +21,17 @@ ansible-playbook -i inventories/prod/hosts.yml playbooks/preflight.yml
 - base and ops domain presence and placeholder values
 - SSH public key population
 - Ubuntu release selector validity
-- HTTPS firewall exposure for Traefik on TCP 443 when enabled features require it
-- Traefik ACME email and DNS provider configuration
-- ACME provider env placeholders
-- CouchDB admin credentials when Obsidian is enabled
-- synced Obsidian account definitions
-- duplicate vault names, database names, or users
-- placeholder synced account passwords
+- nested `features.*` and `host.*` contract presence
+- flat legacy inventory variables that should no longer exist
+- Obsidian access mode, URL, CouchDB contract, and sync-account uniqueness
+- public HTTPS bindings through `host.traefik`
+- private mesh planning inputs and public-firewall leakage
+- restic target and job structure under `host.restic`
+- unsupported `host.restic.feature_owned_jobs`
+- composed platform bindings such as ingress routes, datasets, and firewall requests
 - path collisions between `vault_root` and `exports_root`
 - restic repository and password placeholders
+- broad backup dataset root warnings
 - Tailscale auth and bootstrap placeholders
 - placeholder marker sweep across key variables
 - remote connectivity probe

docs/QUICKSTART.md

@@ -71,11 +71,12 @@ Primary configuration surface:
 
 That file should remain the main place where you enable features and define non-secret behavior:
 
-- domains and ingress settings
+- top-level deployment identity, paths, and lockdown settings
 - host bootstrap settings
-- synced Obsidian account definitions
+- feature config under `features.*`
+- shared host capability config under `host.*`
 - local markdown workspace names
-- backup targets, logical jobs, feature contributions, and lockdown behavior
+- host backup targets and logical jobs, with backup datasets composed later by the site layer
 
 Secrets belong in:
 
@@ -110,9 +111,10 @@ At minimum set:
 - DNS and ACME values only for `public_https`
 - a concrete private mesh base URL only for `private_mesh`
 - keep `5984` off the public firewall in `private_mesh`; that path assumes your VPN or mesh provides reachability
-- synced account structure in `all/main.yml` and CouchDB passwords in `all/vault.yml` if Obsidian is enabled
+- synced account structure in `features.obsidian_livesync.couchdb.vaults` and CouchDB passwords in `all/vault.yml` if Obsidian is enabled
 - local markdown workspace names in `all.yml` if you want local-only content directories scaffolded
-- backup targets, backup jobs, and contribution wiring
+- backup targets and backup jobs under `host.restic`; dataset scope comes from site composition, not from feature-owned schedules
+- optional host-owned extensions such as `host.traefik.routes`, `host.restic.datasets`, and `host.ufw.requests` if this site needs extra ingress, backup scope, or firewall rules beyond the composed defaults
 - Tailscale hostname/tags in `all/main.yml` and optional auth key in `all/vault.yml`
 
 Notes:
@@ -123,7 +125,7 @@ Notes:
 - after a successful resumed run, the wizard best-effort securely deletes its own temporary resume-state file
 - Tailscale join is automated during bootstrap when `tailscale_auth_key` is set
 - if you intentionally leave `tailscale_auth_key` blank, join manually and then run `./scripts/ssh-lockdown.sh --confirm` after confirming SSH over Tailscale works
-- SFTP backup transport supports SSH keys on a per-target basis by storing `ssh_private_key` and `ssh_known_hosts` under each `restic_targets` entry, but the wizard now asks for a local `ssh_private_key_file` path so the key itself does not have to be pasted into the terminal or resume state
+- SFTP backup transport supports SSH keys on a per-target basis by storing `ssh_private_key` and `ssh_known_hosts` under each generated `host.restic.targets` entry, but the wizard now asks for a local `ssh_private_key_file` path so the key itself does not have to be pasted into the terminal or resume state
 - the wizard can guide SFTP backup targets by asking for host, user, path, and port, then deriving the restic repository URL and attempting `ssh-keyscan` automatically
 - for Linux backup destinations you control, the wizard can still generate a prerequisite setup script that prepares backup users, SSH keys, and repository paths first
 - staged SSH lockdown is two-phase: `./scripts/ssh-lockdown.sh --phase1-only` validates while preserving public SSH, and `./scripts/ssh-lockdown.sh --confirm` enables the restrictive path

docs/RESTORE.md

@@ -4,8 +4,8 @@
 
 1. Rebuild host base.
 2. Reapply service configuration from Ansible so `docker-compose.yml`, `local.ini`, and routing config are recreated from source of truth.
-3. Restore `{{ couchdb_dir }}/data` from restic after stopping CouchDB.
-4. If public HTTPS is enabled, restore `{{ traefik_acme_storage }}` so Traefik keeps its ACME account and issued certificates.
+3. Restore the `obsidian-couchdb-data` dataset after stopping CouchDB.
+4. If public HTTPS is enabled, restore the `traefik-acme` dataset so Traefik keeps its ACME account and issued certificates.
 5. Recreate or verify routing and HTTPS.
 6. Confirm database security objects and per-vault users.
 7. Reconnect device with correct CouchDB credentials.
@@ -28,7 +28,7 @@ Back up durable state only:
 
 - host identity and operator-managed security config under `/etc/ssh`, `/etc/fail2ban`, and `/etc/ufw`
 - local markdown workspaces under `{{ vault_root }}/workspaces`
-- CouchDB data under `{{ couchdb_dir }}/data`
-- Traefik ACME state in `{{ traefik_acme_storage }}` when public HTTPS is enabled
+- CouchDB data under `features.obsidian_livesync.couchdb.dir + '/data'`
+- Traefik ACME state under `host.traefik.acme_storage_path` when public HTTPS is enabled
 
-Do not treat generated compose files, rendered service config, package caches, or broad parent directories such as `/srv/crownops`, `{{ couchdb_dir }}`, or `{{ traefik_dir }}` as primary backup scope when the deployment is reproducible from Ansible.
+Do not treat generated compose files, rendered service config, package caches, or broad parent directories such as `/srv/crownops`, `features.obsidian_livesync.couchdb.dir`, or `host.traefik.layout_root` as primary backup scope when the deployment is reproducible from Ansible.