Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions changelogs/fragments/aap_password_rotate-new-role.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
---
major_changes:
- >-
aap_password_rotate - new role to rotate PostgreSQL database passwords
and admin user passwords across all AAP 2.7 components (controller,
gateway, hub, EDA). Supports Podman (via installer re-run) and Operator
(via K8s secret patching) deployment types, with optional external
database handling through a user-provided hook tasks file.
226 changes: 226 additions & 0 deletions roles/aap_password_rotate/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,226 @@
# aap_password_rotate

Rotates PostgreSQL database passwords and admin user passwords across all Ansible Automation Platform 2.7 components: Controller, Gateway, Event-Driven Ansible (EDA), and Automation Hub.

Supports both **podman** (containerised installer) and **operator** (Kubernetes/OpenShift) deployments, with both **installer-managed** and **external** databases.

> This role complements the `aap_secret_rotate` role, which handles SECRET_KEY (encryption key) rotation. These are two separate operations: password rotation changes authentication credentials, while secret rotation changes the keys used to encrypt database fields.

## Requirements

- Ansible >= 2.15
- AAP 2.7 (tested on 2.7.2 and 2.7.4)
- For podman: SSH access to the AAP host with podman permissions
- For operator: `kubectl` or `oc` with cluster-admin or namespace-scoped permissions
- For external DB: no special requirements (the role pauses or runs a user-provided hook)

## What Gets Rotated

### Database Passwords

| Component | DB User | Inventory Variable |
| --- | --- | --- |
| Controller | `awx` | `controller_pg_password` |
| Gateway | `gateway` | `gateway_pg_password` |
| EDA | `eda` | `eda_pg_password` |
| Hub | `pulp` | `hub_pg_password` |
| PostgreSQL superuser | `postgres` | `postgresql_admin_password` |

### Admin User Passwords

| Component | Method |
| --- | --- |
| Gateway | Django shell via `aap-gateway-manage shell` (`get_user_model().set_password`) |
| Controller | Django shell via `awx-manage shell` (`get_user_model().set_password`) |
| Hub | `pulpcore-manager reset-admin-password --password <pw>` |
| EDA | `aap-eda-manage update_password --username admin --password <pw>` |

> **Note:** Gateway and Controller use the Django ORM directly instead of the
> interactive `changepassword` command, which requires a TTY that is not
> available inside `podman exec` or `kubectl exec` contexts.

## How It Works

### Podman (via installer, recommended)

Per [KCS 7145426](https://access.redhat.com/solutions/7145426), the containerised installer natively supports password rotation:

1. If rotating the postgres superuser: `ALTER ROLE postgres` first
2. Update the inventory file with new `*_pg_password` values
3. Re-run the installer (handles ALTER ROLE for component users, updates podman secrets, restarts containers)
4. Rotate admin passwords via management commands

### Podman (manual, for external DB)

When the installer cannot reach the external database:

1. `ALTER ROLE` for each component DB user directly on the external PostgreSQL
2. Update podman secrets
3. Restart component containers
4. Rotate admin passwords via management commands

### Operator

1. `ALTER ROLE` for each component via exec into the PG pod (internal DB), or pause/hook for user action (external DB)
2. Patch Kubernetes `postgres-configuration` Secrets with new passwords
3. Rollout restart all component Deployments
4. Rotate admin passwords via exec into component pods
5. Patch `admin-password` Secrets to keep them in sync (per KCS 7144197)

## Usage

### Dry run (safe, read-only)

```yaml
- name: Dry run password rotation
hosts: aap
roles:
- role: infra.aap_utilities.aap_password_rotate
aap_password_rotate_dry_run: true
```

### Full rotation, podman (DB + admin passwords)

```yaml
- name: Rotate all passwords
hosts: aap
roles:
- role: infra.aap_utilities.aap_password_rotate
aap_password_rotate_deployment_type: podman
aap_password_rotate_scope:
- db
- admin
aap_password_rotate_include_postgres_admin: true
aap_password_rotate_podman_use_installer: true
```

### DB passwords only, operator

```yaml
- name: Rotate DB passwords
hosts: localhost
connection: local
roles:
- role: infra.aap_utilities.aap_password_rotate
aap_password_rotate_deployment_type: operator
aap_password_rotate_namespace: my-aap
aap_password_rotate_cr_name: my-aap
aap_password_rotate_scope:
- db
```

### Custom passwords

```yaml
- name: Rotate with specific passwords
hosts: aap
roles:
- role: infra.aap_utilities.aap_password_rotate
aap_password_rotate_deployment_type: podman
aap_password_rotate_controller_pg_password: "MyNewCtrlPw-2026!"
aap_password_rotate_gateway_pg_password: "MyNewGwPw-2026!"
aap_password_rotate_eda_pg_password: "MyNewEdaPw-2026!"
aap_password_rotate_hub_pg_password: "MyNewHubPw-2026!"
```

### External database (interactive)

When `external_db: true`, the role generates new passwords, writes `ALTER ROLE` SQL
to a helper file, and pauses for you to apply the SQL on your external database.
After you confirm, the role patches application secrets and restarts services.

```yaml
- name: Rotate with external DB (interactive pause)
hosts: localhost
connection: local
roles:
- role: infra.aap_utilities.aap_password_rotate
aap_password_rotate_deployment_type: operator
aap_password_rotate_namespace: my-aap
aap_password_rotate_cr_name: my-aap
aap_password_rotate_external_db: true
```

### External database (custom hook)

If you want to automate the external DB password change, provide your own tasks
file via `aap_password_rotate_external_db_tasks`. The role calls `include_tasks`
on it instead of pausing. Your tasks file receives these variables:

| Variable | Type | Description |
| --- | --- | --- |
| `aap_password_rotate_components` | list | Components being rotated (e.g. `[controller, hub, eda, gateway]`) |
| `__pw_db_users` | dict | Component to DB username (e.g. `{controller: automationcontroller}`) |
| `__pw_db_passwords` | dict | Component to new password |
| `__pw_postgres_admin_password` | str | New postgres superuser password (when `include_postgres_admin: true`) |

Example hook for AWS RDS:

```yaml
# my_rds_password_rotate.yml
- name: Update RDS password for each component
amazon.aws.rds_instance:
db_instance_identifier: "aap-{{ item }}"
master_user_password: "{{ __pw_db_passwords[item] }}"
loop: "{{ aap_password_rotate_components }}"
no_log: true
```

Example hook using `psql` on a bastion host:

```yaml
# my_bastion_alter_role.yml
- name: ALTER ROLE via psql on bastion
ansible.builtin.command:
cmd: >-
psql -h {{ my_pg_host }} -U postgres -c
"ALTER ROLE {{ __pw_db_users[item] }} WITH PASSWORD '{{ __pw_db_passwords[item] }}';"
loop: "{{ aap_password_rotate_components }}"
delegate_to: bastion
no_log: true
```

Playbook using the hook:

```yaml
- name: Rotate with external DB (automated via hook)
hosts: localhost
connection: local
roles:
- role: infra.aap_utilities.aap_password_rotate
aap_password_rotate_deployment_type: operator
aap_password_rotate_namespace: my-aap
aap_password_rotate_cr_name: my-aap
aap_password_rotate_external_db: true
aap_password_rotate_external_db_tasks: "{{ playbook_dir }}/my_rds_password_rotate.yml"
```

## Role Variables

See [`defaults/main.yml`](defaults/main.yml) for all configurable variables and [`meta/argument_specs.yml`](meta/argument_specs.yml) for full documentation.

## Verification

After rotation, the role automatically verifies:

1. Gateway ping responds (HTTP 200)
2. Admin authentication works with the new password
3. Controller, Hub, and EDA APIs respond
4. DB connectivity with new passwords (podman only)

## Related

- `aap_secret_rotate`: Rotates SECRET_KEY (encryption keys) across AAP components
- [KCS 7145426](https://access.redhat.com/solutions/7145426): How to rotate PostgreSQL database passwords in AAP 2.7 Containerized
- [KCS 7100528](https://access.redhat.com/solutions/7100528): How to change PostgreSQL Database password of gateway
- [KCS 6746191](https://access.redhat.com/solutions/6746191): How to change Admin and PostgreSQL Database Passwords (AAP 2.4 and earlier)
- [KCS 7130353](https://access.redhat.com/solutions/7130353): How to change Automation Gateway Admin Password on AAP 2.5
- [KCS 7144197](https://access.redhat.com/solutions/7144197): How to find auto-generated passwords in AAP

## License

GPL-3.0-or-later

## Author

Alexey Masolov (@amasolov), Red Hat
98 changes: 98 additions & 0 deletions roles/aap_password_rotate/defaults/main.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@
---
# Deployment type: "podman" or "operator" (auto-detected if omitted)
aap_password_rotate_deployment_type: ""

# What to rotate: "db", "admin", or both
aap_password_rotate_scope:
- db
- admin

# Components to rotate (order: Gateway last for session continuity)
aap_password_rotate_components:
- controller
- hub
- eda
- gateway

# Dry run: report what would change without writing
aap_password_rotate_dry_run: false

# Backup directory for current passwords (on the control node)
aap_password_rotate_backup_dir: "{{ playbook_dir }}/password_backups/{{ ansible_date_time.date | default(lookup('pipe', 'date +%Y-%m-%d')) }}"

# Custom passwords (empty = auto-generate a 24-char random password)
aap_password_rotate_controller_pg_password: ""
aap_password_rotate_gateway_pg_password: ""
aap_password_rotate_eda_pg_password: ""
aap_password_rotate_hub_pg_password: ""
aap_password_rotate_postgres_admin_password: ""

# Whether to rotate the postgresql_admin_password (postgres superuser)
aap_password_rotate_include_postgres_admin: false

# Admin user passwords (empty = auto-generate)
aap_password_rotate_gateway_admin_password: ""
aap_password_rotate_controller_admin_password: ""
aap_password_rotate_hub_admin_password: ""
aap_password_rotate_eda_admin_password: ""

# Password generation settings
aap_password_rotate_password_length: 24
aap_password_rotate_password_chars: "ascii_letters,digits"

# ── Podman-specific ────────────────────────────────────────────────────
# Path to the containerised installer inventory file
aap_password_rotate_installer_inventory: ""

# Path to the containerised installer directory (contains ansible.containerized_installer)
aap_password_rotate_installer_dir: ""

# Whether to re-run the installer for DB password rotation (recommended).
# If false, the role does manual ALTER ROLE + podman secret update + restart.
aap_password_rotate_podman_use_installer: true

# Container names (defaults match AAP 2.7 containerised installer)
aap_password_rotate_controller_container: "automation-controller-task"
aap_password_rotate_gateway_container: "automation-gateway"
aap_password_rotate_eda_container: "automation-eda-api"
aap_password_rotate_hub_container: "automation-hub-api"
aap_password_rotate_pg_container: "postgresql"

# Podman systemd service scope
aap_password_rotate_systemd_scope: "user"

# ── Operator-specific ──────────────────────────────────────────────────
aap_password_rotate_namespace: "aap"
aap_password_rotate_cr_name: "aap"
aap_password_rotate_rollout_timeout: 300

# Path to the kubectl-compatible CLI binary (auto-detected if empty)
aap_password_rotate_kubectl_binary: ""

# ── External database ─────────────────────────────────────────────────
# Set to true when the database is external (not managed by the installer/operator).
# When true, the role generates ALTER ROLE SQL, writes it to a helper file,
# and pauses for the user to run the SQL on their external database manually.
# After the user confirms, the role proceeds to update application secrets
# and restart services.
aap_password_rotate_external_db: false

# Skip the interactive pause (for non-interactive/CI usage).
# When true (and external_db is true), the role assumes ALTER ROLE has already
# been applied and proceeds directly to patching secrets and restarting.
aap_password_rotate_external_db_no_pause: false

# Path to a custom tasks file for external DB password changes.
# When set (and external_db is true), the role includes this file
# instead of pausing. The tasks receive password variables and are
# expected to apply ALTER ROLE or equivalent on the external DB.
# If empty, the role falls back to the pause-and-show-SQL flow.
#
# Available variables inside the hook:
# aap_password_rotate_components - list of components being rotated
# __pw_db_users - dict: component -> DB username
# __pw_db_passwords - dict: component -> new password
# __pw_postgres_admin_password - new postgres superuser password
# (only when include_postgres_admin is true)
aap_password_rotate_external_db_tasks: ""
...
Loading
Loading