Skip to content

Separate scalable local and AWS runtime roles - #45

Merged
xormania merged 1 commit into
devfrom
feat/scalable-deployments
Oct 6, 2026
Merged

xormania merged 1 commit into
devfrom
feat/scalable-deployments

Conversation

@xormania

@xormania xormania commented Oct 6, 2026 •

Copy link
Copy Markdown
Owner

Why is this PR needed?

Hosted deployments need to scale acquisition and processing independently while preserving a straightforward local Docker Compose setup. Local and AWS role configuration must express the same operational boundaries.

What does this PR change?

Keeps the default combined-worker Compose experience and adds scalable acquisition/processing profiles, replicated API routing and a managed-auth overlay. External PostgreSQL and the polling overlay now cover every stage role. Shared environment anchors forward provider, archive, quota and usage settings consistently; source access is writable for acquisition and read-only for processing/API.

CloudFormation adds independently sized API, acquisition, processing, dispatcher and migration tasks, separate stage queues/DLQs and scoped IAM. Managed-auth wiring preserves the health credential separation introduced in PR42. Existing combined queue upgrade instructions remain explicit; service counts default to zero for operator provisioning.

Documents public backend/private hosted product boundaries and local-to-cloud operation without embedding billing plans or private inference behavior.

Evidence that it solves the reason

Acceptance condition Baseline evidence Result and verification
Scale stage roles independently Combined worker/task configuration Separate stage queue/task and Compose role contracts
Compose overlays combine correctly Static credentials can leak through overlays All overlay combination/rendering and managed-auth tests
Runtime and cloud settings agree Numeric bounds and secrets can diverge Actual owner parsing, CloudFormation lint and configuration tests

117 real Compose/cloud/health/configuration tests passed locally; cfn-lint, HAProxy syntax, Ruff, Mypy and Bandit passed. Full Python 3.11/3.13 offline suites and live-container Compose smoke run in CI.

Changelog

Added scalable deployment profiles, role configuration and upgrade guidance under Unreleased.

xormania commented Oct 6, 2026

Copy link
Copy Markdown
Owner Author

@codex review

xormania commented Oct 6, 2026

Copy link
Copy Markdown
Owner Author

@codex security review

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 6, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-06T01:43:51.528672Z b6f361e Manual request
🔒 Security Review ✅ Completed 2026-10-06T01:44:23.842454Z b6f361e Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. 🎉

Reviewed commit: b6f361e3a9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@chatgpt-codex-connector

Copy link
Copy Markdown

🛡️ Codex Security Review

Security review completed. No security issues were found in this pull request.

Reviewed commit: b6f361e3a9

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@xormania
xormania merged commit 19ff013 into dev Oct 6, 2026
8 of 10 checks passed
@xormania
xormania deleted the feat/scalable-deployments branch October 6, 2026 01:44
xormania added a commit that referenced this pull request Oct 6, 2026
…#49)

## Why is this PR needed?

Promote the reviewed backend changes on `dev` to `master`. These changes
prepare Chess-Crawl for a separately maintained hosted product while
keeping basic local installations usable.

## What does this PR change?

| Delivery PR | Result |
| --- | --- |
| [#41](#41) | Shared typed
environment/TOML configuration, redacted inspection and offline role
validation |
| [#42](#42) | Workspace
provisioning, hashed credentials, revocation and versioned quota
policies |
| [#43](#43) | Durable
acquisition separated from bounded processing, with recovery and atomic
game-quota admission |
| [#44](#44) | Optional
event delivery, bounded retention and polling-only deployment |
| [#45](#45) | Independently
scalable API/acquisition/processing roles in local Compose and AWS |
| [#46](#46) | Anonymous
operational backlog, waiting and usage metrics |
| [#47](#47) | Bounded
asynchronous working-set/export jobs and workspace-owned local/S3
artifacts |
| [#48](#48) | Live provider
pacing/retry policy and bounded dedicated database sessions |

The public backend supplies reusable contracts, credentials, work
policies and usage. Customer billing and proprietary chess
inference/evaluation/training belong in the separate private product
layer; see [hosted-service
boundaries](https://github.com/xormania/chess-crawl/blob/dev/docs/saas-integration.md).

### Upgrade notes

- Drain older workers and apply migrations `0017`–`0021` before starting
this application version. Restart services after changing deployment
configuration. See [execution and
upgrades](https://github.com/xormania/chess-crawl/blob/dev/docs/execution.md).
- Rerun cloud database role bootstrap to update runtime permissions.
Existing AWS stacks must replace legacy combined-worker/shared-size
parameters and inventory retained queues; see [AWS
deployment](https://github.com/xormania/chess-crawl/blob/dev/docs/aws-deployment.md).
- Database authentication requires a separately provisioned readiness
credential. Asynchronous archive-job admission defaults off; retain
artifact storage access while draining jobs or serving existing
downloads. See [workspace
administration](https://github.com/xormania/chess-crawl/blob/dev/docs/workspace-administration.md)
and [archive
jobs](https://github.com/xormania/chess-crawl/blob/dev/docs/archive-jobs.md).

## Evidence that it solves the reason

| Claim / acceptance condition | Baseline evidence | Result and
verification |
| --- | --- | --- |
| Shared deployment configuration and independently scalable roles |
Prior deployment coupled worker stages and configuration paths |
Configuration/deployment contracts and real Compose API/Mercure smoke
passed in the linked delivery PRs |
| Workspace ownership, quotas and durable recovery survive replica
concurrency | Hosted-service requirements and focused regressions in the
delivery PRs | Real PostgreSQL 18 tests cover credentials, policy
revisions, admission races, fencing, recovery and owned artifacts |
| Complete integrated backend passes supported offline checks | `master`
lacks the eight merged delivery PRs | Final [PR #48
checks](https://github.com/xormania/chess-crawl/pull/48/checks) passed:
1,584 tests on each of Python 3.11 and 3.13, Mypy, Ruff, Bandit,
packaged CLI and Compose smoke |

Every delivery PR received exactly one `@codex review` and one `@codex
security review` after CI was green. Findings were resolved and updated
checks passed before merge. The promotion's own CI checks validate its
merge result separately.

## Changelog

Carries the substantive `Unreleased` entries already accumulated on
`dev`; no duplicate promotion entry or release version is added.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant