Support stream scale-to-zero drain

[ghkdqhrbals]

Summary

• allow stream scale targets to reach zero and keep producer routing empty while drained
• block scale-in completion until every Redis consumer group on removed shards reports pending=0 and known lag=0
• remove obsolete local Redis compose variants and keep pod/stress compose paths documented
• update PRD, design docs, OpenAPI, and tests for scale-to-zero semantics

Tests

• ./gradlew :coordinator-server:test --tests '*CoordinatorServiceTest' --tests '*CoordinatorHttpIntegrationTest' --no-daemon
• python3 .github/scripts/test_docker_distribution.py
• ./gradlew :coordinator-server:test --no-daemon
• ./gradlew test --no-daemon

[github-actions]

✅ Coordinator tests passed

Check	Details
Status	✅ Passed
Run attempt	1
Command	./gradlew :coordinator-server:test :coordinator-server:build --rerun-tasks --no-daemon
Result	177 passed, 0 failed, 0 errored, 7 skipped
Reports	Workflow run / Coordinator test report
Raw Gradle test report	Open Gradle report

Remaining work before public release

• Cut the first public Docker image release through the manual GHCR workflow.
• Add a compatibility fixture suite for old metadata JSON and old client coordination versions.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6b831bcfe8

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Treat cold Redis health as degraded

When Redis health is required and the cache is cold or expired, this now kicks off the ping asynchronously and immediately returns UNKNOWN; health() still maps every value except DOWN to overall UP, so the first readiness/deployment check after startup (or after cache expiry) can report healthy even while Redis is unreachable until the background refresh completes. For the /coord/v1/monitoring/health endpoint used by deployment checks, either wait for the ping on a cold cache or treat UNKNOWN as degraded.

Useful? React with 👍 / 👎.

[chatgpt-codex-connector]

Allow restarted consumers to tolerate zero-shard routing

Emitting shardCount=0 with an empty shard list makes a drained stream's routing view incompatible with the starter: CoordinatorManagedConsumer.validateInitialRouting() calls CoordinatorRoutingMetadataValidator, which still requires metadata.shardCount > 0 and non-empty shards before the consumer is allowed to start. After a scale-to-zero, any autostarting consumer pod for that group that restarts will fail during bean initialization instead of joining and idling with an empty assignment; the consumer-side validation needs to permit zero-shard routing (or skip this producer-routing validation for consumers).

Useful? React with 👍 / 👎.

[github-actions]

Design docs render

• Source: design-docs
• Result: Rendered HTML was uploaded as the coordinator-design-docs-html artifact.
• Publish: Manual only via the Design docs workflow_dispatch run.
• Links: workflow run

[github-actions]

✅ Coordinator tests passed

Check	Details
Status	✅ Passed
Run attempt	1
Command	./gradlew :coordinator-server:test :coordinator-server:build --rerun-tasks --no-daemon
Result	179 passed, 0 failed, 0 errored, 7 skipped
Reports	Workflow run / Coordinator test report
Raw Gradle test report	Open Gradle report

Remaining work before public release

• Cut the first public Docker image release through the manual GHCR workflow.
• Add a compatibility fixture suite for old metadata JSON and old client coordination versions.