docs: surface Redpanda Connect pipelines for operator users (DOC-2275)#1808
Open
JakeSCahill wants to merge 10 commits into
Open
docs: surface Redpanda Connect pipelines for operator users (DOC-2275)#1808JakeSCahill wants to merge 10 commits into
JakeSCahill wants to merge 10 commits into
Conversation
Three discovery surfaces for the 26.2 Pipeline CRD (DOC-2275): - new Manage > Kubernetes page routing to the full guide in the Connect docs, positioned next to Manage Kafka Connect - Pipeline row in the CRD reference index - TIP on Manage Kafka Connect pointing at the first-class alternative
✅ Deploy Preview for redpanda-docs-preview ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
Contributor
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Contributor
Author
|
Update: the Redpanda Connect operator support (Pipeline CRD, redpanda-operator#1677) is expected to slip past 26.2, most likely to 26.3. This PR stays in draft until the CRD ships in a released operator. Do not merge for the 26.2 GA on July 28. |
Contributor
|
PR is merged redpanda-data/redpanda-operator#1677, this will go out as Beta for the 26.2 date. Could you ensure that examples are provided with examples of how to use HPA and KEDA for autoscaling? |
…censing, and what's new (DOC-2275) Turn the discovery page into a full operator-side guide covering every flow from redpanda-operator#1677: enable the Connect controller, first pipeline, clusterRef+userRef cluster binding (with the rendered-config injection contract), staticConfiguration for external clusters, per-pipeline cloud IAM via serviceAccountName (IRSA worked example), and valueSources with all four source types. Adds status/condition reference and troubleshooting. Add a dedicated autoscaling page with step-by-step HPA (cpu/memory) and KEDA (consumer-group lag, Connect metrics via Prometheus, scale-to-zero) examples, all grounded in manifests validated end-to-end on EKS. Document the no-license/expired-license behavior for pipelines in the licensing overview, and add the operator what's-new entry (version section marked TODO until the ship vehicle is confirmed). Route the CRD index row to the local guide instead of the unpublished connect-docs page, which removes the two expected broken-xref warnings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
david-yu
marked this pull request as ready for review
July 24, 2026 18:09
The feature ships with operator v26.2.x, not v26.3.x. Drop the provisional v26.3.x section and its TODO. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ing overview Give the operator its own enterprise-features table, matching the Redpanda / Console / Connect sections: Stretch Clusters (multicluster operator refuses to start without a valid license; LicenseValid condition + status.licenseStatus on running clusters) and Connect pipelines (moved out of the Connect table). Add a 'Manage licenses for the Redpanda Operator' section covering enterprise.licenseSecretRef and per-cluster licensing via the Redpanda and StretchCluster resources. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ect product entitlement Replace the bespoke license sentence on the pipelines page with the shared enterprise-license partial, and state everywhere the license is discussed (notice, prerequisites, license behavior, troubleshooting, licensing overview table) that the license must include the Redpanda Connect product — matching the operator's IncludesProduct(ProductConnect) check. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
david-yu
approved these changes
Jul 24, 2026
david-yu
force-pushed
the
doc-2275-connect-pipelines-discovery
branch
from
July 24, 2026 20:37
44b1fe4 to
ddb3fc5
Compare
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The Pipeline resource ships as a beta feature in the 26.2 GA operator (redpanda-operator#1677 merged to main on July 22), so remove the draft-hold markers and keep the release-note section under v26.2.x. Also apply standards review fixes: - Rename the page to 'Run Redpanda Connect Pipelines in Kubernetes' (verb phrase for a how-to) - Use the standard beta notice on both pipeline pages, including the autoscaling page, which set page-beta without an in-body disclosure - Mark the feature (beta) at its other discovery surfaces: the CRD index row, the licensing overview row, and the Kafka Connect page - Replace list-bullet double hyphens with colons in the release notes - Split semicolon-joined sentences, fix a passive section opener, avoid starting a sentence with inline code, spell out IAM on first use, and add lead-in sentences before the status command and the troubleshooting table - Cross-link the Redpanda Connect operator install guide from Next steps (resolves when rp-connect-docs#457 merges)
Register the shared badge inline macro in the playbook (already used by cloud-docs; the docs playbook never registered it, which is why no page here could use it) and apply badge::[label=beta] at the places where beta status applies to a feature mention inside a GA page: the CRD index row, the licensing overview row, and the Kafka Connect page cross-link. Whole-page beta status stays on the page-beta attribute.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The Pipeline CRD ships as a beta feature in the 26.2 GA operator (redpanda-operator#1677 merged to main July 22). Merge together with redpanda-data/rp-connect-docs#457: the Next steps cross-link to
connect:install:kubernetes-operator.adocresolves once #457 publishes.Documents the Pipeline resource (redpanda-operator#1677) for operator users. Originally a discovery-only PR; now expanded into the full operator-side guide, per review feedback, covering every flow in the eng PR description with worked, e2e-validated examples.
Pages
manage:kubernetes/k-connect-pipelines.adoc(nav: "Redpanda Connect Pipelines") — full guide:connectController.enabled+enterprise.licenseSecretRef)configYaml), lint-gated startup, config/credential checksum rollouts, pausecluster.clusterRef+userRefwith a scopedUserCR, the rendered-config injection contract (seed_brokers/tls/sasl+ top-level shared client), user-keys-win escape hatch, ACL-scoping guidancecluster.staticConfigurationwith Secret/ConfigMap-backed TLS CA, inline SASL, CEL exclusivity withuserRefserviceAccountName+ IRSA worked example (RDS IAM auth, no DB password anywhere)inline/secretKeyRef/configMapKeyRef/externalSecretRef) in one fan-out exampleextraInitContainers), monitoring (PodMonitor), license behavior, status/conditions reference, troubleshooting. Uses the standardshared:partial$enterprise-license.adocnotice and states that the license must include the Redpanda Connect product (matches the operator'sIncludesProduct(ProductConnect)check).manage:kubernetes/k-autoscale-connect-pipelines.adoc(new, nav: nested "Autoscaling") — step-by-step autoscaling through the scale subresource, grounded in manifests validated e2e on EKS (HPA cpu 1→4; KEDA kafka-lag 1→6; KEDA prometheusinput_received1→3):kubectl scale+ raw/scalesmoke testkubectl get hpaoutput)connectController.monitoring.enabled+ PodMonitor), scale-to-zeropausedprecedence, explicitreplicas: 0, GitOps ownership ofreplicas, one autoscaler per pipelineget-started:licensing/overview.adoc): new "Redpanda Operator enterprise features" section with its own table (matching the Redpanda/Console/Connect sections) covering both operator-gated features — Stretch Clusters (multicluster operator requires a valid license to start;LicenseValidcondition +status.licenseStatusreport expiry/violations on running clusters) and Connect pipelines (missing/expired license or missing Connect product entitlement → new pipelines reportLicense: Falseand get no workload; running pipelines keep processing but stop receiving updates; recovery is automatic once a valid license is applied). Also adds a "Manage licenses for the Redpanda Operator" section (enterprise.licenseSecretRefchart values + per-cluster licensing via the Redpanda/StretchClusterenterprisefield).get-started:release-notes/operator.adoc): "Redpanda Connect pipelines (beta)" entry under thev26.2.xsection.Build note
The two previously-expected broken-xref warnings for
connect:install:kubernetes-operator.adocare gone: all routing now targets the local pages.origin/betahas been merged into the branch.Before un-drafting (GA batch)
Fill in the exact minimum operator version— done: the page states Redpanda Operator v26.2.1 or laterconnect:install:kubernetes-operator.adoc(comment markers in place)Part of DOC-2275; eng feature is redpanda-operator#1677.
Preview pages
🤖 Generated with Claude Code