From 26d0bd594f7a54d5fb8c7d0f9cc4675af3f3ea8b Mon Sep 17 00:00:00 2001 From: micheleRP Date: Fri, 20 Mar 2026 14:19:35 -0600 Subject: [PATCH 1/5] DOC-1896 Document feature RPCN Improve troubleshooting --- .../pages/connect/connect-quickstart.adoc | 46 +++++++------------ .../get-started/pages/whats-new-cloud.adoc | 4 +- .../partials/cloud-quickstart.adoc | 2 +- .../partials/get-started-serverless.adoc | 2 +- 4 files changed, 20 insertions(+), 34 deletions(-) diff --git a/modules/develop/pages/connect/connect-quickstart.adoc b/modules/develop/pages/connect/connect-quickstart.adoc index c628df500..d0bae0a32 100644 --- a/modules/develop/pages/connect/connect-quickstart.adoc +++ b/modules/develop/pages/connect/connect-quickstart.adoc @@ -1,9 +1,7 @@ = Redpanda Connect Quickstart :description: Learn how to quickly start building data pipelines with Redpanda Connect. -The *Connect* page provides a wizard to create pipelines for streaming data into and out of Redpanda. The wizard populates the YAML configuration automatically, so you can get started quickly. - -TIP: Advanced users can skip directly to the *Edit pipeline* step in the wizard to configure the YAML file themselves. +The *Connect* page lets you build data pipelines using a YAML editor. Add connectors from the left pane and configure them in the editor. Use slash commands (`/`) to insert variables, secrets, topics, and users. This quickstart shows how Redpanda Connect can generate, transform, and handle streaming data end-to-end. It creates the following pipelines: @@ -71,19 +69,13 @@ NOTE: Serverless clusters support up to 10 pipelines. Follow these steps to create the producer pipeline: -. Go to the **Connect** page for your cluster to create a pipeline. - -. **Add an input**: Search for and select `generate` from the list of connectors. Click **Next**. +. Go to the *Connect* page for your cluster and click *Create pipeline*. -. **Add an output**: Search for and select `redpanda` from the list of connectors. Click **Next**. +. Enter this name for the pipeline: `joke-generator-producer`. -. **Add a topic**: Create a new topic called `dad-jokes`. This is where Redpanda will store the generated messages. Click **Next**. +. In the left pane, click to add the `generate` input connector and the `redpanda` output connector. The YAML for this connectorappears in the editor. -. **Add permissions**: Create a new user called `connect`. Leave the rest of the default settings. The user will be created with the necessary permissions. Click **Next**. - -. **Edit pipeline**: -.. Enter this name for the pipeline: `joke-generator-producer`. -.. The **Configuration** section automatically populates the YAML with your selected components. Under *Connectors*, you can add the processors `log` and `catch` to log generated jokes and monitor for errors. However, for simplicity in this quickstart, replace the entire configuration with the following YAML, which includes the processors and the `mapping` for joke generation: +. Replace the generated YAML with the following configuration, which includes the `log` and `catch` processors and the `mapping` for joke generation: + [source,yaml] ---- @@ -151,13 +143,16 @@ output: ---- -.. Click **Create**. +. Click *Create*. + ++ +TIP: In the YAML editor, type `/` to open the slash command menu. Use it to insert variables, secrets, topics, and users. You can select existing items or create new ones. + [NOTE] ==== -* Notice the `$\{REDPANDA_BROKERS}` xref:develop:connect/configuration/contextual-variables.adoc[contextual variable] in the configuration. This automatically references your cluster's bootstrap server address, so you can use it in any pipeline without hardcoding connection details. -* Notice `${secrets.KAFKA_USER_CONNECT}` and `${secrets.KAFKA_PASSWORD_CONNECT}`. These were generated when you created the `connect` user. +* Notice the `$\{REDPANDA_BROKERS}` xref:develop:connect/configuration/contextual-variables.adoc[contextual variable] in the configuration. This automatically references your cluster's bootstrap server address, so you can use it in any pipeline without hardcoding connection details. +* Notice `${secrets.KAFKA_USER_CONNECT}` and `${secrets.KAFKA_PASSWORD_CONNECT}`. These reference secrets that you can create using the slash command menu in the YAML editor or the *Security* page. * The Brave browser does not fully support code snippets. ==== @@ -208,22 +203,13 @@ After a minute, select the pipeline and click **Stop** so you can examine the re This pipeline rates the jokes that you generated in the first pipeline. Follow these steps to create the consumer pipeline: -. On the **Connect** page for your cluster, click **Create pipeline**. - -. **Add an input**: Search for and select `redpanda` from the list of connectors. Click **Next**. - -. **Add an output**: Search for and select `drop` from the list of connectors. (Because this quickstart is just for testing, this output drops the message instead of sending it anywhere else. In a real scenario you'd replace the `drop` connector with your real destination.) Click **Next**. +. On the *Connect* page for your cluster, click *Create pipeline*. -. **Add a topic**: Select the existing topic called `dad-jokes`. Click **Next**. +. Enter this name for the pipeline: `joke-generator-consumer`. -. **Add permissions**: -.. Select the existing user called `connect`. -.. Add a consumer group: Enter `dad-joke-raters` as the name for the consumer group. -.. Click **Next**. +. In the left pane, click to add the `redpanda` input connector and the `drop` output connector. (Because this quickstart is just for testing, this output drops the message instead of sending it anywhere else. In a real scenario you'd replace the `drop` connector with your real destination.) Their YAML appears in the editor. -. **Edit pipeline**: -.. Enter this name for the pipeline: `joke-generator-consumer`. -.. The **Configuration** section automatically populates the YAML with your selected components. To add the `bloblang`, `log`, and `catch` processors, replace the entire configuration with the following YAML. Bloblang is Redpanda Connect's scripting language used to add logic (for example, random ratings). +. Replace the generated YAML with the following configuration, which includes the `bloblang`, `log`, and `catch` processors. Bloblang is Redpanda Connect's scripting language used to add logic (for example, random ratings). + [source,yaml] @@ -317,7 +303,7 @@ output: ---- -.. Click **Create** to start your pipeline. +. Click *Create* to start your pipeline. + NOTE: This example explicitly includes several optional configuration fields for the `redpanda` input. They're shown here for demonstration purposes, so you can see a range of available settings. diff --git a/modules/get-started/pages/whats-new-cloud.adoc b/modules/get-started/pages/whats-new-cloud.adoc index 69c095763..566dc4cfb 100644 --- a/modules/get-started/pages/whats-new-cloud.adoc +++ b/modules/get-started/pages/whats-new-cloud.adoc @@ -110,11 +110,11 @@ The message size setting on existing topics is not changed, but the message size === Redpanda Connect updates -Redpanda Connect provides a simplified xref:develop:connect/connect-quickstart.adoc[quickstart] experience in the UI that helps you to start building data pipelines. The quickstart wizard creates pipelines to stream data into and out of Redpanda, populating the required YAML configuration automatically. +Redpanda Connect provides a simplified xref:develop:connect/connect-quickstart.adoc[quickstart] experience in the UI that helps you to start building data pipelines. The quickstart creates pipelines to stream data into and out of Redpanda using the pipeline editor. === Get Started with Serverless -A Serverless cluster's *Overview* page now provides a *Get Started* wizard to help you start streaming your own data with a xref:develop:connect/connect-quickstart.adoc[Redpanda Connect] pipeline. The wizard lets you stream data into and out of Redpanda without writing producer/consumer code. +A Serverless cluster's *Overview* page now provides a *Get Started* guide to help you start streaming your own data with a xref:develop:connect/connect-quickstart.adoc[Redpanda Connect] pipeline. It lets you stream data into and out of Redpanda without writing producer/consumer code. === Remote read replicas: GA diff --git a/modules/get-started/partials/cloud-quickstart.adoc b/modules/get-started/partials/cloud-quickstart.adoc index 75e6bc380..6d99dbab1 100644 --- a/modules/get-started/partials/cloud-quickstart.adoc +++ b/modules/get-started/partials/cloud-quickstart.adoc @@ -1 +1 @@ -TIP: The cluster's *Overview* page includes a *Get Started* wizard to quickly pipeline your data into and out of Redpanda. The wizard guides you to select an input data source (to produce data) or an output data sink (to consume data) and automatically populates the necessary permissions and configurations. See also: xref:develop:connect/connect-quickstart.adoc[Redpanda Connect Quickstart] +TIP: The cluster's *Overview* page includes a *Get Started* guide to help you start streaming data into and out of Redpanda. See also: xref:develop:connect/connect-quickstart.adoc[Redpanda Connect Quickstart] diff --git a/modules/get-started/partials/get-started-serverless.adoc b/modules/get-started/partials/get-started-serverless.adoc index fea7e8848..803cfe7e5 100644 --- a/modules/get-started/partials/get-started-serverless.adoc +++ b/modules/get-started/partials/get-started-serverless.adoc @@ -9,7 +9,7 @@ When either the credits expire or the days in the trial expire, the clusters mov After you start a trial, Redpanda instantly prepares an account for you. Your account includes a `welcome` cluster with a `hello-world` demo topic you can explore. It includes sample data so you can see how real-time messaging works before sending your own data. -xref:get-started:cluster-types/serverless.adoc#interact-with-your-cluster[Get started] by following the wizard to create a Redpanda Connect glossterm:pipeline[], or by following the steps in the Console to use `rpk` to interact with your cluster from the command line: +xref:get-started:cluster-types/serverless.adoc#interact-with-your-cluster[Get started] by creating a Redpanda Connect glossterm:pipeline[], or by following the steps in the Console to use `rpk` to interact with your cluster from the command line: . Log in with `rpk cloud login`. . Consume from the `hello-world` topic with `rpk topic consume hello-world`. From 234654ce850408b2838d38a3edef41587efbd2f8 Mon Sep 17 00:00:00 2001 From: micheleRP Date: Fri, 20 Mar 2026 14:45:18 -0600 Subject: [PATCH 2/5] minor edits --- .../develop/pages/connect/connect-quickstart.adoc | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/modules/develop/pages/connect/connect-quickstart.adoc b/modules/develop/pages/connect/connect-quickstart.adoc index d0bae0a32..726c14338 100644 --- a/modules/develop/pages/connect/connect-quickstart.adoc +++ b/modules/develop/pages/connect/connect-quickstart.adoc @@ -1,7 +1,9 @@ = Redpanda Connect Quickstart :description: Learn how to quickly start building data pipelines with Redpanda Connect. -The *Connect* page lets you build data pipelines using a YAML editor. Add connectors from the left pane and configure them in the editor. Use slash commands (`/`) to insert variables, secrets, topics, and users. +The *Connect* page lets you build data pipelines using a YAML editor. Add connectors and processors from the left pane and configure them in the editor. You can use slash commands (`/`) in the YAML editor to insert existing variables, secrets, topics, and users, or to create new ones. + +== Quickstart pipelines This quickstart shows how Redpanda Connect can generate, transform, and handle streaming data end-to-end. It creates the following pipelines: @@ -67,13 +69,13 @@ NOTE: Serverless clusters support up to 10 pipelines. == Build a producer pipeline -Follow these steps to create the producer pipeline: +To create the producer pipeline: . Go to the *Connect* page for your cluster and click *Create pipeline*. . Enter this name for the pipeline: `joke-generator-producer`. -. In the left pane, click to add the `generate` input connector and the `redpanda` output connector. The YAML for this connectorappears in the editor. +. In the left pane, click to add the `generate` input connector and the `redpanda` output connector. The YAML for this connector appears in the editor. . Replace the generated YAML with the following configuration, which includes the `log` and `catch` processors and the `mapping` for joke generation: + @@ -145,9 +147,6 @@ output: . Click *Create*. -+ -TIP: In the YAML editor, type `/` to open the slash command menu. Use it to insert variables, secrets, topics, and users. You can select existing items or create new ones. - + [NOTE] ==== @@ -155,6 +154,9 @@ TIP: In the YAML editor, type `/` to open the slash command menu. Use it to inse * Notice `${secrets.KAFKA_USER_CONNECT}` and `${secrets.KAFKA_PASSWORD_CONNECT}`. These reference secrets that you can create using the slash command menu in the YAML editor or the *Security* page. * The Brave browser does not fully support code snippets. ++ +TIP: In the YAML editor, type `/` to open the slash command menu. Use it to insert variables, secrets, topics, and users. You can select existing items or create new ones. + ==== . Your pipeline details display, and after a few seconds, the status changes from **Starting** to **Running**. If you don't see this change, refresh the page. Once running, your pipeline generates a new joke every five seconds and writes the joke to your topic. From b090406b5eecdac27375d79aa11d9784d9e74450 Mon Sep 17 00:00:00 2001 From: micheleRP Date: Fri, 20 Mar 2026 15:29:20 -0600 Subject: [PATCH 3/5] minor edits --- .../pages/connect/connect-quickstart.adoc | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/modules/develop/pages/connect/connect-quickstart.adoc b/modules/develop/pages/connect/connect-quickstart.adoc index 726c14338..ec82cc0de 100644 --- a/modules/develop/pages/connect/connect-quickstart.adoc +++ b/modules/develop/pages/connect/connect-quickstart.adoc @@ -3,6 +3,12 @@ The *Connect* page lets you build data pipelines using a YAML editor. Add connectors and processors from the left pane and configure them in the editor. You can use slash commands (`/`) in the YAML editor to insert existing variables, secrets, topics, and users, or to create new ones. +== Prerequisites + +You must have a Redpanda Cloud account with a Serverless, Dedicated, or standard BYOC cluster. If you don't already have an account, https://redpanda.com/try-redpanda/cloud-trial[sign up for a free trial^]. + +NOTE: Serverless clusters support up to 10 pipelines. + == Quickstart pipelines This quickstart shows how Redpanda Connect can generate, transform, and handle streaming data end-to-end. It creates the following pipelines: @@ -61,12 +67,6 @@ The *consumer pipeline* uses the following Redpanda Connect components: |Catches errors |=== -== Prerequisites - -You must have a Redpanda Cloud account with a Serverless, Dedicated, or standard BYOC cluster. If you don't already have an account, https://redpanda.com/try-redpanda/cloud-trial[sign up for a free trial^]. - -NOTE: Serverless clusters support up to 10 pipelines. - == Build a producer pipeline To create the producer pipeline: @@ -75,7 +75,7 @@ To create the producer pipeline: . Enter this name for the pipeline: `joke-generator-producer`. -. In the left pane, click to add the `generate` input connector and the `redpanda` output connector. The YAML for this connector appears in the editor. +. In the left pane, click to add the `generate` input connector and the `redpanda` output connector. The YAML for these connectors appears in the editor. . Replace the generated YAML with the following configuration, which includes the `log` and `catch` processors and the `mapping` for joke generation: + @@ -203,7 +203,7 @@ After a minute, select the pipeline and click **Stop** so you can examine the re == Build a consumer pipeline -This pipeline rates the jokes that you generated in the first pipeline. Follow these steps to create the consumer pipeline: +This pipeline rates the jokes that you generated in the first pipeline. To create the consumer pipeline: . On the *Connect* page for your cluster, click *Create pipeline*. @@ -340,7 +340,7 @@ When you've finished experimenting with your data pipeline, you can delete the p == Next steps -* Try one of our xref:cookbooks:index.adoc[Redpanda Connect cookbooks]. +* Try one of the xref:cookbooks:index.adoc[Redpanda Connect cookbooks]. * Choose xref:develop:connect/components/about.adoc[connectors for your use case]. * Learn how to xref:develop:connect/configuration/secret-management.adoc[add secrets to your pipeline]. * Learn how to xref:develop:connect/configuration/monitor-connect.adoc[monitor a data pipeline on a BYOC or Dedicated cluster]. From 9a55db5018cba6940df66251aaa29481b44f75df Mon Sep 17 00:00:00 2001 From: micheleRP Date: Mon, 23 Mar 2026 11:54:27 -0600 Subject: [PATCH 4/5] minor edits --- .../pages/connect/connect-quickstart.adoc | 19 +++++++++++++------ 1 file changed, 13 insertions(+), 6 deletions(-) diff --git a/modules/develop/pages/connect/connect-quickstart.adoc b/modules/develop/pages/connect/connect-quickstart.adoc index ec82cc0de..d95fb34a9 100644 --- a/modules/develop/pages/connect/connect-quickstart.adoc +++ b/modules/develop/pages/connect/connect-quickstart.adoc @@ -1,7 +1,14 @@ = Redpanda Connect Quickstart :description: Learn how to quickly start building data pipelines with Redpanda Connect. -The *Connect* page lets you build data pipelines using a YAML editor. Add connectors and processors from the left pane and configure them in the editor. You can use slash commands (`/`) in the YAML editor to insert existing variables, secrets, topics, and users, or to create new ones. +The *Connect* page lets you build data pipelines using a YAML editor. Add connectors and processors from the left pane and configure them in the editor. + +[TIP] +==== +* Click any connector to learn more about it in the documentation. +* Use slash commands (`/`) in the YAML editor to insert existing variables, secrets, topics, and users, or to create new ones. + +==== == Prerequisites @@ -18,7 +25,7 @@ This quickstart shows how Redpanda Connect can generate, transform, and handle s The *producer pipeline* uses the following Redpanda Connect components: -[cols="1,2,3"] +[cols="1,2,3"]a |=== |Component type |Component |Purpose @@ -71,7 +78,7 @@ The *consumer pipeline* uses the following Redpanda Connect components: To create the producer pipeline: -. Go to the *Connect* page for your cluster and click *Create pipeline*. +. Go to the *Connect* page for your cluster and click *Create a pipeline*. . Enter this name for the pipeline: `joke-generator-producer`. @@ -154,11 +161,11 @@ output: * Notice `${secrets.KAFKA_USER_CONNECT}` and `${secrets.KAFKA_PASSWORD_CONNECT}`. These reference secrets that you can create using the slash command menu in the YAML editor or the *Security* page. * The Brave browser does not fully support code snippets. +==== + + TIP: In the YAML editor, type `/` to open the slash command menu. Use it to insert variables, secrets, topics, and users. You can select existing items or create new ones. -==== - . Your pipeline details display, and after a few seconds, the status changes from **Starting** to **Running**. If you don't see this change, refresh the page. Once running, your pipeline generates a new joke every five seconds and writes the joke to your topic. + @@ -205,7 +212,7 @@ After a minute, select the pipeline and click **Stop** so you can examine the re This pipeline rates the jokes that you generated in the first pipeline. To create the consumer pipeline: -. On the *Connect* page for your cluster, click *Create pipeline*. +. On the *Connect* page for your cluster, click *Create a pipeline*. . Enter this name for the pipeline: `joke-generator-consumer`. From 5cfc7c85aa70cfe4a224cd5bc6750f77b45e26ee Mon Sep 17 00:00:00 2001 From: micheleRP Date: Mon, 23 Mar 2026 12:22:16 -0600 Subject: [PATCH 5/5] minor edit --- modules/develop/pages/connect/connect-quickstart.adoc | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/modules/develop/pages/connect/connect-quickstart.adoc b/modules/develop/pages/connect/connect-quickstart.adoc index d95fb34a9..4d5fc539f 100644 --- a/modules/develop/pages/connect/connect-quickstart.adoc +++ b/modules/develop/pages/connect/connect-quickstart.adoc @@ -164,7 +164,7 @@ output: ==== + -TIP: In the YAML editor, type `/` to open the slash command menu. Use it to insert variables, secrets, topics, and users. You can select existing items or create new ones. +TIP: In the YAML editor, you can type `/` to open the slash command menu. Use it to insert variables, secrets, topics, and users. You can select existing items or create new ones. . Your pipeline details display, and after a few seconds, the status changes from **Starting** to **Running**. If you don't see this change, refresh the page. Once running, your pipeline generates a new joke every five seconds and writes the joke to your topic. @@ -173,7 +173,7 @@ After a minute, select the pipeline and click **Stop** so you can examine the re === Review the pipeline logs -. Click the **Logs** tab to see the pipeline's activity log. +. On your pipeline page, switch to the **Live** mode to see the pipeline's activity log. The live mode loads new log messages as they come in. When Live mode is disabled, the log shows activity for from the past five hours. . Click through the log messages to see the startup sequence. For example, you'll see when the output becomes active: +