docs: add chain-of-thought streaming guides for OpenAI and Anthropic by GregHolmes · Pull Request #3135 · ably/docs

GregHolmes · 2026-01-21T13:57:01Z

docs: add chain-of-thought streaming guides for OpenAI and Anthropic
Add two new AI Transport guides demonstrating how to stream
chain-of-thought reasoning from thinking models over Ably:

openai-chain-of-thought.mdx: Stream reasoning from OpenAI o1/o3 models
anthropic-chain-of-thought.mdx: Stream extended thinking from Claude

Both guides cover:

Inline pattern (reasoning and output on same channel)
Threading pattern (reasoning on separate on-demand channels)
Event handling for reasoning vs output tokens
Message-per-response pattern with serial-based tracking

coderabbitai · 2026-01-21T13:57:13Z

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests
Post copyable unit tests in a comment
Commit unit tests in branch AIT-137-Guide-documentation-Threading-chain-of-thought

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

zknill

I'm not sure that Step 3 and Step 5 make sense, they aren't "steps" in how to do it, they are entirely different patterns.

I think we need an intro that explains the two patterns and the differences, and then I think we need to remove the "Step" headers, or make it clear that they are not really "Steps" like how the other guides are

zknill · 2026-02-17T16:14:06Z

src/pages/docs/guides/ai-transport/openai/openai-chain-of-thought.mdx

+```json
+// 1. Reasoning summary tokens stream in
+{"type":"response.reasoning_summary_text.delta","delta":"**Determining the capital of France**\n\n"}
+{"type":"response.reasoning_summary_text.delta","delta":"The user's question is straightforward..."}
+
+// 2. Output tokens stream in
+{"type":"response.output_text.delta","delta":"The capital of France is Paris."}
+{"type":"response.output_text.delta","delta":" Here's the reasoning:"}
+
+// 3. Response completes
+{"type":"response.completed","response":{...}}
+```
+</Code>


I don't think these tokens make sense in this order?

You're right! The output content didn't really make sense when we've already got the reasoning being shown separately. I've changed the second output delta to the actual information. It has been the capital since the 10th century. instead if Here's the reasoning:

zknill · 2026-02-17T16:16:45Z

src/pages/docs/guides/ai-transport/openai-chain-of-thought.mdx

+```
+</Code>
+
+The `publish` call creates an initial empty message and returns its `serial`. Subsequent `appendMessage` calls add tokens to that message. Don't await `appendMessage` calls - Ably batches acknowledgments for efficiency.


Don't await appendMessage calls - Ably guarantees that deltas appended to the same message serial will always remain in-order.

I don't think anyone will know / understand what ACKs are here, and batching them isn't really the reason to await an append. It's a throughput thing really.

Good point, I've removed ACK, talked about the throughput and why we're not awaiting.

zknill · 2026-02-17T16:23:03Z

src/pages/docs/guides/ai-transport/openai-chain-of-thought.mdx

+
+## Step 5: Stream reasoning with the threading pattern <a id="step-5"/>
+
+The threading pattern publishes reasoning to a separate channel. This allows clients to subscribe to reasoning alongside output, while keeping them on separate channels for flexibility.


What is the "flexibility" that this provides?

Let's give a real reason here, because I don't think "flexibility" is a reason (without qualifying why it's more flexible)

In fact, I'd argue it's harder to work with because the customer won't get ordering between reasoning and regular message. Which makes me think; do we even want to promote 2 channels as a pattern? I can't see why you'd want that.

True, I've replaced the flexibility it with more concrete reasons.

GregHolmes · 2026-02-19T17:00:55Z

I'm not sure that Step 3 and Step 5 make sense, they aren't "steps" in how to do it, they are entirely different patterns.

I think we need an intro that explains the two patterns and the differences, and then I think we need to remove the "Step" headers, or make it clear that they are not really "Steps" like how the other guides are

Thanks Zak for the review. I've moved things around and made changes following from your feedback. I've kept it in a fixup so you can see the difference. When you have some time could I please ask you to re-review this guide?

zknill · 2026-02-23T18:03:15Z

I'm getting 404 trying to run this locally and navigate to the anthropic and openai chain of though guides. I assume something needs updating since we merged the new grouped-by-provider nav?

Add two new AI Transport guides demonstrating how to stream chain-of-thought reasoning from thinking models over Ably: - openai-chain-of-thought.mdx: Stream reasoning from OpenAI o1/o3 models - anthropic-chain-of-thought.mdx: Stream extended thinking from Claude Both guides cover: - Inline pattern (reasoning and output on same channel) - Threading pattern (reasoning on separate on-demand channels) - Event handling for reasoning vs output tokens - Message-per-response pattern with serial-based tracking

GregHolmes · 2026-02-26T13:31:20Z

I'm getting 404 trying to run this locally and navigate to the anthropic and openai chain of though guides. I assume something needs updating since we merged the new grouped-by-provider nav?

Sorry @zknill yep, it was broken. I've fixed and made a few further changes through the docs. I hope you enjoy.

GregHolmes self-assigned this Jan 21, 2026

GregHolmes force-pushed the AIT-137-Guide-documentation-Threading-chain-of-thought branch 2 times, most recently from 044afc9 to 0940eea Compare January 21, 2026 13:59

GregHolmes added the review-app Create a Heroku review app label Jan 21, 2026

GregHolmes requested review from mschristensen and rainbowFi January 21, 2026 13:59

ably-ci temporarily deployed to ably-docs-ait-137-guide-i4shiu January 21, 2026 13:59 Inactive

mschristensen added review-app Create a Heroku review app and removed review-app Create a Heroku review app labels Jan 22, 2026

ably-ci temporarily deployed to ably-docs-ait-137-guide-ci09ak January 22, 2026 16:27 Inactive

m-hulbert temporarily deployed to ably-docs-ait-137-guide-qnldl6 February 6, 2026 13:14 Inactive

zknill self-requested a review February 12, 2026 16:27

GregHolmes requested a review from lawrence-forooghian February 17, 2026 15:14

zknill reviewed Feb 17, 2026

View reviewed changes

GregHolmes force-pushed the AIT-137-Guide-documentation-Threading-chain-of-thought branch from 0940eea to c080243 Compare February 19, 2026 13:34

GregHolmes removed request for lawrence-forooghian, mschristensen and rainbowFi February 19, 2026 13:44

GregHolmes force-pushed the AIT-137-Guide-documentation-Threading-chain-of-thought branch from ae4969e to c9dc512 Compare February 19, 2026 16:40

GregHolmes requested a review from zknill February 19, 2026 17:00

GregHolmes force-pushed the AIT-137-Guide-documentation-Threading-chain-of-thought branch from c9dc512 to 20f5f00 Compare February 23, 2026 09:19

GregHolmes force-pushed the AIT-137-Guide-documentation-Threading-chain-of-thought branch from 20f5f00 to 84df719 Compare February 26, 2026 11:09

GregHolmes force-pushed the AIT-137-Guide-documentation-Threading-chain-of-thought branch from 207b166 to a7651d9 Compare February 26, 2026 12:13

docs: move chain-of-thought guides into provider subdirectories

77a07eb

GregHolmes force-pushed the AIT-137-Guide-documentation-Threading-chain-of-thought branch from a7651d9 to 77a07eb Compare February 26, 2026 12:14

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

docs: add chain-of-thought streaming guides for OpenAI and Anthropic#3135

docs: add chain-of-thought streaming guides for OpenAI and Anthropic#3135
GregHolmes wants to merge 2 commits intomainfrom
AIT-137-Guide-documentation-Threading-chain-of-thought

GregHolmes commented Jan 21, 2026 •

edited

Loading

Uh oh!

coderabbitai bot commented Jan 21, 2026 •

edited

Loading

Review skipped

Uh oh!

zknill left a comment

Uh oh!

zknill Feb 17, 2026

Uh oh!

GregHolmes Feb 19, 2026

Uh oh!

zknill Feb 17, 2026

Uh oh!

GregHolmes Feb 19, 2026

Uh oh!

zknill Feb 17, 2026

Uh oh!

GregHolmes Feb 19, 2026

Uh oh!

GregHolmes commented Feb 19, 2026

Uh oh!

zknill commented Feb 23, 2026

Uh oh!

GregHolmes commented Feb 26, 2026

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

5 participants


		## Step 5: Stream reasoning with the threading pattern <a id="step-5"/>

		The threading pattern publishes reasoning to a separate channel. This allows clients to subscribe to reasoning alongside output, while keeping them on separate channels for flexibility.

Conversation

GregHolmes commented Jan 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

coderabbitai bot commented Jan 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review skipped

Uh oh!

zknill left a comment

Choose a reason for hiding this comment

Uh oh!

zknill Feb 17, 2026

Choose a reason for hiding this comment

Uh oh!

GregHolmes Feb 19, 2026

Choose a reason for hiding this comment

Uh oh!

zknill Feb 17, 2026

Choose a reason for hiding this comment

Uh oh!

GregHolmes Feb 19, 2026

Choose a reason for hiding this comment

Uh oh!

zknill Feb 17, 2026

Choose a reason for hiding this comment

Uh oh!

GregHolmes Feb 19, 2026

Choose a reason for hiding this comment

Uh oh!

GregHolmes commented Feb 19, 2026

Uh oh!

zknill commented Feb 23, 2026

Uh oh!

GregHolmes commented Feb 26, 2026

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

5 participants

GregHolmes commented Jan 21, 2026 •

edited

Loading

coderabbitai bot commented Jan 21, 2026 •

edited

Loading