diff --git a/examples/ai-transport-message-per-response/javascript/README.md b/examples/ai-transport-message-per-response/javascript/README.md deleted file mode 100644 index ebb5760df6..0000000000 --- a/examples/ai-transport-message-per-response/javascript/README.md +++ /dev/null @@ -1,61 +0,0 @@ -# AI Transport message per response streaming - -Enable realtime streaming of AI/LLM responses by appending tokens to a single message over Ably. - -AI Transport message-per-response streaming allows applications to provide immediate, responsive AI interactions by streaming tokens in realtime. Unlike the message-per-token pattern, all tokens for a response are appended to a single message, which appears as one entry in channel history. This makes it easy to retrieve and display conversation history while still delivering live tokens in realtime. - -The streaming approach significantly improves perceived performance and user engagement. Instead of waiting 5-10 seconds for a complete AI response, users see tokens appearing progressively, creating a more natural conversation flow similar to watching someone type in realtime. - -Token streaming is implemented using [Ably AI Transport](/docs/ai-transport). AI Transport provides purpose-built APIs for realtime AI applications, offering reliable message delivery, automatic ordering, and seamless reconnection handling to ensure no tokens are lost during network interruptions. - -## Resources - -Use the following methods to implement AI Transport message-per-response streaming: - -- [`client.channels.get()`](/docs/channels#create): creates a new or retrieves an existing channel for AI Transport token streaming. -- [`channel.publish()`](/docs/channels#publish): publishes the initial message and captures the serial for token appending. -- [`channel.appendMessage()`](/docs/messages#append): appends individual tokens to the message as they arrive from the LLM service. -- [`channel.subscribe()`](/docs/channels#subscribe): subscribes to messages, handling `message.create`, `message.append`, and `message.update` actions. -- [`channel.setOptions()`](/docs/channels/options) with [`rewind`](/docs/channels/options/rewind): enables seamless message recovery during reconnections, delivering historical messages as `message.update` events. - -Find out more about [AI Transport](/docs/ai-transport) and [message appending](/docs/ai-transport/token-streaming/message-per-response). - -## Getting started - -1. Clone the [Ably docs](https://github.com/ably/docs) repository where this example can be found: - - ```sh - git clone git@github.com:ably/docs.git - ``` - -2. Change directory: - - ```sh - cd examples/ - ``` - -3. Rename the environment file: - - ```sh - mv .env.example .env.local - ``` - -4. In `.env.local` update the value of `VITE_ABLY_KEY` to be your Ably API key. - -5. Install dependencies: - - ```sh - yarn install - ``` - -6. Run the server: - - ```sh - yarn run ai-transport-message-per-response-javascript - ``` - -7. Try it out by opening [http://localhost:5173/](http://localhost:5173/) with your browser and selecting a prompt to see realtime AI token streaming. - -## Open in CodeSandbox - -In CodeSandbox, rename the `.env.example` file to `.env.local` and update the value of your `VITE_ABLY_KEY` variable to use your Ably API key. diff --git a/examples/ai-transport-message-per-response/javascript/package.json b/examples/ai-transport-message-per-response/javascript/package.json deleted file mode 100644 index 710f1c4a12..0000000000 --- a/examples/ai-transport-message-per-response/javascript/package.json +++ /dev/null @@ -1,10 +0,0 @@ -{ - "name": "ai-transport-message-per-response-javascript", - "version": "1.0.0", - "type": "module", - "scripts": { - "dev": "vite", - "build": "vite build", - "preview": "vite preview" - } -} diff --git a/examples/ai-transport-message-per-response/react/README.md b/examples/ai-transport-message-per-response/react/README.md deleted file mode 100644 index b0b5e85be2..0000000000 --- a/examples/ai-transport-message-per-response/react/README.md +++ /dev/null @@ -1,70 +0,0 @@ -# AI Transport message per response streaming - -Enable realtime streaming of AI/LLM responses using the message-per-response pattern, where all tokens are appended to a single Ably message. - -AI Transport message-per-response streaming allows applications to provide immediate, responsive AI interactions by streaming tokens in realtime while maintaining a clean message history. Each complete AI response appears as a single message in channel history, making it easy to retrieve and display multi-response conversation history. - -The streaming approach significantly improves perceived performance and user engagement. Instead of waiting 5-10 seconds for a complete AI response, users see tokens appearing progressively, creating a more natural conversation flow similar to watching someone type in realtime. - -Token streaming is implemented using [Ably AI Transport](/docs/ai-transport). AI Transport provides purpose-built APIs for realtime AI applications, offering reliable message delivery, automatic ordering, and seamless reconnection handling to ensure no tokens are lost during network interruptions. - -## Resources - -Use the following components to implement AI Transport message-per-response streaming: - -- [`AblyProvider`](/docs/getting-started/react-hooks#ably-provider): initializes and manages a shared Ably client instance, passing it down through React context to enable realtime AI Transport functionality across the application. -- [`ChannelProvider`](/docs/getting-started/react-hooks#channel-provider): manages the state and functionality of a specific channel, providing access to AI response tokens and streaming state via React context. -- [`useChannel()`](/docs/getting-started/react-hooks#useChannel) hook: a hook to subscribe to messages with `message.create`, `message.append`, and `message.update` actions. -- [`rewind`](/docs/channels/options/rewind) channel option: enables seamless message recovery during reconnections, delivering historical messages as `message.update` events. -- [`appendMessage()`](/docs/api/realtime-sdk/channels#append-message): appends tokens to an existing message using its serial. - -Find out more about [AI Transport](/docs/ai-transport) and [message-per-response](/docs/ai-transport/token-streaming/message-per-response). - -## Getting started - -1. Clone the [Ably docs](https://github.com/ably/docs) repository where this example can be found: - - ```sh - git clone git@github.com:ably/docs.git - ``` - -2. Change directory: - - ```sh - cd examples/ - ``` - -3. Rename the environment file: - - ```sh - mv .env.example .env.local - ``` - -4. In `.env.local` update the value of `VITE_ABLY_KEY` to be your Ably API key. - -5. Install dependencies: - - ```sh - yarn install - ``` - -6. Run the server: - - ```sh - yarn run ai-transport-message-per-response-react - ``` - -7. Try it out by opening [http://localhost:5173/](http://localhost:5173/) with your browser and selecting a prompt to see realtime AI token streaming. - -## Open in CodeSandbox - -In CodeSandbox, rename the `.env.example` file to `.env.local` and update the value of your `VITE_ABLY_KEY` variable to use your Ably API key. - -## How it works - -The message-per-response pattern works by: - -1. **Initial message**: When an agent response begins, publish an initial message with `message.create` action to the Ably channel with an empty or the first token as content. -2. **Token streaming**: Append subsequent tokens to the original message by publishing those tokens with the `message.append` action. -3. **Live delivery**: Clients subscribed to the channel receive each appended token in realtime, allowing them to progressively render the response. -4. **Compacted history**: The channel history contains only one message per agent response, which includes all tokens appended to it concatenated together. diff --git a/examples/ai-transport-message-per-token/javascript/README.md b/examples/ai-transport-message-per-token/javascript/README.md deleted file mode 100644 index ffed30477a..0000000000 --- a/examples/ai-transport-message-per-token/javascript/README.md +++ /dev/null @@ -1,60 +0,0 @@ -# AI Transport message per token streaming - -Enable realtime streaming of AI/LLM responses by publishing tokens as they arrive from Large Language Model services. - -AI Transport token streaming allows applications to provide immediate, responsive AI interactions by streaming tokens in realtime rather than waiting for complete responses. This pattern is essential for creating engaging AI-powered experiences where users can see responses being generated as they happen. - -The streaming approach significantly improves perceived performance and user engagement. Instead of waiting 5-10 seconds for a complete AI response, users see tokens appearing progressively, creating a more natural conversation flow similar to watching someone type in realtime. - -Token streaming is implemented using [Ably AI Transport](/docs/ai-transport). AI Transport provides purpose-built APIs for realtime AI applications, offering reliable message delivery, automatic ordering, and seamless reconnection handling to ensure no tokens are lost during network interruptions. - -## Resources - -Use the following methods to implement AI Transport token streaming: - -- [`client.channels.get()`](/docs/channels#create): creates a new or retrieves an existing channel for AI Transport token streaming. -- [`channel.subscribe()`](/docs/channels#subscribe): subscribes to token messages from AI services by registering a listener for realtime streaming. -- [`channel.publish()`](/docs/channels#publish): publishes individual tokens as they arrive from the LLM service with response tracking headers. -- [`channel.history()`](/docs/channels/history) with [`untilAttach`](/docs/channels/options#attach): enables seamless message recovery during reconnections, ensuring no tokens are lost. - -Find out more about [AI Transport](/docs/ai-transport), [token streaming](/docs/ai-transport/token-streaming), and [message history](/docs/storage-history/history). - -## Getting started - -1. Clone the [Ably docs](https://github.com/ably/docs) repository where this example can be found: - - ```sh - git clone git@github.com:ably/docs.git - ``` - -2. Change directory: - - ```sh - cd examples/ - ``` - -3. Rename the environment file: - - ```sh - mv .env.example .env.local - ``` - -4. In `.env.local` update the value of `VITE_ABLY_KEY` to be your Ably API key. - -5. Install dependencies: - - ```sh - yarn install - ``` - -6. Run the server: - - ```sh - yarn run ai-transport-message-per-token-javascript - ``` - -7. Try it out by opening [http://localhost:5173/](http://localhost:5173/) with your browser and selecting a prompt to see realtime AI token streaming. - -## Open in CodeSandbox - -In CodeSandbox, rename the `.env.example` file to `.env.local` and update the value of your `VITE_ABLY_KEY` variable to use your Ably API key. diff --git a/examples/ai-transport-message-per-token/javascript/index.html b/examples/ai-transport-message-per-token/javascript/index.html deleted file mode 100644 index 0d6a7baa7f..0000000000 --- a/examples/ai-transport-message-per-token/javascript/index.html +++ /dev/null @@ -1,46 +0,0 @@ - - - - - - - AI Transport Token Streaming - JavaScript - - - -
- -
-
-
- -
- - ready - - - -
-
-
- Select a prompt below to get started - -
-
-
- - -
-
- -
-
-
- - - - diff --git a/examples/ai-transport-message-per-token/javascript/src/agent.ts b/examples/ai-transport-message-per-token/javascript/src/agent.ts deleted file mode 100644 index d8a8f6d5f3..0000000000 --- a/examples/ai-transport-message-per-token/javascript/src/agent.ts +++ /dev/null @@ -1,63 +0,0 @@ -// Agent Service -// This consumes LLM streams and publishes events to Ably - -import * as Ably from 'ably'; -import { MockLLM } from './llm'; - -export class Agent { - private client: Ably.Realtime; - private channel: Ably.RealtimeChannel; - private llm: MockLLM; - - constructor(ablyKey: string, channelName: string) { - this.client = new Ably.Realtime({ - key: ablyKey, - clientId: 'ai-agent', - }); - this.channel = this.client.channels.get(channelName); - this.llm = new MockLLM(); - } - - async processPrompt(prompt: string, responseId: string): Promise { - const stream = await this.llm.responses.create(prompt); - - for await (const event of stream) { - if (event.type === 'message_start') { - // Publish response start - this.channel.publish({ - name: 'start', - data: {}, - extras: { - headers: { - responseId, - }, - }, - }); - } else if (event.type === 'message_delta') { - // Publish tokens - this.channel.publish({ - name: 'token', - data: { - token: event.text, - }, - extras: { - headers: { - responseId, - }, - }, - }); - } else if (event.type === 'message_stop') { - // Publish response stop - this.channel.publish({ - name: 'stop', - data: {}, - extras: { - headers: { - responseId, - }, - }, - }); - } - } - } -} diff --git a/examples/ai-transport-message-per-token/javascript/src/config.ts b/examples/ai-transport-message-per-token/javascript/src/config.ts deleted file mode 100644 index 8617022a2d..0000000000 --- a/examples/ai-transport-message-per-token/javascript/src/config.ts +++ /dev/null @@ -1,3 +0,0 @@ -export const config = { - ABLY_KEY: import.meta.env.VITE_ABLY_KEY || 'YOUR_ABLY_KEY_HERE', -}; diff --git a/examples/ai-transport-message-per-token/javascript/src/llm.ts b/examples/ai-transport-message-per-token/javascript/src/llm.ts deleted file mode 100644 index ab9f1061f8..0000000000 --- a/examples/ai-transport-message-per-token/javascript/src/llm.ts +++ /dev/null @@ -1,49 +0,0 @@ -// Mock LLM Service -// This simulates a generic LLM SDK with streaming capabilities - -interface StreamEvent { - type: 'message_start' | 'message_delta' | 'message_stop'; - text?: string; - responseId: string; -} - -export class MockLLM { - private readonly responseText = - 'Ably AI Transport is a solution for building stateful, steerable, multi-device AI experiences into new or existing applications. You can use AI Transport as the transport layer with any LLM or agent framework, without rebuilding your existing stack or being locked to a particular vendor.'; - - responses = { - create: (prompt: string) => this.createStream(prompt), - }; - - private async *createStream(_prompt: string): AsyncIterable { - const responseId = `resp_${crypto.randomUUID()}`; - - // Yield start event - yield { type: 'message_start', responseId }; - - // Chunk text into tokens (simulates LLM tokenization) - const tokens = this.chunkTextLikeAI(this.responseText); - - for (const token of tokens) { - // Simulate realistic delay between tokens - await new Promise((resolve) => setTimeout(resolve, Math.random() * 150 + 50)); - - // Yield token event - yield { type: 'message_delta', text: token, responseId }; - } - - // Yield stop event - yield { type: 'message_stop', responseId }; - } - - private chunkTextLikeAI(text: string): string[] { - const chunks: string[] = []; - let pos = 0; - while (pos < text.length) { - const size = Math.floor(Math.random() * 8) + 1; - chunks.push(text.slice(pos, pos + size)); - pos += size; - } - return chunks.filter((chunk) => chunk.length > 0); - } -} diff --git a/examples/ai-transport-message-per-token/javascript/src/script.ts b/examples/ai-transport-message-per-token/javascript/src/script.ts deleted file mode 100644 index 398368acff..0000000000 --- a/examples/ai-transport-message-per-token/javascript/src/script.ts +++ /dev/null @@ -1,126 +0,0 @@ -import * as Ably from 'ably'; -import { Agent } from './agent'; -import { config } from './config'; - -// Generate unique channel name for this session -const CHANNEL_NAME = `ai-transport-${crypto.randomUUID()}`; -const client = new Ably.Realtime({ - key: config.ABLY_KEY, -}); -const channel = client.channels.get(CHANNEL_NAME); - -// Agent for processing prompts -const agent = new Agent(config.ABLY_KEY, CHANNEL_NAME); - -const responseTextElement = document.getElementById('response-text') as HTMLDivElement; -const connectionToggle = document.getElementById('connection-toggle') as HTMLButtonElement; -const promptButton = document.getElementById('prompt-button') as HTMLButtonElement; -const processingStatus = document.getElementById('processing-status') as HTMLSpanElement; - -let currentResponseId: string | null = null; -let responseCompleted = false; -let responseText = ''; -let isHydrating = false; -let pendingTokens: string[] = []; - -const updateDisplay = () => { - responseTextElement.innerText = responseText; - processingStatus.innerText = responseCompleted ? 'Completed' : 'In Progress'; -}; - -channel.subscribe('start', (message) => { - const responseId = message.extras?.headers?.responseId; - if (responseId && currentResponseId === responseId) { - responseCompleted = false; - responseText = ''; - pendingTokens = []; - updateDisplay(); - } -}); - -channel.subscribe('token', (message) => { - const responseId = message.extras?.headers?.responseId; - if (responseId && currentResponseId === responseId) { - if (isHydrating) { - pendingTokens.push(message.data.token); - } else { - responseText += message.data.token; - updateDisplay(); - } - } -}); - -channel.subscribe('stop', (message) => { - const responseId = message.extras?.headers?.responseId; - if (responseId && currentResponseId === responseId) { - responseCompleted = true; - updateDisplay(); - } -}); - -// Hydrate from history after reattaching -const hydrateFromHistory = async () => { - if (!currentResponseId) { - isHydrating = false; - return; - } - - let page = await channel.history({ untilAttach: true }); - - const historyTokens: string[] = []; - while (page) { - for (const message of page.items) { - const responseId = message.extras?.headers?.responseId; - if (responseId !== currentResponseId) { - continue; - } - if (message.name === 'token') { - historyTokens.push(message.data.token); - } else if (message.name === 'stop') { - responseCompleted = true; - } - } - page = page.hasNext() ? await page.next() : null; - } - - // History arrives newest-first, so reverse it - // Then append any tokens that arrived during hydration - responseText = historyTokens.reverse().join('') + pendingTokens.join(''); - isHydrating = false; - updateDisplay(); -}; - -const handlePromptClick = () => { - currentResponseId = `request-${crypto.randomUUID()}`; - responseText = ''; - updateDisplay(); - agent.processPrompt('What is Ably AI Transport?', currentResponseId); -}; - -const handleConnect = async () => { - // Set hydrating before attach to buffer any live tokens - isHydrating = true; - pendingTokens = []; - - await channel.attach(); - await hydrateFromHistory(); - - connectionToggle.innerText = 'Disconnect'; -}; - -const handleDisconnect = async () => { - await channel.detach(); - processingStatus.innerText = 'Paused'; - connectionToggle.innerText = 'Connect'; -}; - -const handleConnectionToggle = () => { - if (channel.state === 'attached') { - handleDisconnect(); - } else { - handleConnect(); - } -}; - -connectionToggle.onclick = handleConnectionToggle; -promptButton.onclick = handlePromptClick; diff --git a/examples/ai-transport-message-per-token/react/README.md b/examples/ai-transport-message-per-token/react/README.md deleted file mode 100644 index d696ea421e..0000000000 --- a/examples/ai-transport-message-per-token/react/README.md +++ /dev/null @@ -1,60 +0,0 @@ -# AI Transport message per token streaming - -Enable realtime streaming of AI/LLM responses by publishing tokens as they arrive from Large Language Model services. - -AI Transport token streaming allows applications to provide immediate, responsive AI interactions by streaming tokens in realtime rather than waiting for complete responses. This pattern is essential for creating engaging AI-powered experiences where users can see responses being generated as they happen. - -The streaming approach significantly improves perceived performance and user engagement. Instead of waiting 5-10 seconds for a complete AI response, users see tokens appearing progressively, creating a more natural conversation flow similar to watching someone type in realtime. - -Token streaming is implemented using [Ably AI Transport](/docs/ai-transport). AI Transport provides purpose-built APIs for realtime AI applications, offering reliable message delivery, automatic ordering, and seamless reconnection handling to ensure no tokens are lost during network interruptions. - -## Resources - -Use the following components to implement AI Transport token streaming: - -- [`AblyProvider`](/docs/getting-started/react-hooks#ably-provider): initializes and manages a shared Ably client instance, passing it down through React context to enable realtime AI Transport functionality across the application. -- [`ChannelProvider`](/docs/getting-started/react-hooks#channel-provider): manages the state and functionality of a specific channel, providing access to AI response tokens and streaming state via React context. -- [`useChannel()`](/docs/getting-started/react-hooks#useChannel) hook: a hook to subscribe to token messages from AI services and manage streaming state. -- [`untilAttach`](/docs/channels/options#attach) history option: enables seamless message recovery during reconnections, ensuring no tokens are lost when connectivity is restored. - -Find out more about [AI Transport](/docs/ai-transport) and [message history](/docs/channels/history). - -## Getting started - -1. Clone the [Ably docs](https://github.com/ably/docs) repository where this example can be found: - - ```sh - git clone git@github.com:ably/docs.git - ``` - -2. Change directory: - - ```sh - cd examples/ - ``` - -3. Rename the environment file: - - ```sh - mv .env.example .env.local - ``` - -4. In `.env.local` update the value of `VITE_ABLY_KEY` to be your Ably API key. - -5. Install dependencies: - - ```sh - yarn install - ``` - -6. Run the server: - - ```sh - yarn run ai-transport-message-per-token-react - ``` - -7. Try it out by opening [http://localhost:5173/](http://localhost:5173/) with your browser and selecting a prompt to see realtime AI token streaming. - -## Open in CodeSandbox - -In CodeSandbox, rename the `.env.example` file to `.env.local` and update the value of your `VITE_ABLY_KEY` variable to use your Ably API key. \ No newline at end of file diff --git a/examples/ai-transport-message-per-token/react/src/App.tsx b/examples/ai-transport-message-per-token/react/src/App.tsx deleted file mode 100644 index c7ecc026ee..0000000000 --- a/examples/ai-transport-message-per-token/react/src/App.tsx +++ /dev/null @@ -1,183 +0,0 @@ -import React, { useState, useRef } from 'react'; -import { AblyProvider, ChannelProvider, useChannel, useConnectionStateListener } from 'ably/react'; -import { Realtime, Message } from 'ably'; -import { Agent } from './agent'; -import { config } from './config'; -import './styles/styles.css'; - -// Generate unique channel name for this session -const CHANNEL_NAME = `ai-transport-${crypto.randomUUID()}`; -const client = new Realtime({ - key: config.ABLY_KEY, -}); - -const AITransportDemo: React.FC = () => { - const [currentResponse, setCurrentResponse] = useState(''); - const [isProcessing, setIsProcessing] = useState(false); - const [connectionState, setConnectionState] = useState('disconnected'); - const [isChannelDetached, setIsChannelDetached] = useState(false); - - const currentResponseId = useRef(null); - const isHydrating = useRef(false); - const pendingTokens = useRef([]); - - // Agent persists across renders to avoid creating new connections - const agentRef = React.useRef(null); - if (!agentRef.current) { - agentRef.current = new Agent(config.ABLY_KEY, CHANNEL_NAME); - } - - const { channel } = useChannel(CHANNEL_NAME, (message: Message) => { - const responseId = message.extras?.headers?.responseId; - - if (!currentResponseId.current || responseId !== currentResponseId.current) { - return; - } - - if (message.name === 'start') { - setCurrentResponse(''); - pendingTokens.current = []; - } else if (message.name === 'token') { - if (isHydrating.current) { - // Buffer tokens while hydrating from history - pendingTokens.current.push(message.data.token); - } else { - setCurrentResponse((prev) => prev + message.data.token); - } - } else if (message.name === 'stop') { - setIsProcessing(false); - } - }); - - useConnectionStateListener((stateChange: { current: string }) => { - setConnectionState(stateChange.current); - }); - - const handlePromptClick = () => { - if (isProcessing || connectionState !== 'connected' || isChannelDetached) { - return; - } - - setIsProcessing(true); - setCurrentResponse(''); - - const responseId = `request-${crypto.randomUUID()}`; - currentResponseId.current = responseId; - agentRef.current?.processPrompt('What is Ably AI Transport?', responseId); - }; - - const handleDisconnect = () => { - channel.detach(); - setIsChannelDetached(true); - }; - - const handleReconnect = async () => { - isHydrating.current = true; - pendingTokens.current = []; - - setIsChannelDetached(false); - await channel.attach(); - - if (currentResponseId.current) { - let page = await channel.history({ untilAttach: true }); - - const historyTokens: string[] = []; - let foundStreamComplete = false; - - while (page) { - for (const message of page.items) { - const responseId = message.extras?.headers?.responseId; - if (responseId === currentResponseId.current) { - if (message.name === 'token') { - historyTokens.push(message.data.token); - } else if (message.name === 'stop') { - foundStreamComplete = true; - } - } - } - page = page.hasNext() ? await page.next() : null; - } - - // History arrives newest-first, so reverse it - // Then append any tokens that arrived during hydration - setCurrentResponse(historyTokens.reverse().join('') + pendingTokens.current.join('')); - - if (foundStreamComplete) { - setIsProcessing(false); - } - } - - isHydrating.current = false; - }; - - return ( -
- {/* Response section with always visible status */} -
-
-
-
- - - {isChannelDetached && isProcessing - ? 'Paused' - : isProcessing - ? 'Streaming' - : currentResponse - ? 'Complete' - : connectionState === 'connected' - ? 'Ready' - : 'Disconnected'} - - {/* Disconnect/Reconnect button */} - -
-
-
- {currentResponse || (isProcessing ? 'Thinking...' : 'Select a prompt below to get started')} - {isProcessing && ▋} -
-
-
- - {/* Prompt selection */} -
-
- -
-
-
- ); -}; - -// Main App component with providers -const App: React.FC = () => { - return ( - - - - - - ); -}; - -export default App; diff --git a/examples/ai-transport-message-per-token/react/src/agent.ts b/examples/ai-transport-message-per-token/react/src/agent.ts deleted file mode 100644 index d8a8f6d5f3..0000000000 --- a/examples/ai-transport-message-per-token/react/src/agent.ts +++ /dev/null @@ -1,63 +0,0 @@ -// Agent Service -// This consumes LLM streams and publishes events to Ably - -import * as Ably from 'ably'; -import { MockLLM } from './llm'; - -export class Agent { - private client: Ably.Realtime; - private channel: Ably.RealtimeChannel; - private llm: MockLLM; - - constructor(ablyKey: string, channelName: string) { - this.client = new Ably.Realtime({ - key: ablyKey, - clientId: 'ai-agent', - }); - this.channel = this.client.channels.get(channelName); - this.llm = new MockLLM(); - } - - async processPrompt(prompt: string, responseId: string): Promise { - const stream = await this.llm.responses.create(prompt); - - for await (const event of stream) { - if (event.type === 'message_start') { - // Publish response start - this.channel.publish({ - name: 'start', - data: {}, - extras: { - headers: { - responseId, - }, - }, - }); - } else if (event.type === 'message_delta') { - // Publish tokens - this.channel.publish({ - name: 'token', - data: { - token: event.text, - }, - extras: { - headers: { - responseId, - }, - }, - }); - } else if (event.type === 'message_stop') { - // Publish response stop - this.channel.publish({ - name: 'stop', - data: {}, - extras: { - headers: { - responseId, - }, - }, - }); - } - } - } -} diff --git a/examples/ai-transport-message-per-token/react/src/llm.ts b/examples/ai-transport-message-per-token/react/src/llm.ts deleted file mode 100644 index ab9f1061f8..0000000000 --- a/examples/ai-transport-message-per-token/react/src/llm.ts +++ /dev/null @@ -1,49 +0,0 @@ -// Mock LLM Service -// This simulates a generic LLM SDK with streaming capabilities - -interface StreamEvent { - type: 'message_start' | 'message_delta' | 'message_stop'; - text?: string; - responseId: string; -} - -export class MockLLM { - private readonly responseText = - 'Ably AI Transport is a solution for building stateful, steerable, multi-device AI experiences into new or existing applications. You can use AI Transport as the transport layer with any LLM or agent framework, without rebuilding your existing stack or being locked to a particular vendor.'; - - responses = { - create: (prompt: string) => this.createStream(prompt), - }; - - private async *createStream(_prompt: string): AsyncIterable { - const responseId = `resp_${crypto.randomUUID()}`; - - // Yield start event - yield { type: 'message_start', responseId }; - - // Chunk text into tokens (simulates LLM tokenization) - const tokens = this.chunkTextLikeAI(this.responseText); - - for (const token of tokens) { - // Simulate realistic delay between tokens - await new Promise((resolve) => setTimeout(resolve, Math.random() * 150 + 50)); - - // Yield token event - yield { type: 'message_delta', text: token, responseId }; - } - - // Yield stop event - yield { type: 'message_stop', responseId }; - } - - private chunkTextLikeAI(text: string): string[] { - const chunks: string[] = []; - let pos = 0; - while (pos < text.length) { - const size = Math.floor(Math.random() * 8) + 1; - chunks.push(text.slice(pos, pos + size)); - pos += size; - } - return chunks.filter((chunk) => chunk.length > 0); - } -} diff --git a/examples/ai-transport-message-per-token/react/src/styles/styles.css b/examples/ai-transport-message-per-token/react/src/styles/styles.css deleted file mode 100644 index bd6213e1df..0000000000 --- a/examples/ai-transport-message-per-token/react/src/styles/styles.css +++ /dev/null @@ -1,3 +0,0 @@ -@tailwind base; -@tailwind components; -@tailwind utilities; \ No newline at end of file diff --git a/examples/ai-transport-message-per-token/react/tailwind.config.ts b/examples/ai-transport-message-per-token/react/tailwind.config.ts deleted file mode 100644 index 1c86e1c371..0000000000 --- a/examples/ai-transport-message-per-token/react/tailwind.config.ts +++ /dev/null @@ -1,9 +0,0 @@ -import baseConfig from '../../tailwind.config'; -import type { Config } from 'tailwindcss'; - -const config: Config = { - ...baseConfig, - content: ['./src/**/*.{js,ts,tsx}', './index.html'], -}; - -export default config; diff --git a/examples/ai-transport-message-per-token/react/vite.config.ts b/examples/ai-transport-message-per-token/react/vite.config.ts deleted file mode 100644 index 3b1cf13b4f..0000000000 --- a/examples/ai-transport-message-per-token/react/vite.config.ts +++ /dev/null @@ -1,7 +0,0 @@ -import { defineConfig } from 'vite'; -import baseConfig from '../../vite.config'; - -export default defineConfig({ - ...baseConfig, - envDir: '../../', -}); diff --git a/examples/ai-transport-token-streaming/react/README.md b/examples/ai-transport-token-streaming/react/README.md new file mode 100644 index 0000000000..1516108be0 --- /dev/null +++ b/examples/ai-transport-token-streaming/react/README.md @@ -0,0 +1,58 @@ +# AI Transport token streaming + +Stream AI/LLM responses to clients in realtime using the [Ably AI Transport SDK](/docs/ai-transport). + +AI Transport streams a response by appending tokens to a single durable message as the model generates them. A subscribing client receives each token as it arrives, and a client that joins late, refreshes, or reconnects sees the accumulated message rather than replaying every token. The SDK handles the channel, encoding, ordering, and recovery, so your application code stays close to the model and the UI. + +This example mirrors the [Core SDK getting started guide](/docs/ai-transport/getting-started/core-sdk). The model is mocked so the example runs without an API key, but the AI Transport SDK code is identical to a production integration. In a real app the agent runs on a server and the token stream comes from your model provider (for example `streamText(...).toUIMessageStream()` from the Vercel AI SDK). + +## Resources + +Use the following AI Transport SDK methods to implement token streaming: + +- [`createAgentSession()`](/docs/ai-transport/getting-started/core-sdk): creates the server-side session that publishes the response to the channel. +- [`session.createRun()`](/docs/ai-transport/concepts/runs) and [`run.start()`](/docs/ai-transport/concepts/runs): start a turn and wait for the triggering input event. +- [`run.pipe()`](/docs/ai-transport/features/token-streaming): reads the model's output stream and appends each token to the channel message. +- [`ClientSessionProvider`](/docs/ai-transport/getting-started/core-sdk) and [`useView()`](/docs/ai-transport/getting-started/core-sdk): subscribe to the conversation and re-render as tokens are appended. + +Find out more about [AI Transport](/docs/ai-transport) and [token streaming](/docs/ai-transport/features/token-streaming). + +## Getting started + +1. Clone the [Ably docs](https://github.com/ably/docs) repository where this example can be found: + + ```sh + git clone git@github.com:ably/docs.git + ``` + +2. Change directory: + + ```sh + cd examples/ + ``` + +3. Rename the environment file: + + ```sh + mv .env.example .env.local + ``` + +4. In `.env.local` update the value of `VITE_ABLY_KEY` to be your Ably API key. + +5. Install dependencies: + + ```sh + yarn install + ``` + +6. Run the server: + + ```sh + yarn run ai-transport-token-streaming-react + ``` + +7. Try it out by opening [http://localhost:5173/](http://localhost:5173/) with your browser and selecting a prompt to see realtime AI token streaming. + +## Open in CodeSandbox + +In CodeSandbox, rename the `.env.example` file to `.env.local` and update the value of your `VITE_ABLY_KEY` variable to use your Ably API key. diff --git a/examples/ai-transport-message-per-token/react/index.html b/examples/ai-transport-token-streaming/react/index.html similarity index 84% rename from examples/ai-transport-message-per-token/react/index.html rename to examples/ai-transport-token-streaming/react/index.html index d7655daeb7..8324c8cca6 100644 --- a/examples/ai-transport-message-per-token/react/index.html +++ b/examples/ai-transport-token-streaming/react/index.html @@ -3,7 +3,7 @@ - AI Transport Token Streaming + AI Transport token streaming
diff --git a/examples/ai-transport-message-per-token/react/package.json b/examples/ai-transport-token-streaming/react/package.json similarity index 73% rename from examples/ai-transport-message-per-token/react/package.json rename to examples/ai-transport-token-streaming/react/package.json index b9e509d03f..3a13893bb9 100644 --- a/examples/ai-transport-message-per-token/react/package.json +++ b/examples/ai-transport-token-streaming/react/package.json @@ -1,5 +1,5 @@ { - "name": "ai-transport-message-per-token-react", + "name": "ai-transport-token-streaming-react", "version": "1.0.0", "type": "module", "scripts": { diff --git a/examples/ai-transport-message-per-response/react/postcss.config.js b/examples/ai-transport-token-streaming/react/postcss.config.js similarity index 100% rename from examples/ai-transport-message-per-response/react/postcss.config.js rename to examples/ai-transport-token-streaming/react/postcss.config.js diff --git a/examples/ai-transport-token-streaming/react/src/App.tsx b/examples/ai-transport-token-streaming/react/src/App.tsx new file mode 100644 index 0000000000..70cfd7c2a0 --- /dev/null +++ b/examples/ai-transport-token-streaming/react/src/App.tsx @@ -0,0 +1,103 @@ +import React from 'react'; +import * as Ably from 'ably'; +import { AblyProvider } from 'ably/react'; +import { createSessionHooks, ActiveRun } from '@ably/ai-transport/react'; +import { UIMessageCodec, VercelInput, VercelOutput, VercelProjection } from '@ably/ai-transport/vercel'; +import type { UIMessage } from 'ai'; +import { runAgent } from './agent'; +import { config } from './config'; +import './styles/styles.css'; + +// Bind the codec's types once so the hooks need no type parameters at call sites. +const { ClientSessionProvider, useClientSession, useView } = createSessionHooks< + VercelInput, + VercelOutput, + VercelProjection, + UIMessage +>(); + +// Generate a unique session (channel) name for this demo. +const CHANNEL_NAME = `ai:token-streaming-${crypto.randomUUID()}`; + +const client = new Ably.Realtime({ + key: config.ABLY_KEY, + clientId: 'user', +}); + +// Wake the agent. In production this POSTs `run.toInvocation().toJSON()` to your +// agent endpoint; here it invokes the in-browser agent directly. +const wakeAgent = (run: ActiveRun) => runAgent(run.toInvocation().toJSON()); + +const Chat: React.FC = () => { + const { session } = useClientSession(); + const view = useView({ limit: 30 }); + const { messages, runOf } = view; + + // The latest Run is still streaming until it reaches a terminal status. + const latestRun = runOf(messages.at(-1)?.codecMessageId ?? ''); + const isStreaming = latestRun !== undefined && latestRun.status !== 'complete' && latestRun.status !== 'cancelled'; + + const handlePrompt = async () => { + // Publish the user message on the channel, then wake the agent. + const run = await view.send( + UIMessageCodec.createUserMessage({ + id: crypto.randomUUID(), + role: 'user', + parts: [{ type: 'text', text: 'What is Ably AI Transport?' }], + }), + ); + await wakeAgent(run); + }; + + const stop = async () => { + if (latestRun) { + await session.cancel(latestRun.runId); + } + }; + + return ( +
+
+ {messages.length === 0 ? ( +

Select the prompt below to stream a response.

+ ) : ( + messages.map(({ codecMessageId, message }) => ( +
+ {message.role}: + {message.parts.map((part, i) => (part.type === 'text' ? {part.text} : null))} +
+ )) + )} +
+ +
+ + {isStreaming && ( + + )} +
+
+ ); +}; + +const App: React.FC = () => ( + + + + + +); + +export default App; diff --git a/examples/ai-transport-token-streaming/react/src/agent.ts b/examples/ai-transport-token-streaming/react/src/agent.ts new file mode 100644 index 0000000000..453542ab75 --- /dev/null +++ b/examples/ai-transport-token-streaming/react/src/agent.ts @@ -0,0 +1,46 @@ +// Agent +// In production this runs on a server (for example a Next.js route handler). It +// is run in the browser here so the example is self-contained, but the AI +// Transport SDK code is identical to a real agent endpoint. +// See https://ably.com/docs/ai-transport/getting-started/core-sdk +import * as Ably from 'ably'; +import { createAgentSession, Invocation, InvocationData } from '@ably/ai-transport'; +import { UIMessageCodec } from '@ably/ai-transport/vercel'; +import { streamMockResponse } from './llm'; +import { config } from './config'; + +// Receive an invocation, start a Run, hydrate the conversation, stream the +// response to the channel, and end the Run. +export const runAgent = async (invocationData: InvocationData): Promise => { + const ably = new Ably.Realtime({ key: config.ABLY_KEY, clientId: 'agent' }); + const invocation = Invocation.fromJSON(invocationData); + + const session = createAgentSession({ + client: ably, + channelName: invocation.sessionName, + codec: UIMessageCodec, + }); + + await session.connect(); + const run = session.createRun(invocation); + + try { + await run.start(); + await run.loadConversation(); + + // In production, stream from your model provider instead: + // const result = streamText({ + // model: anthropic('claude-sonnet-4-20250514'), + // messages: convertToModelMessages(run.messages), + // abortSignal: run.abortSignal, + // }); + // const { reason } = await run.pipe(result.toUIMessageStream()); + const { reason } = await run.pipe(streamMockResponse(run.abortSignal)); + await run.end({ reason }); + } catch { + await run.end({ reason: 'error' }); + } finally { + await session.close(); + ably.close(); + } +}; diff --git a/examples/ai-transport-message-per-response/react/src/config.ts b/examples/ai-transport-token-streaming/react/src/config.ts similarity index 100% rename from examples/ai-transport-message-per-response/react/src/config.ts rename to examples/ai-transport-token-streaming/react/src/config.ts diff --git a/examples/ai-transport-message-per-response/react/src/index.tsx b/examples/ai-transport-token-streaming/react/src/index.tsx similarity index 100% rename from examples/ai-transport-message-per-response/react/src/index.tsx rename to examples/ai-transport-token-streaming/react/src/index.tsx diff --git a/examples/ai-transport-token-streaming/react/src/llm.ts b/examples/ai-transport-token-streaming/react/src/llm.ts new file mode 100644 index 0000000000..ddd0331ab8 --- /dev/null +++ b/examples/ai-transport-token-streaming/react/src/llm.ts @@ -0,0 +1,46 @@ +// Fake LLM +// Produces the same `UIMessageChunk` stream that Vercel AI SDK's +// `streamText(...).toUIMessageStream()` emits. Because the shape is identical, +// the agent code that consumes it (see agent.ts) is the same as a production +// integration. Only the source of the tokens is mocked. +import type { UIMessageChunk } from 'ai'; + +const RESPONSE_TEXT = + 'Ably AI Transport is a solution for building stateful, steerable, multi-device AI experiences into new or existing applications. You can use AI Transport as the transport layer with any LLM or agent framework, without rebuilding your existing stack or being locked to a particular vendor.'; + +// Split text into small chunks to simulate model tokenization. +const tokenize = (text: string): string[] => { + const tokens: string[] = []; + let pos = 0; + while (pos < text.length) { + const size = Math.floor(Math.random() * 8) + 1; + tokens.push(text.slice(pos, pos + size)); + pos += size; + } + return tokens; +}; + +// Stream a mocked response as `UIMessageChunk` events. In production this is +// `streamText({ model, messages, abortSignal }).toUIMessageStream()`. +export const streamMockResponse = (signal?: AbortSignal): ReadableStream => { + const textId = crypto.randomUUID(); + const tokens = tokenize(RESPONSE_TEXT); + + return new ReadableStream({ + async start(controller) { + controller.enqueue({ type: 'start' }); + controller.enqueue({ type: 'text-start', id: textId }); + + for (const token of tokens) { + if (signal?.aborted) break; + // Simulate inter-token latency from the model. + await new Promise((resolve) => setTimeout(resolve, Math.random() * 120 + 40)); + controller.enqueue({ type: 'text-delta', id: textId, delta: token }); + } + + controller.enqueue({ type: 'text-end', id: textId }); + controller.enqueue({ type: 'finish' }); + controller.close(); + }, + }); +}; diff --git a/examples/ai-transport-message-per-response/javascript/src/styles.css b/examples/ai-transport-token-streaming/react/src/styles/styles.css similarity index 100% rename from examples/ai-transport-message-per-response/javascript/src/styles.css rename to examples/ai-transport-token-streaming/react/src/styles/styles.css diff --git a/examples/ai-transport-message-per-response/javascript/tailwind.config.ts b/examples/ai-transport-token-streaming/react/tailwind.config.ts similarity index 100% rename from examples/ai-transport-message-per-response/javascript/tailwind.config.ts rename to examples/ai-transport-token-streaming/react/tailwind.config.ts diff --git a/examples/ai-transport-message-per-response/react/tsconfig.json b/examples/ai-transport-token-streaming/react/tsconfig.json similarity index 100% rename from examples/ai-transport-message-per-response/react/tsconfig.json rename to examples/ai-transport-token-streaming/react/tsconfig.json diff --git a/examples/ai-transport-message-per-response/react/tsconfig.node.json b/examples/ai-transport-token-streaming/react/tsconfig.node.json similarity index 100% rename from examples/ai-transport-message-per-response/react/tsconfig.node.json rename to examples/ai-transport-token-streaming/react/tsconfig.node.json diff --git a/examples/ai-transport-message-per-response/javascript/vite.config.ts b/examples/ai-transport-token-streaming/react/vite.config.ts similarity index 100% rename from examples/ai-transport-message-per-response/javascript/vite.config.ts rename to examples/ai-transport-token-streaming/react/vite.config.ts diff --git a/examples/package.json b/examples/package.json index b8a58f6e4e..d332529b43 100644 --- a/examples/package.json +++ b/examples/package.json @@ -6,10 +6,9 @@ "node": ">=20.0.0" }, "workspaces": [ - "ai-transport-message-per-response/javascript", - "ai-transport-message-per-response/react", - "ai-transport-message-per-token/javascript", - "ai-transport-message-per-token/react", + "ai-transport-token-streaming/react", + "pub-sub-message-append/javascript", + "pub-sub-message-append/react", "auth-generate-jwt/react", "auth-generate-jwt/javascript", "auth-generate-jwt/server", @@ -59,10 +58,9 @@ "spaces-member-location/javascript" ], "scripts": { - "ai-transport-message-per-response-javascript": "yarn workspace ai-transport-message-per-response-javascript dev", - "ai-transport-message-per-response-react": "yarn workspace ai-transport-message-per-response-react dev", - "ai-transport-message-per-token-javascript": "yarn workspace ai-transport-message-per-token-javascript dev", - "ai-transport-message-per-token-react": "yarn workspace ai-transport-message-per-token-react dev", + "ai-transport-token-streaming-react": "yarn workspace ai-transport-token-streaming-react dev", + "pub-sub-message-append-javascript": "yarn workspace pub-sub-message-append-javascript dev", + "pub-sub-message-append-react": "yarn workspace pub-sub-message-append-react dev", "auth-generate-jwt-javascript": "yarn workspace auth-generate-jwt-javascript dev", "auth-generate-jwt-react": "yarn workspace auth-generate-jwt-react dev", "auth-generate-jwt-server": "yarn workspace auth-generate-jwt-server dev", @@ -112,10 +110,12 @@ "spaces-member-location-react": "yarn workspace spaces-member-location-react dev" }, "dependencies": { + "@ably/ai-transport": "~0.3.0", "@ably/chat": "~1.3.1", "@ably/chat-react-ui-kit": "~0.3.0", "@ably/spaces": "~0.4.0", - "ably": "~2.21.0", + "ably": "~2.23.0", + "ai": "^6", "cors": "^2.8.5", "franken-ui": "^2.0.0", "lodash": "^4.18.1", diff --git a/examples/pub-sub-message-append/javascript/README.md b/examples/pub-sub-message-append/javascript/README.md new file mode 100644 index 0000000000..8067c89bb2 --- /dev/null +++ b/examples/pub-sub-message-append/javascript/README.md @@ -0,0 +1,59 @@ +# Pub/Sub message append + +Stream a response by appending tokens to a single Ably message, so the full response is one compacted entry in channel history. + +This example uses the Ably Pub/Sub SDK directly. An agent publishes an initial message, then appends each token to it as the response is generated. Subscribing clients receive each appended token in realtime, while a client that joins late uses `rewind` to load the accumulated message rather than replaying every token. + +If you are building AI token streaming, use the [Ably AI Transport SDK](/docs/ai-transport) instead. AI Transport implements this append pattern for you, along with stream lifecycle, codecs, reconnection, and history. See the AI Transport [token streaming](/docs/ai-transport/features/token-streaming) example for the recommended approach. + +## Resources + +Use the following methods to implement message appending with the Pub/Sub SDK: + +- [`client.channels.get()`](/docs/channels#create): creates a new or retrieves an existing channel. +- [`channel.publish()`](/docs/channels#publish): publishes the initial message and captures the serial for appending. +- [`channel.appendMessage()`](/docs/api/realtime-sdk/channels#append-message): appends each token to the message as it arrives. +- [`channel.subscribe()`](/docs/channels#subscribe): subscribes to messages, handling `message.create`, `message.append`, and `message.update` actions. +- [`channel.setOptions()`](/docs/channels/options) with [`rewind`](/docs/channels/options/rewind): recovers history on reconnection, delivering historical messages as `message.update` events. + +Find out more about [message appending](/docs/api/realtime-sdk/channels#append-message) and [AI Transport token streaming](/docs/ai-transport/features/token-streaming). + +## Getting started + +1. Clone the [Ably docs](https://github.com/ably/docs) repository where this example can be found: + + ```sh + git clone git@github.com:ably/docs.git + ``` + +2. Change directory: + + ```sh + cd examples/ + ``` + +3. Rename the environment file: + + ```sh + mv .env.example .env.local + ``` + +4. In `.env.local` update the value of `VITE_ABLY_KEY` to be your Ably API key. + +5. Install dependencies: + + ```sh + yarn install + ``` + +6. Run the server: + + ```sh + yarn run pub-sub-message-append-javascript + ``` + +7. Try it out by opening [http://localhost:5173/](http://localhost:5173/) with your browser and selecting a prompt to see tokens appended to a single message in realtime. + +## Open in CodeSandbox + +In CodeSandbox, rename the `.env.example` file to `.env.local` and update the value of your `VITE_ABLY_KEY` variable to use your Ably API key. diff --git a/examples/ai-transport-message-per-response/javascript/index.html b/examples/pub-sub-message-append/javascript/index.html similarity index 96% rename from examples/ai-transport-message-per-response/javascript/index.html rename to examples/pub-sub-message-append/javascript/index.html index b002b18e24..d5f055db1d 100644 --- a/examples/ai-transport-message-per-response/javascript/index.html +++ b/examples/pub-sub-message-append/javascript/index.html @@ -4,7 +4,7 @@ - AI Transport Message Per Response - JavaScript + Pub/Sub message append - JavaScript diff --git a/examples/ai-transport-message-per-response/react/package.json b/examples/pub-sub-message-append/javascript/package.json similarity index 72% rename from examples/ai-transport-message-per-response/react/package.json rename to examples/pub-sub-message-append/javascript/package.json index efedff88ad..2faa0d5d90 100644 --- a/examples/ai-transport-message-per-response/react/package.json +++ b/examples/pub-sub-message-append/javascript/package.json @@ -1,5 +1,5 @@ { - "name": "ai-transport-message-per-response-react", + "name": "pub-sub-message-append-javascript", "version": "1.0.0", "type": "module", "scripts": { diff --git a/examples/ai-transport-message-per-response/react/src/agent.ts b/examples/pub-sub-message-append/javascript/src/agent.ts similarity index 85% rename from examples/ai-transport-message-per-response/react/src/agent.ts rename to examples/pub-sub-message-append/javascript/src/agent.ts index 2c297e000f..df1a07ca3d 100644 --- a/examples/ai-transport-message-per-response/react/src/agent.ts +++ b/examples/pub-sub-message-append/javascript/src/agent.ts @@ -1,6 +1,8 @@ // Agent Service -// This consumes LLM streams and publishes tokens using the message-per-response pattern -// All tokens are appended to a single message, which appears as one entry in channel history +// This consumes a stream and appends each chunk to a single Pub/Sub message, +// which appears as one entry in channel history. +// For AI token streaming, the AI Transport SDK handles this for you. See the +// AI Transport token streaming example. import * as Ably from 'ably'; import { MockLLM } from './llm'; diff --git a/examples/ai-transport-message-per-response/javascript/src/config.ts b/examples/pub-sub-message-append/javascript/src/config.ts similarity index 100% rename from examples/ai-transport-message-per-response/javascript/src/config.ts rename to examples/pub-sub-message-append/javascript/src/config.ts diff --git a/examples/ai-transport-message-per-response/javascript/src/llm.ts b/examples/pub-sub-message-append/javascript/src/llm.ts similarity index 100% rename from examples/ai-transport-message-per-response/javascript/src/llm.ts rename to examples/pub-sub-message-append/javascript/src/llm.ts diff --git a/examples/ai-transport-message-per-response/javascript/src/script.ts b/examples/pub-sub-message-append/javascript/src/script.ts similarity index 100% rename from examples/ai-transport-message-per-response/javascript/src/script.ts rename to examples/pub-sub-message-append/javascript/src/script.ts diff --git a/examples/ai-transport-message-per-response/react/src/styles/styles.css b/examples/pub-sub-message-append/javascript/src/styles.css similarity index 100% rename from examples/ai-transport-message-per-response/react/src/styles/styles.css rename to examples/pub-sub-message-append/javascript/src/styles.css diff --git a/examples/ai-transport-message-per-response/react/tailwind.config.ts b/examples/pub-sub-message-append/javascript/tailwind.config.ts similarity index 100% rename from examples/ai-transport-message-per-response/react/tailwind.config.ts rename to examples/pub-sub-message-append/javascript/tailwind.config.ts diff --git a/examples/ai-transport-message-per-response/react/vite.config.ts b/examples/pub-sub-message-append/javascript/vite.config.ts similarity index 100% rename from examples/ai-transport-message-per-response/react/vite.config.ts rename to examples/pub-sub-message-append/javascript/vite.config.ts diff --git a/examples/pub-sub-message-append/react/README.md b/examples/pub-sub-message-append/react/README.md new file mode 100644 index 0000000000..3b6ea7aff8 --- /dev/null +++ b/examples/pub-sub-message-append/react/README.md @@ -0,0 +1,68 @@ +# Pub/Sub message append + +Stream a response by appending tokens to a single Ably message, so the full response is one compacted entry in channel history. + +This example uses the Ably Pub/Sub SDK directly. An agent publishes an initial message, then appends each token to it as the response is generated. Subscribing clients receive each appended token in realtime, while a client that joins late uses `rewind` to load the accumulated message rather than replaying every token. + +If you are building AI token streaming, use the [Ably AI Transport SDK](/docs/ai-transport) instead. AI Transport implements this append pattern for you, along with stream lifecycle, codecs, reconnection, and history. See the AI Transport [token streaming](/docs/ai-transport/features/token-streaming) example for the recommended approach. + +## Resources + +Use the following components to implement message appending with the Pub/Sub SDK: + +- [`AblyProvider`](/docs/getting-started/react-hooks#ably-provider): initializes and manages a shared Ably client instance, passing it down through React context. +- [`ChannelProvider`](/docs/getting-started/react-hooks#channel-provider): manages the state and functionality of a specific channel via React context. +- [`useChannel()`](/docs/getting-started/react-hooks#useChannel) hook: subscribes to messages with `message.create`, `message.append`, and `message.update` actions. +- [`rewind`](/docs/channels/options/rewind) channel option: recovers history on reconnection, delivering historical messages as `message.update` events. +- [`appendMessage()`](/docs/api/realtime-sdk/channels#append-message): appends data to an existing message using its serial. + +Find out more about [message appending](/docs/api/realtime-sdk/channels#append-message) and [AI Transport token streaming](/docs/ai-transport/features/token-streaming). + +## Getting started + +1. Clone the [Ably docs](https://github.com/ably/docs) repository where this example can be found: + + ```sh + git clone git@github.com:ably/docs.git + ``` + +2. Change directory: + + ```sh + cd examples/ + ``` + +3. Rename the environment file: + + ```sh + mv .env.example .env.local + ``` + +4. In `.env.local` update the value of `VITE_ABLY_KEY` to be your Ably API key. + +5. Install dependencies: + + ```sh + yarn install + ``` + +6. Run the server: + + ```sh + yarn run pub-sub-message-append-react + ``` + +7. Try it out by opening [http://localhost:5173/](http://localhost:5173/) with your browser and selecting a prompt to see tokens appended to a single message in realtime. + +## Open in CodeSandbox + +In CodeSandbox, rename the `.env.example` file to `.env.local` and update the value of your `VITE_ABLY_KEY` variable to use your Ably API key. + +## How it works + +The message append pattern works by: + +1. Initial message: when a response begins, publish an initial message with the `message.create` action to the Ably channel. +2. Token streaming: append subsequent tokens to the original message by publishing them with the `message.append` action. +3. Live delivery: clients subscribed to the channel receive each appended token in realtime, allowing them to progressively render the response. +4. Compacted history: the channel history contains only one message per response, with all appended tokens concatenated together. diff --git a/examples/ai-transport-message-per-response/react/index.html b/examples/pub-sub-message-append/react/index.html similarity index 83% rename from examples/ai-transport-message-per-response/react/index.html rename to examples/pub-sub-message-append/react/index.html index 2c7da1dce4..52e719ae1e 100644 --- a/examples/ai-transport-message-per-response/react/index.html +++ b/examples/pub-sub-message-append/react/index.html @@ -3,7 +3,7 @@ - AI Transport Message Per Response + Pub/Sub message append
diff --git a/examples/ai-transport-message-per-token/javascript/package.json b/examples/pub-sub-message-append/react/package.json similarity index 71% rename from examples/ai-transport-message-per-token/javascript/package.json rename to examples/pub-sub-message-append/react/package.json index 45840dfa78..0afe00f10d 100644 --- a/examples/ai-transport-message-per-token/javascript/package.json +++ b/examples/pub-sub-message-append/react/package.json @@ -1,5 +1,5 @@ { - "name": "ai-transport-message-per-token-javascript", + "name": "pub-sub-message-append-react", "version": "1.0.0", "type": "module", "scripts": { diff --git a/examples/ai-transport-message-per-token/react/postcss.config.js b/examples/pub-sub-message-append/react/postcss.config.js similarity index 100% rename from examples/ai-transport-message-per-token/react/postcss.config.js rename to examples/pub-sub-message-append/react/postcss.config.js diff --git a/examples/ai-transport-message-per-response/react/src/App.tsx b/examples/pub-sub-message-append/react/src/App.tsx similarity index 100% rename from examples/ai-transport-message-per-response/react/src/App.tsx rename to examples/pub-sub-message-append/react/src/App.tsx diff --git a/examples/ai-transport-message-per-response/javascript/src/agent.ts b/examples/pub-sub-message-append/react/src/agent.ts similarity index 85% rename from examples/ai-transport-message-per-response/javascript/src/agent.ts rename to examples/pub-sub-message-append/react/src/agent.ts index 2c297e000f..df1a07ca3d 100644 --- a/examples/ai-transport-message-per-response/javascript/src/agent.ts +++ b/examples/pub-sub-message-append/react/src/agent.ts @@ -1,6 +1,8 @@ // Agent Service -// This consumes LLM streams and publishes tokens using the message-per-response pattern -// All tokens are appended to a single message, which appears as one entry in channel history +// This consumes a stream and appends each chunk to a single Pub/Sub message, +// which appears as one entry in channel history. +// For AI token streaming, the AI Transport SDK handles this for you. See the +// AI Transport token streaming example. import * as Ably from 'ably'; import { MockLLM } from './llm'; diff --git a/examples/ai-transport-message-per-token/react/src/config.ts b/examples/pub-sub-message-append/react/src/config.ts similarity index 100% rename from examples/ai-transport-message-per-token/react/src/config.ts rename to examples/pub-sub-message-append/react/src/config.ts diff --git a/examples/ai-transport-message-per-token/react/src/index.tsx b/examples/pub-sub-message-append/react/src/index.tsx similarity index 61% rename from examples/ai-transport-message-per-token/react/src/index.tsx rename to examples/pub-sub-message-append/react/src/index.tsx index e17d50b103..53a330f5c2 100644 --- a/examples/ai-transport-message-per-token/react/src/index.tsx +++ b/examples/pub-sub-message-append/react/src/index.tsx @@ -1,8 +1,8 @@ import { StrictMode } from 'react'; import { createRoot } from 'react-dom/client'; -import App from './App.tsx'; +import App from './App'; -createRoot(document.getElementById('root')!).render( +createRoot(document.getElementById('root')).render( , diff --git a/examples/ai-transport-message-per-response/react/src/llm.ts b/examples/pub-sub-message-append/react/src/llm.ts similarity index 100% rename from examples/ai-transport-message-per-response/react/src/llm.ts rename to examples/pub-sub-message-append/react/src/llm.ts diff --git a/examples/ai-transport-message-per-token/javascript/src/styles.css b/examples/pub-sub-message-append/react/src/styles/styles.css similarity index 64% rename from examples/ai-transport-message-per-token/javascript/src/styles.css rename to examples/pub-sub-message-append/react/src/styles/styles.css index bd6213e1df..b5c61c9567 100644 --- a/examples/ai-transport-message-per-token/javascript/src/styles.css +++ b/examples/pub-sub-message-append/react/src/styles/styles.css @@ -1,3 +1,3 @@ @tailwind base; @tailwind components; -@tailwind utilities; \ No newline at end of file +@tailwind utilities; diff --git a/examples/ai-transport-message-per-token/javascript/tailwind.config.ts b/examples/pub-sub-message-append/react/tailwind.config.ts similarity index 100% rename from examples/ai-transport-message-per-token/javascript/tailwind.config.ts rename to examples/pub-sub-message-append/react/tailwind.config.ts diff --git a/examples/ai-transport-message-per-token/react/tsconfig.json b/examples/pub-sub-message-append/react/tsconfig.json similarity index 100% rename from examples/ai-transport-message-per-token/react/tsconfig.json rename to examples/pub-sub-message-append/react/tsconfig.json diff --git a/examples/ai-transport-message-per-token/react/tsconfig.node.json b/examples/pub-sub-message-append/react/tsconfig.node.json similarity index 100% rename from examples/ai-transport-message-per-token/react/tsconfig.node.json rename to examples/pub-sub-message-append/react/tsconfig.node.json diff --git a/examples/ai-transport-message-per-token/javascript/vite.config.ts b/examples/pub-sub-message-append/react/vite.config.ts similarity index 100% rename from examples/ai-transport-message-per-token/javascript/vite.config.ts rename to examples/pub-sub-message-append/react/vite.config.ts diff --git a/src/components/Examples/ExamplesRenderer.tsx b/src/components/Examples/ExamplesRenderer.tsx index 8ff34448aa..25c634b12d 100644 --- a/src/components/Examples/ExamplesRenderer.tsx +++ b/src/components/Examples/ExamplesRenderer.tsx @@ -52,6 +52,7 @@ const getDependencies = (id: string, products: string[], activeLanguage: Languag ? { '@ably/chat': '~1.3.1', '@ably/chat-react-ui-kit': '~0.3.0', clsx: '^2.1.1' } : {}), ...(products.includes('spaces') ? { '@ably/spaces': '~0.4.0' } : {}), + ...(products.includes('ai_transport') ? { ably: '~2.23.0', '@ably/ai-transport': '~0.3.0', ai: '^6' } : {}), ...(id === 'spaces-component-locking' ? { 'usehooks-ts': '^3.1.0' } : {}), ...(id === 'pub-sub-live-voting' ? { qrcode: '^1.5.4' } : {}), ...(activeLanguage === 'react' || products.includes('chat') || products.includes('spaces') diff --git a/src/data/examples/index.ts b/src/data/examples/index.ts index 21e57f3987..6d6ee73836 100644 --- a/src/data/examples/index.ts +++ b/src/data/examples/index.ts @@ -4,24 +4,15 @@ export const DEFAULT_EXAMPLE_LANGUAGES = ['javascript', 'react']; export const examples: Example[] = [ { - id: 'ai-transport-message-per-token', - name: 'Message per token streaming', - description: 'Stream AI responses token-by-token using the message-per-token pattern.', - products: ['ai_transport'], - layout: 'single-horizontal', - visibleFiles: ['src/script.ts', 'src/llm.ts', 'src/agent.ts', 'App.tsx', 'llm.ts', 'agent.ts', 'index.tsx'], - metaTitle: 'Build AI message-per-token streaming with Ably AI Transport', - metaDescription: `Stream AI-generated tokens in realtime using the message-per-token pattern with Ably's AI Transport. Implement scalable token streaming with low latency.`, - }, - { - id: 'ai-transport-message-per-response', - name: 'Message per response streaming', - description: 'Stream AI responses by appending tokens to a single message using the message-per-response pattern.', + id: 'ai-transport-token-streaming', + name: 'Token streaming', + description: 'Stream AI responses token-by-token using the Ably AI Transport SDK.', products: ['ai_transport'], + languages: ['react'], layout: 'single-horizontal', - visibleFiles: ['src/script.ts', 'src/llm.ts', 'src/agent.ts', 'App.tsx', 'llm.ts', 'agent.ts', 'index.tsx'], - metaTitle: 'Build AI message-per-response streaming with Ably AI Transport', - metaDescription: `Stream AI-generated tokens by appending them to a single message using Ably AI Transport. Each response appears as one compacted message in channel history.`, + visibleFiles: ['App.tsx', 'agent.ts', 'llm.ts', 'index.tsx'], + metaTitle: 'Build AI token streaming with the Ably AI Transport SDK', + metaDescription: `Stream AI-generated tokens in realtime with the Ably AI Transport SDK. A durable session streams tokens to every client and serves the full response to clients that join later.`, }, { id: 'auth-generate-jwt', @@ -148,6 +139,16 @@ export const examples: Example[] = [ metaTitle: `Build a live map with Ably’s LiveObjects`, metaDescription: `Use Ably’s LiveObjects to synchronize key/value data in realtime with a LiveMap. Build dynamic, collaborative apps with reliable, low-latency synchronization at scale.`, }, + { + id: 'pub-sub-message-append', + name: 'Message append', + description: 'Append data to a single message so a streamed response is one compacted entry in channel history.', + products: ['pubsub'], + layout: 'single-horizontal', + visibleFiles: ['src/script.ts', 'src/llm.ts', 'src/agent.ts', 'App.tsx', 'llm.ts', 'agent.ts', 'index.tsx'], + metaTitle: 'Append to messages with the Ably Pub/Sub SDK', + metaDescription: `Use the Ably Pub/Sub SDK to append data to a single message. Stream a response token-by-token while keeping one compacted entry in channel history.`, + }, { id: 'pub-sub-channel-messages', name: 'Channel messages', diff --git a/src/data/languages/languageData.ts b/src/data/languages/languageData.ts index b7bbd5e8a2..e1788fdb19 100644 --- a/src/data/languages/languageData.ts +++ b/src/data/languages/languageData.ts @@ -44,7 +44,7 @@ export default { android: '1.2', }, aiTransport: { - javascript: '0.2', + javascript: '0.3', }, spaces: { javascript: '0.5', diff --git a/src/pages/docs/ai-transport/features/token-streaming.mdx b/src/pages/docs/ai-transport/features/token-streaming.mdx index b5c0e3dabb..204238fd49 100644 --- a/src/pages/docs/ai-transport/features/token-streaming.mdx +++ b/src/pages/docs/ai-transport/features/token-streaming.mdx @@ -1,6 +1,6 @@ --- title: "Token streaming" -meta_description: "Stream AI-generated tokens to clients in realtime using AI Transport, with support for message-per-response and message-per-token patterns." +meta_description: "Stream AI-generated tokens to clients in realtime using AI Transport. Tokens are appended to a single durable message, and the full response is served to clients that join later." meta_keywords: "AI Transport, token streaming, LLM streaming, message appends, real-time AI, durable streaming, Ably" intro: "Tokens are streamed to subscribing clients in realtime, as the model generates them. The same response is available as a single aggregated message to clients connecting later. AI Transport streams tokens by appending to a durable channel message." redirect_from: diff --git a/src/pages/docs/platform/pricing/examples/ai-chatbot.mdx b/src/pages/docs/platform/pricing/examples/ai-chatbot.mdx index 78afe882b2..d020cec789 100644 --- a/src/pages/docs/platform/pricing/examples/ai-chatbot.mdx +++ b/src/pages/docs/platform/pricing/examples/ai-chatbot.mdx @@ -1,6 +1,6 @@ --- title: AI support chatbot pricing example -meta_description: "Calculate AI Transport pricing for conversations with an AI chatbot. Example shows how using the message-per-response pattern and modifying the append rollup window can generate cost savings." +meta_description: "Calculate AI Transport pricing for conversations with an AI chatbot. Example shows how token streaming and modifying the append rollup window can generate cost savings." meta_keywords: "chatbot, support chat, token streaming, token cost, AI Transport pricing, Ably AI Transport pricing, stream cost, Pub/Sub pricing, realtime data delivery, Ably Pub/Sub pricing" intro: "This example shows pricing for an AI support chatbot using Ably's AI Transport, where an agent streams token responses to users. It uses Ably's per-message and per-minute billing model." ---