Skip to content

Conversation

@ben-kluger-scale
Copy link

@ben-kluger-scale ben-kluger-scale commented Jan 31, 2026

Add comprehensive voice agent support to the Agentex SDK:

New SDK Module (src/agentex/voice/)

  • VoiceAgentBase: Base class with state management, interruption handling, guardrail execution, and streaming support
  • AgentState/AgentResponse: Pydantic models for conversation state
  • Guardrail/LLMGuardrail: Abstract base classes for implementing guardrails

New CLI Template (agentex init)

  • Full project scaffolding for voice agents
  • Multi-provider LLM support (OpenAI, Azure, SGP, Vertex AI, Mock)
  • Docker + Kubernetes deployment configuration
  • Example guardrail structure

CLI Changes

  • Added Conversational Agent template option in 'agentex init' command
  • Generates voice-specific project structure

Dependencies

  • Added partial-json-parser for streaming JSON parsing
  • Added google-auth for Vertex AI authentication

What's Different About This Template

Unlike standard templates where you write ACP handlers from scratch, the Conversational Agent template uses VoiceAgentBase which provides:

  • Automatic state persistence across messages
  • Interruption handling when new messages arrive mid-processing
  • Concurrent guardrail execution during streaming
  • Conversation history management

Developers implement just 2 methods (get_system_prompt, update_state_and_tracing_from_response) and get production-ready conversational agents.

What Gets Generated

Running agentex init → "Conversational Agent" creates:

my-agent/
├── project/
│   ├── __init__.py
│   └── acp.py          # Your agent implementation
├── Dockerfile
├── manifest.yaml       # Deployment configuration
├── environments.yaml   # Environment-specific settings
├── pyproject.toml      # Dependencies
├── dev.ipynb           # Development notebook
└── README.md           # Setup instructions

acp.py specifics

project/acp.py
├── LLM Configuration (lines 30-153)
│   └── Auto-detects which LLM provider you configured
│
├── State & Response Models (lines 163-184)
│   ├── MyAgentState(AgentState)     ← Track conversation data
│   └── MyAgentResponse(AgentResponse) ← Structured LLM outputs
│
├── Agent Class (lines 191-259)
│   └── MyAgent(VoiceAgentBase)
│       ├── get_system_prompt()      ← Agent prompt
│       └── update_state_and_tracing_from_response()
│
├── Guardrails (optional, lines ~270)
│   └── GUARDRAILS = [...]
│
├── Tools (optional, lines ~290)
│   └── TOOLS = [...]
│
├── Agent Instance (lines ~310)
│   └── AGENT = MyAgent(...)
│
└── ACP Handler (lines ~320)
    └── @acp.on_message_send → delegates to AGENT.send_message()

Key Code: project/acp.py

class MyAgentState(AgentState):
    """Custom state - extend to track conversation data."""
    pass

class MyAgent(VoiceAgentBase):
    """Your conversational agent."""
    
    def get_system_prompt(self, state, guardrail_override=None):
        """Define your agent's behavior."""
        return "You are a helpful assistant..."
    
    def update_state_and_tracing_from_response(self, state, response, span):
        """Update state after each response."""
        return state

# ACP Handler - delegates to VoiceAgentBase
@acp.on_message_send
async def handle_message_send(params: SendMessageParams):
    async for chunk in AGENT.send_message(params):
        yield chunk

@socket-security
Copy link

socket-security bot commented Jan 31, 2026

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff Package Supply Chain
Security
Vulnerability Quality Maintenance License
Addedpartial-json-parser@​0.2.1.1.post7100100100100100

View full report

Add comprehensive voice agent support to the Agentex SDK:

## New SDK Module (src/agentex/voice/)
- VoiceAgentBase: Base class with state management, interruption handling,
  guardrail execution, and streaming support
- AgentState/AgentResponse: Pydantic models for conversation state
- Guardrail/LLMGuardrail: Abstract base classes for implementing guardrails

## New CLI Template (agentex init --voice)
- Full project scaffolding for voice agents
- Multi-provider LLM support (OpenAI, Azure, SGP, Vertex AI, Mock)
- Docker + Kubernetes deployment configuration
- Example guardrail structure

## CLI Changes
- Added hidden --voice flag to 'agentex init' command
- Generates voice-specific project structure

## Dependencies
- Added partial-json-parser for streaming JSON parsing
- Added google-auth for Vertex AI authentication
@ben-kluger-scale ben-kluger-scale force-pushed the feat/voice-agent-integration branch from 5eecbb6 to b019504 Compare January 31, 2026 01:36
- Removed hidden --voice flag
- Added 'Conversational Agent' to template selection table
- Added description: real-time conversational agent with interruption
  handling, state management, and guardrail support
Make the template voice-provider agnostic:
- Renamed 'Voice Agent powered by LiveKit' to 'Conversational Agent'
- Removed LiveKit credential examples from manifest
- Removed LiveKit env var examples from README
- Updated docstrings to be provider-neutral

The agent exposes a standard ACP endpoint that any voice
provider (LiveKit, Twilio, etc.) can connect to.
ben-kluger-scale and others added 4 commits February 2, 2026 13:54
Co-authored-by: Michael Sun <55160142+MichaelSun48@users.noreply.github.com>
Per team feedback, keep the voice template as a hidden flag until
audio package integration is complete. Users who understand the
limitations can use: agentex init --voice
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants