feat(agent): add update_models() to swap STT/VAD/LLM/TTS at runtime#6440
Open
longcw wants to merge 1 commit into
Open
feat(agent): add update_models() to swap STT/VAD/LLM/TTS at runtime#6440longcw wants to merge 1 commit into
longcw wants to merge 1 commit into
Conversation
Agent.update_models(*, stt, vad, llm, tts) swaps a pipeline model in place, for cases like changing the STT language or TTS voice mid-call without a full agent handoff. When the agent isn't running it replaces the stored model; when running it applies to the live pipeline: STT/VAD streams are rewired, LLM/TTS take effect on the next generation/synthesis, and metrics/error listeners follow the new model. None disables a model, matching Agent(stt=None). Swapping to or from a RealtimeModel while running raises (its session is created at start); use AgentSession.update_agent instead. Adds KeytermDetector.swap_stt to rebind the recognizer in place so the swap stays synchronous.
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Agent.update_models(*, stt, vad, llm, tts)swaps a model in place. Only the models passed are changed; strings resolve to inference models like the constructor, andNonedisables a model (overriding the session), matchingAgent(stt=None).When the agent isn't running it replaces the stored model, applied on the next start. When running, it applies to the live pipeline: the STT and VAD streams are rewired, the LLM and TTS take effect on the next generation and synthesis respectively, and each model's
metrics_collected/errorlisteners follow the new instance. The call is synchronous and atomic.The Agent is the source of truth — the activity resolves
agent.<model> if given else session.<model>live — soupdate_modelsjust writes to the agent. This is distinct fromAgentSession.update_agent(full handoff, new instance) and from a model's ownupdate_options(same instance, provider options).Alternative to #6235
Realtime models
A
RealtimeModelowns a live session created when the agent starts, so it can't be swapped in place. While running, passing aRealtimeModelasllm(or swapping away from one) raisesRuntimeErrorpointing toAgentSession.update_agent. Validation happens before any mutation, so the call stays all-or-nothing.