fix(realtime): surface pending-reply drop on reconnect via SpeechHandle.exception()#6436
Open
ByteMaster-1 wants to merge 2 commits into
Open
fix(realtime): surface pending-reply drop on reconnect via SpeechHandle.exception()#6436ByteMaster-1 wants to merge 2 commits into
ByteMaster-1 wants to merge 2 commits into
Conversation
…le.exception()
On a transient socket drop, the OpenAI realtime session reconnects and discards
any pending generate_reply future, then never re-creates the response — so the
turn silently produced nothing. The discard was raised as a bare
llm.RealtimeError, indistinguishable from a terminal failure.
Fail the discarded futures with APIConnectionError instead (retryable=True):
the session is coming back, so re-prompting is sensible. The voice turn loop
already lands generate_reply failures on the SpeechHandle via _mark_done(error=);
widen its catch to APIError so the discard reaches SpeechHandle.exception().
Apps can now distinguish a retryable drop without string-matching:
exc = handle.exception()
if isinstance(exc, APIError) and exc.retryable:
... # transient reconnect discard — safe to re-prompt
The terminal "realtime session closed" path (fatal error / retries exhausted)
stays a bare RealtimeError — nothing to retry against.
Follow-up agreed in the livekit#6352 review.
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
…eption() The say() await in _realtime_reply_task caught only RealtimeError, while the sibling generate_reply await catches APIError too. Make them symmetric so a transient reconnect discard (retryable APIError) lands on the SpeechHandle for any realtime plugin that implements say() via the response-future mechanism.
Contributor
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up agreed in #6352 review ("yes we can make the drop on reconnect inside the plugin observable via SpeechHandle.exception()").
Incident. On a transient WebSocket drop, the OpenAI realtime session reconnects and _reconnect() discards any pending generate_reply future without re-creating the response. The turn silently produced nothing, and the discard was a bare RealtimeError — indistinguishable from a terminal failure, so an application couldn't tell it was safe to re-prompt.
Fix. Discarded futures are now failed with APIConnectionError (retryable=True) — the session is coming back, so re-prompting is sensible. The voice turn loop already routes generate_reply failures onto the SpeechHandle (_mark_done(error=), from #6304); its catch is widened to APIError so the discard deterministically lands on SpeechHandle.exception(). This reuses the same retryable-flag contract #6352 introduced in this file for fatal server errors.
App contract (no new types, no string-matching):
exc = handle.exception()
if isinstance(exc, APIError) and exc.retryable:
# transient reconnect discard — safe to re-prompt
Deliberately out of scope:
generate_reply timeout still raises bare RealtimeError; converting it to APITimeoutError for the same retryable contract is a follow-up PR.
The terminal "realtime session closed" path (fatal error / retries exhausted) stays a bare RealtimeError — nothing to retry against.
A response already streaming when the socket drops is closed via _close_current_generation (partial output, no error) — that's playout truncation, a separate issue.
Timing note: generate_reply has a 10s client-side timeout. If reconnection outlasts it, the timeout wins the race and the handle gets the (bare RealtimeError) timeout instead of the discard — acceptable, and the follow-up timeout PR will fold that into the same retryable contract.
Behavior change for direct callers: anyone consuming a raw RealtimeSession.generate_reply() future and catching only RealtimeError will now see APIConnectionError on reconnect. No in-repo caller is affected (only the voice turn loop consumes that future, and it's updated here).