Barge-in refactoring and remote session event support

[chenghao-mou]

• final clean up for barge-in
• dropped BaseModel support
• add remote session events for playground debug panel

[theomonnom]

changes made to the new RemoteSession lgtm!

[devin-ai-integration]

Devin Review found 1 new potential issue.

View 34 additional findings in Devin Review.

[devin-ai-integration]

🟡 _flush_held_transcripts prematurely clears _ignore_user_transcript_until when transcript buffer is empty

When the agent finishes speaking, _ignore_user_transcript_until is set to filter out STT transcripts that were generated from audio played during agent speech. Then _flush_held_transcripts is scheduled as an async task. If this task runs before any held transcripts have arrived from the STT pipeline (i.e., the buffer is still empty), the new code clears _ignore_user_transcript_until — the old code intentionally kept it.

Root Cause and Impact

In the old code at audio_recognition.py, the early-return conditions were separate:

if not self._transcript_buffer:
    return  # kept _ignore_user_transcript_until intact

In the new code, ALL four conditions trigger _reset_interruption_detection() which clears both the buffer and _ignore_user_transcript_until:

if (
    not self._interruption_enabled
    or not is_given(self._ignore_user_transcript_until)
    or not self._transcript_buffer  # <-- this branch now clears the ignore timestamp
    or self._input_started_at is None
):
    self._reset_interruption_detection()  # clears buffer AND _ignore_user_transcript_until
    return

This matters in the following race:

1. on_end_of_agent_speech sets _ignore_user_transcript_until and schedules _flush_held_transcripts
2. _flush_held_transcripts runs — buffer is still empty (STT hasn't delivered events yet)
3. New code: _ignore_user_transcript_until is cleared
4. A delayed STT event with a timestamp during agent speech arrives → _should_hold_stt_event returns False (ignore is cleared) → stale transcript leaks through

The old code kept _ignore_user_transcript_until so step 4 would still filter the stale event.

Impact: Stale transcripts from during agent speech can leak into the conversation, potentially causing the agent to respond to backchannel noise or the user's echo of the agent's own words.

Suggested change

	"""
	if not self._interruption_enabled:
	return
	if not is_given(self._ignore_user_transcript_until):
	return
	if not self._transcript_buffer:
	return
	if not self._input_started_at:
	self._transcript_buffer.clear()
	self._ignore_user_transcript_until = NOT_GIVEN
	if (
	not self._interruption_enabled
	or not is_given(self._ignore_user_transcript_until)
	or not self._transcript_buffer
	or self._input_started_at is None
	):
	self._reset_interruption_detection()
	if (
	not self._interruption_enabled
	or not is_given(self._ignore_user_transcript_until)
	or self._input_started_at is None
	):
	self._reset_interruption_detection()
	return
	if not self._transcript_buffer:
	return

---

Was this helpful? React with 👍 or 👎 to provide feedback.