Skip to content

fix(stt): correct off-by-one in MultiSpeakerAdapter RMS timerange window#6446

Open
chuenchen309 wants to merge 1 commit into
livekit:mainfrom
chuenchen309:fix/multi-speaker-rms-timerange-off-by-one
Open

fix(stt): correct off-by-one in MultiSpeakerAdapter RMS timerange window#6446
chuenchen309 wants to merge 1 commit into
livekit:mainfrom
chuenchen309:fix/multi-speaker-rms-timerange-off-by-one

Conversation

@chuenchen309

Copy link
Copy Markdown

What

_PrimarySpeakerDetector._get_rms_for_timerange() (used by stt.MultiSpeakerAdapter
to decide which diarized speaker is the "primary" one, based on RMS loudness) had an
off-by-one when converting a [start_time, end_time) speech-segment window into
self._rms_buffer indices:

start = int((self._pushed_duration - start_time) / self._frame_size)
end = int((self._pushed_duration - end_time) / self._frame_size)
start = len(self._rms_buffer) - start - 1   # <- extra "- 1"
end = len(self._rms_buffer) - end

The - 1 on the start side has no counterpart on the end side, so the computed
window always includes one extra RMS frame from before start_time. In practice
this means the loudness measurement used to pick/switch the primary speaker gets
contaminated by whatever audio (silence, or the other speaker) immediately preceded
the segment — most noticeable on short segments, where one spurious ~100ms frame is
a large fraction of the window.

Fix

Drop the stray - 1 so the start index lines up the same way the end index already
does.

Verification

Added tests/test_multi_speaker_rms_timerange.py with two tests that construct a
_PrimarySpeakerDetector directly, push synthetic audio frames with known RMS
values via push_audio(), and query _get_rms_for_timerange():

  • test_excludes_frame_before_start_time: pushes one silent frame followed by one
    loud frame, then queries only the loud frame's time range. Before the fix this
    returns the average of the silent and loud frame (500.0); after the fix it
    correctly returns just the loud frame's RMS (1000.0).
  • test_window_contains_exactly_the_requested_frame_count: pushes 5 silent + 5 loud
    frames and confirms a query spanning exactly the 5 loud frames returns a result
    when min_rms_samples=5 but None when min_rms_samples=6 (i.e. the window
    contains exactly 5 samples, not 6).

Both tests fail against the current code and pass after the fix (confirmed by
temporarily reverting the source change with the tests in place).

uv run pytest tests/test_multi_speaker_rms_timerange.py --unit -v
# 2 passed

Also ran the full --unit suite before/after: 1011 passed both times (my 2 new
tests included), 0 regressions. grep'd the whole repo (including
livekit-plugins/) for _get_rms_for_timerange/_PrimarySpeakerDetector/
MultiSpeakerAdapter — the changed method has exactly one call site
(_update_primary_speaker, in the same file), and MultiSpeakerAdapter isn't
referenced anywhere outside livekit-agents/livekit/agents/stt/, so this change
has no other call sites to check.

make check (format-check + lint + strict mypy) passes clean on the changed files.

AI-assisted contribution disclosure

This bug was found, the fix implemented, and the verification above run by an
AI coding agent (Claude), under human supervision — the bug hunt, the diagnosis,
and the diff were reviewed by a human before this PR was opened. This is not an
unsupervised/autonomous submission.

_PrimarySpeakerDetector._get_rms_for_timerange() computed the start index
as `len(self._rms_buffer) - start - 1`, one frame earlier than the
requested start_time. This silently pulled the RMS frame immediately
*before* a speech segment into its loudness measurement, contaminating
primary-speaker detection with audio that belongs to whatever was
happening just prior (e.g. the other speaker, or silence).

The corresponding end-time calculation (`len(self._rms_buffer) - end`)
has no such offset, so the extra `- 1` on the start side was asymmetric
and unintended, not a deliberate pre-roll.

Verified with two new tests reproducing the bug directly against
_get_rms_for_timerange(): one showing a silent frame leaking into a
window that should contain only a loud frame, another showing the
window contains one extra sample than requested. Both fail on the
current code and pass after the fix.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@chuenchen309 chuenchen309 requested a review from a team as a code owner July 15, 2026 18:24
@CLAassistant

CLAassistant commented Jul 15, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

Open in Devin Review

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants