fix(stt): correct off-by-one in MultiSpeakerAdapter RMS timerange window#6446
Open
chuenchen309 wants to merge 1 commit into
Open
fix(stt): correct off-by-one in MultiSpeakerAdapter RMS timerange window#6446chuenchen309 wants to merge 1 commit into
chuenchen309 wants to merge 1 commit into
Conversation
_PrimarySpeakerDetector._get_rms_for_timerange() computed the start index as `len(self._rms_buffer) - start - 1`, one frame earlier than the requested start_time. This silently pulled the RMS frame immediately *before* a speech segment into its loudness measurement, contaminating primary-speaker detection with audio that belongs to whatever was happening just prior (e.g. the other speaker, or silence). The corresponding end-time calculation (`len(self._rms_buffer) - end`) has no such offset, so the extra `- 1` on the start side was asymmetric and unintended, not a deliberate pre-roll. Verified with two new tests reproducing the bug directly against _get_rms_for_timerange(): one showing a silent frame leaking into a window that should contain only a loud frame, another showing the window contains one extra sample than requested. Both fail on the current code and pass after the fix. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
_PrimarySpeakerDetector._get_rms_for_timerange()(used bystt.MultiSpeakerAdapterto decide which diarized speaker is the "primary" one, based on RMS loudness) had an
off-by-one when converting a
[start_time, end_time)speech-segment window intoself._rms_bufferindices:The
- 1on the start side has no counterpart on the end side, so the computedwindow always includes one extra RMS frame from before
start_time. In practicethis means the loudness measurement used to pick/switch the primary speaker gets
contaminated by whatever audio (silence, or the other speaker) immediately preceded
the segment — most noticeable on short segments, where one spurious ~100ms frame is
a large fraction of the window.
Fix
Drop the stray
- 1so the start index lines up the same way the end index alreadydoes.
Verification
Added
tests/test_multi_speaker_rms_timerange.pywith two tests that construct a_PrimarySpeakerDetectordirectly, push synthetic audio frames with known RMSvalues via
push_audio(), and query_get_rms_for_timerange():test_excludes_frame_before_start_time: pushes one silent frame followed by oneloud frame, then queries only the loud frame's time range. Before the fix this
returns the average of the silent and loud frame (
500.0); after the fix itcorrectly returns just the loud frame's RMS (
1000.0).test_window_contains_exactly_the_requested_frame_count: pushes 5 silent + 5 loudframes and confirms a query spanning exactly the 5 loud frames returns a result
when
min_rms_samples=5butNonewhenmin_rms_samples=6(i.e. the windowcontains exactly 5 samples, not 6).
Both tests fail against the current code and pass after the fix (confirmed by
temporarily reverting the source change with the tests in place).
Also ran the full
--unitsuite before/after: 1011 passed both times (my 2 newtests included), 0 regressions.
grep'd the whole repo (includinglivekit-plugins/) for_get_rms_for_timerange/_PrimarySpeakerDetector/MultiSpeakerAdapter— the changed method has exactly one call site(
_update_primary_speaker, in the same file), andMultiSpeakerAdapterisn'treferenced anywhere outside
livekit-agents/livekit/agents/stt/, so this changehas no other call sites to check.
make check(format-check + lint + strict mypy) passes clean on the changed files.AI-assisted contribution disclosure
This bug was found, the fix implemented, and the verification above run by an
AI coding agent (Claude), under human supervision — the bug hunt, the diagnosis,
and the diff were reviewed by a human before this PR was opened. This is not an
unsupervised/autonomous submission.