Repository navigation
feat(saidwire): add Saidwire STT plugin - #8457
Hudson6124 wants to merge 3 commits into
Conversation
|
|
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
…nc capabilities - STT.aclose() closes every stream it opened (WebSocket and keepalive included) - update_options() during speech sends the buffered audio and Finalize on the old connection and waits (bounded) for its final before reconnecting; the item read after the drain began carries over to the new connection - update_options() with no options, or only unchanged values, does not reconnect - update_options(interim_results=...) updates capabilities.interim_results - tests for all four
| if self._carry is not None: | ||
| data, self._carry = self._carry, None | ||
| return data |
There was a problem hiding this comment.
🟡 Reconnected speech gets incorrect timestamps
When update_options() carries a frame into the next connection, its transcripts use the previous connection's time offset. restart_provider_clock() cannot re-anchor time because _carry bypasses the channel read that anchors each connection.
Learn more
The input channel records the capture time of each queued audio frame when it is read. restart_provider_clock resets that anchor at the start of each provider connection. A frame read while reconnect starts is saved in _carry, then sent on the new connection without another channel read. Its original capture timestamp has already been removed from the channel's queue, so the new connection retains the old offset until a later frame arrives. The resulting word timestamps can point earlier than the audio they describe.
Example: A stream sends two seconds of audio, then reads the next frame just as update_options() starts reconnecting. The next server connection starts counting at zero, but that carried frame keeps the offset from the first connection; its words appear near the start of the session instead of after two seconds.
Recommended fix: Preserve the carried frame's capture anchor and explicitly re-anchor start_time_offset when it is the first frame sent on a new connection. Keep the queued-frame accounting in _InputChan consistent when carrying a frame between connections.
Was this helpful? React with 👍 or 👎 to provide feedback.
A Finalize that lands mid-word can make the server repeat the word at the cut
("coffee shops", "shop", "shops near me"), and a new segment can replay the
previous segment's tail. Finals and interims now skip words whose timings fall
inside audio an earlier final on the same connection covered, plus the re-decoded
cut word. The mark resets on every new connection. Tests included.
| if ( | ||
| not kept | ||
| and last_word | ||
| and norm | ||
| and start < last_end + _DEDUPE_CUT_S | ||
| and (norm.startswith(last_word) or last_word.startswith(norm)) | ||
| ): |
There was a problem hiding this comment.
🔴 Distinct adjacent words vanish from transcripts
When consecutive finals contain “go” then “good,” _dedupe_words discards “good” if it starts within 350 ms. Prefix matching ignores that the words occupy separate audio intervals, so the transcript loses spoken content.
Learn more
The stream records the end and normalized text of the last word in a final. The next result can contain a new word that begins soon afterward. The prefix check removes that word even when its audio starts after the prior word ends. This changes the transcript presented to the agent, including short common phrases, rather than just removing repeated decoding.
Example: A first final delivers go with end 0.40; a second final delivers good at 0.50–0.80. The second word starts after the first finishes, but good.startswith(go) and 0.50 < 0.40 + 0.35, so the second final emits no good.
Recommended fix: Require actual overlap of the candidate's timing with the last delivered word before treating prefix similarity as evidence of a cut-word replay, or track the original word's start and a reliable replay boundary. Add adjacent-word tests across finals such as go good and no no with short pauses.
Was this helpful? React with 👍 or 👎 to provide feedback.
Adds
livekit-plugins-saidwire: streaming speech-to-text with Saidwire (real-time English), from the Saidwire team.START_OF_SPEECH/END_OF_SPEECH(the EOS carries the last final's end time on the audio timeline),RECOGNITION_USAGE.Finalize; the server answers right away with the final transcript (an empty final if nothing is pending), so the session never waits on a timeout.recognize()for pre-recorded audio (POST /v1/listen).KeepAlivewhile no audio flows,update_options()with reconnect,_mark_provider_progress()on results.APIStatusError; the key never appears in error text. While the service answers HTTP 503 / 429 the stream waits forRetry-After, bounded bywarmup_timeout.early_wordevents. Standard plugin layout and alivekit-agents[saidwire]extra;uv.lockupdated withuv lock.Testing:
tests/test_plugin_saidwire_stt.py, 17 unit tests (pytest.mark.unit) against a fake server on 127.0.0.1, no network and no key: endpointing and speech-end timing, flush/Finalize, empty flush, early words, bad key, warm-up wait and timeout, server errors and reconnects, quota errors, keepalive, 48 kHz resampling, 8 kHz telephony,update_optionsandrecognize(). Passing against livekit-agents 1.8.11, plusruff check,ruff format --checkandmypy --strict.The plugin is built and maintained by Saidwire, the company that runs the service. The
livekit-plugins-saidwirename on PyPI is held by Saidwire; on merge we'll add your publisher so the release pipeline can publish it. Contact: support@saidwire.com.