OpenAI still has a WebRTC Problem
Reported by Hacker News1 source ↓
A Hacker News post argues that WebRTC can lose or delay audio needed by voice AI, and proposes using MoQ to retransmit audio and schedule playback by timestamp.
Key points
- The post says WebRTC may discard late audio packets, leaving voice AI without parts of a user's prompt
- It argues retransmission could arrive in time because AI can process audio faster than real-time
- The author says MoQ can retransmit audio and use timestamps to schedule playback
- The post says its playback approach is not IETF-standardized and is supported by moq.dev clients, including Pipecat
Details
The post focuses on WebRTC's handling of audio packet loss in voice AI products. It says retransmissions are disabled by default for audio and that the jitter buffer will not wait for retransmitted packets, which can leave a system with incomplete input or lost generated speech.
The author argues that voice AI differs from human conversations because models can process audio faster than real-time. In that situation, the post says, retransmissions could arrive before the audio is needed for the model's response, avoiding a delay for the user.
The proposed alternative is MoQ, which the post says can retransmit audio and use timestamps to determine when audio should play. It gives an example of one second of Opus audio at 64 Kbps as about 8 KB, and says two minutes of audio is about 1 MB. The author notes that the described playback behavior is not IETF-standardized and is supported by moq.dev clients, including Pipecat.
Written from coverage as of