Google wants video AI to understand what people need as they work
Reported by The Decoder1 source ↓
A Google-led research team built StreamVibe, a video agent designed to watch people cook or assemble furniture and speak up only when help is needed.
Key points
- The benchmark contains about 4,100 examples drawn from more than 183 hours of recordings
- StreamVibe-VL checks every second whether to stay quiet or offer help
- The researchers report a 10.7% improvement in response timing and quality over other models
- Models struggled to respond at the right time in videos longer than five minutes
Details
The Decoder reports that the research team includes members from Google, Stanford, Oxford and several U.S. universities. Its benchmark tests whether video assistants can respond to clear instructions, infer when help may be needed from vague goals, and adapt when a user changes a request. Four human evaluators had to unanimously approve each example.
The team trained StreamVibe-VL, which decides each second whether to speak, and added a separate memory system to track pending requests and completed steps. The researchers report that StreamVibe-VL improved response timing and quality by 10.7% over other models; they say the memory system was designed to further improve proactive interaction and reduce unnecessary interruptions. The Decoder reports that Gemini 3.8 Flash scored higher in several tests, while performance across tested systems dropped when models had to manage their own earlier responses over full videos.
Written from coverage as of
Stocks in this story
Change since open: 20 Nov to 21 Nov 2026
Last refresh