OpenHRI
Open research on real-time, multimodal interaction between people and machines.
NewTwo Faces of a Turn ExchangePeople talking to each other do much more than exchange sentences. They pause without giving up the floor, nod and say “mhm” while the other person speaks, laugh at the right moment and answer within a few hundred milliseconds. Today’s voice and video agents do little of this: they wait out a timeout, ignore the camera, and talk over pauses.
OpenHRI works on that gap in the open. We build benchmarks that measure how agents behave in live, full-duplex conversation and models that decide when a person has finished speaking. Code, models and data manifests are released on GitHub and Hugging Face, with every number in our write-ups reproducible from them.
huggingface.co/OpenHRIResearch
- Two Faces of a Turn ExchangeWhat the listener's and the speaker's faces show around the moment the floor changes hands, measured in 2,670 conversations filmed in the same room and over video calls.
- Listening through the pause: streaming end-of-turn detectionStock Smart Turn asks its model once, 0.2 s after you stop talking. Asking every 100 ms with a model that knows how long the silence has lasted cuts false turn ends on real meetings from 42 % to 5 %.
- Do video-call agents look? An open reproduction of VideoFDBWe rebuilt NVIDIA's full-duplex audio-visual benchmark as runnable code, reproduced its human reference, and scored two realtime APIs and three avatar products. The agents still do not use the camera.
Projects
- Benchmark
OpenVideoFDB
An open, runnable implementation of VideoFDB, a benchmark for full-duplex audio-visual conversational agents.
- Model
smart-turn-streaming
Silence-aware streaming end-of-turn detection for voice agents, built on Pipecat’s Smart Turn v3.2 encoder.