EMNLP · 2026
OVO-S-Bench: A Hierarchical Benchmark for Streaming Spatial Intelligence in Multimodal LLMs
Key takeaway
OVO-S-Bench shows that streaming spatial reasoning remains difficult even for strong multimodal models, with allocentric mapping a major bottleneck under prefix-only video access. arXiv abstract · v2
Abstract
Multimodal agents in robotics, AR, and autonomous driving must reason about places and layouts from continuous egocentric streams, often using evidence outside the current view. Existing benchmarks either evaluate offline over full videos or target events rather than spatial structure. We introduce OVO-S-Bench, a fully human-annotated benchmark for streaming spatial intelligence, comprising 1,680 questions over 348 source videos. Annotation involves 12 trained annotators (each also serving as a blind cross-reviewer) across roughly 804 person-hours of multi-round quality assurance. Each question carries a query timestamp and an evidence interval, and at evaluation, the model sees only the prefix preceding the query. Questions span four levels of increasing abstraction: instantaneous egocentric perception, spatiotemporal context tracking, generative spatial reasoning, and allocentric spatial mapping. Across 38 proprietary and open-source MLLMs, Gemini-3.1-Pro trails human experts evaluated under the same prefix-access protocol by 33 points (59.2 vs. 92.2), with allocentric spatial mapping as the dominant bottleneck. Notably, streaming and spatially fine-tuned MLLMs underperform their own backbones. We further find that chain-of-thought reasoning amplifies spatial errors when ungrounded in the stream. By exposing these limitations, OVO-S-Bench establishes a demanding testbed for next-generation streaming spatial MLLMs.
Author abstract · arXiv abstract · v2
Publication
Empirical Methods in Natural Language Processing (EMNLP), 2026
Paper and resources
Research topics
OVO-S-Bench · Streaming video understanding · Spatial intelligence · Egocentric video · Allocentric mapping · Prefix-only evaluation · Spatiotemporal reasoning · Multimodal benchmarks
Research problem and approach
Offline video evaluation can expose evidence unavailable when a real-time query arrives. OVO-S-Bench evaluates spatial questions using only the video prefix preceding each query timestamp. arXiv abstract · v2
Main contributions
- Provides 1,680 human-annotated questions over 348 videos, with query timestamps and evidence intervals. arXiv abstract · v2
- Organizes questions into perception, context tracking, generative spatial reasoning and allocentric mapping. arXiv abstract · v2
Method comparison
| Approach | Key difference |
|---|---|
| Offline spatial video evaluation | Can provide video evidence that would not yet exist at the time of a real-time query. |
| OVO-S-Bench | Restricts every query to its preceding video prefix and tests four spatial capability levels. |
Selected results
- Gemini-3.1-Pro reaches 59.19 overall, compared with 92.20 for humans with prefix access and 86.61 for humans under streaming observation. These human protocols provide distinct reference points and should not be conflated. Table 2 · spatial understanding benchmark · arXiv v2
- On the L4 allocentric category, Gemini-3.1-Pro scores 54.90 and GPT-5.4 40.53; their overall scores are 59.19 and 50.89. The category-level results expose a world-centered reasoning gap beyond aggregate performance. Table 2 · allocentric spatial reasoning · arXiv v2
Cite this paper
Yifei Li, Pengyiang Liu, Yuhang Zang, Zhongyue Shi, Qi Fu, Hongye Hao, Jiwen Lu. OVO-S-Bench: A Hierarchical Benchmark for Streaming Spatial Intelligence in Multimodal LLMs. arXiv preprint arXiv:2606.03890, 2026.
@article{arxiv260603890,
title = {{OVO-S-Bench: A Hierarchical Benchmark for Streaming Spatial Intelligence in Multimodal LLMs}},
author = {Yifei Li and Pengyiang Liu and Yuhang Zang and Zhongyue Shi and Qi Fu and Hongye Hao and Jiwen Lu},
journal = {arXiv preprint arXiv:2606.03890},
year = {2026},
url = {https://arxiv.org/abs/2606.03890}
}