Sensors for Future VR Applications

International Image Sensor Workshop (IISW)

Abstract

Future generations of virtual reality (VR) systems will incorporate multiple sensors for various capture, tracking, mapping, reconstruction, and other machine perception functions. In this paper, we provide examples of some tracking and mapping functions that illustrate the critical requirements and performance metrics. The sensor performance, form factor, power, and data bandwidth are the main challenges in a battery powered, always on VR devices. We further propose a new figure of merit that incorporates both sensor power consumption and SNR into a single parameter. To overcome bandwidth, compute, and power challenges, on-sensor signal processing and early information extraction are necessary. We expect stacking technology will be the key enabler of new sensor architectures, and innovations on both sensing and processing layers will deliver intelligent machine perception sensors for the future generations of VR devices.

Latest Publications

Boosted Dense Retriever

Patrick Lewis, Barlas Oğuz, Wenhan Xiong, Fabio Petroni, Wen-tau Yih, Sebastian Riedel

NAACL - 2022