Meta Research

Future generations of virtual reality (VR) systems will incorporate multiple sensors for various capture, tracking, mapping, reconstruction, and other machine perception functions. In this paper, we provide examples of some tracking and mapping functions that illustrate the critical requirements and performance metrics. The sensor performance, form factor, power, and data bandwidth are the main challenges in a battery powered, always on VR devices. We further propose a new figure of merit that incorporates both sensor power consumption and SNR into a single parameter. To overcome bandwidth, compute, and power challenges, on-sensor signal processing and early information extraction are necessary. We expect stacking technology will be the key enabler of new sensor architectures, and innovations on both sensing and processing layers will deliver intelligent machine perception sensors for the future generations of VR devices.