Research

Recognizing others is a major challenge for people with visual impairments (VIPs) and can hinder their social engagement. We present Accessibility Bot, a research prototype bot on Facebook Messenger, which leverages state-of-the-art computer vision algorithms and the existing set of tagged photos of a user’s friends on Facebook to help people with visually impairments recognize their friends. Accessibility Bot provides users information about the identity of friends in camera and their facial expressions and attributes. To guide our design, we interviewed eight VIPs to understand their challenges and needs in social activities. We then conducted a diary study with six VIPs to study the use of Accessibility Bot in everyday life. While most participants found the Bot helpful, their experience was undermined by perceived low recognition accuracy, difficulty aiming a camera, and lack of knowledge about the phone’s status. We discuss these real-world challenges, identify suitable use cases for Accessibility Bot, and distill design implications for future face recognition applications.