Facebook Research Scientist Dr. Richard Szeliski was honored today at the International Conference on Computer Vision (ICCV) with the 2017 IEEE PAMI Distinguished Researcher Award. The honor is in recognition of Szeliski’s contributions to the field of computer vision over the past three decades both through major research results and in influencing and inspiring other research.

Szeliski is best known for his pioneering contributions at the intersection of computer graphics and computer vision, particularly his work in image-based modeling and rendering.

He started working on computer vision in the early 1980s as a PhD student in Carnegie Mellon, and has been advancing this work ever since.

Szeliski joined Facebook in the fall of 2015 as one of the founding members of the Computational Photography team that is creating innovative photo and video experiences based on cutting-edge computer vision and graphics technologies. His decades of work have set the foundation for many of today’s breakthroughs in augmented-reality (AR) and virtual-reality (VR) applications.

Before Facebook, he worked at Microsoft Research for 20 years where he managed the Interactive Visual Media group’s research in image-based modeling and rendering, computer vision, and computer graphics, with applications to 3D scene recovery, computational photography, video editing, and multimedia experiences.

Some examples of his research throughout his career include:

• Panoramic image stitching: This technology seamlessly combines images to create a single panorama image. “It’s very exciting to be at the forefront of technology that is now built into every camera and cell phone,” he says.
• Photosynth: A powerful set of tools for capturing and viewing the world in 3D by reconstructing 3D scenes from existing imagery. This enables interactive tours through personal photo collections, as well as Google’s Photo Tours of world landmarks. The software also constructs image-based 3D models to smoothly navigate tours of 3D objects and spaces.
• Casual 3D photography: extending the work on 3D model recovery to multi-layer photorealistic 3D models that can be viewed interactively on smartphones and in VR. This work will be presented at SIGGRAPH Asia in November.

Advancing the field through publishing and open collaborations has been a constant in his career, and he continues to be closely tied to academia as an affiliate professor at University of Washington. He wrote the Computer Vision: Algorithms and Applications textbook as an outgrowth of the classes he taught at University of Washington and Stanford University. “I wanted to capture what we’re teaching in class,” he says. “And I’m delighted when people thank me for writing the book.” He has published over 150 research papers in computer vision, computer graphics, neural nets, and numerical analysis.

This award is just the most recent of several significant awards Szeliski has received during his career. He was also elected to the National Academy of Engineering in 2015 and won the 2011 Computer Graphics Achievement Award at SIGGRAPH.

Throughout his career, Szeliski has been inspired by his love for photography and mathematics. “I am a visual thinker and have an engineering background. While I love doing research, I also like building things that everyone can use. I’m less interested in hypothetical questions than in making fast advances in what is important to people,” he says. “One of the things I’m most proud of at Facebook is how quickly we can take ideas and get them out to more people.”

Looking forward, he is excited about how photography is transforming into new kinds of experiences. People used to think of a photograph as a square image. Szeliski asks: “What more can we do?”  He sees that we can make the experience of photography so much richer—with layers, animations, and immersive environments. “I delight in how a photograph evokes emotion.” Imagine the possibilities for virtual reality in capturing nature and childhood memories.

One aspect of his current work with Facebook’s Computational Photography team is looking at how to make these immersive experiences more accessible to more people. A few examples of their research include:

• 360-degree photo stitching and viewing
• 360-degree video stabilization, which is a very important component to making video watchable in VR headsets
• Augmented reality: He’s most excited about being able to layer digital information onto video in real time. “Imagine having virtual creatures who can live on your table and hide behind your desk, kind of like Pokémon Go but much better.”

Szeliski is quick to share this honor and his career accomplishments with his colleagues in academia and the companies he has worked with. “I am most excited about the people I work with. They are some of the brightest and most creative people in the field. I feel delight and pride about the things we have created and the progress our community has made over the past three decades,” he says.

“We are thrilled for Rick to receive this well-deserved recognition from the research community. At Facebook, he is an invaluable technology leader who has both shipped delightful experiences for billions of people to enjoy in Facebook’s camera applications, and helped shape our culture of inspired risk and openness,” said Joaquin Quiñonero Candela, Director of Applied Machine Learning at Facebook.

We can’t wait to see what experiences he’ll create next.