Research informs everything we do at Facebook, from improving real-time augmented reality experiences to keeping people safe and secure on our platforms. In the realm of video infrastructure, the Facebook Video Quality and Research team works on technology challenges that come from implementing videos at a large scale.

Researchers in video infrastructure work on two fundamental and related problems: (1) How to improve video coding efficiency, or how to spend fewer bits to compress a given video at a certain quality, and (2) how to accurately measure video quality, or how to predict a viewer’s perception of video quality through automated algorithms. An important constraint in their work is that both of these tasks need to be performed with the highest compute efficiency possible, given the billion-scale of Facebook and Instagram videos.

We have a top team of engineers and researchers in video infrastructure, some coming from industry and some straight from PhD programs in the field of video processing at top research universities. To learn more about the Video Quality and Research team’s contributions to the field so far and where to find their research, we talked with Ioannis Katsavounidis, Research Scientist in video infrastructure at Facebook.

Video @Scale 2019 and SPIE 2020

With Katsavounidis’s quality keynote at the Video @Scale 2019 event, the team introduced the concept of compute-efficiency/compression-efficiency convex hull. This allows different encoders, using even different coding standards, such as AVC, VP9, and AV1, to be compared and prioritized for videos of varying popularity.

At the most recent SPIE Applications of Digital Image Processing conference, which took place online in August 2020, Ioannis invited researchers from industry and academia to contribute to a special session on energy-efficient video processing. There were a total of 18 papers, covering all aspects of video processing, including video quality metrics, video encoders, and software and hardware architectures. Among these special session papers were four contributions from two different teams at Facebook:

• Efficient measurement of quality at scale in Facebook video ecosystem by Shankar Regunathan, Haixiong Wang, Yun Zhang, Yu (Ryan) Liu, David Wolstencroft, Srinath Reddy, Cosmin Stejerean, Sonal Gandhi, Minchuan Chen, Pankaj Sethi, Amit Puntambekar, Mike Coward, and Ioannis Katsavounidis (Video Infra)
• Fast encoding parameter selection for convex hull video encoding by Nick Wu, Volodymyr Kondratenko, and Ioannis Katsavounidis (Video Infra)
• Cross-codec encoding optimizations for video transcoding by Gaurang Chaudhari, Hariharan Lalgudi, and Harikrishna Reddy (our sister team, HW ASIC)
• Hardware acceleration of video quality metrics by Deepa Palamadai Sundar, Visala Vaduganathan, and Cindy Chen (HW ASIC)

“All these papers, but also others presented at SPIE, represent major steps toward our goal to secure the highest possible video quality for Facebook videos within the constraints of our data center power and physical capacity,” says Katsavounidis. “Attendees reacted positively to the prerecorded video presentations of each paper, and we received a lot of engagement that still continues today.”

ICIP, Video@Scale, and more

The team is continuing to conduct research in this space and share results with the academic community. A recent example is the IEEE International Conference on Image Processing (ICIP) 2020. “We are proud to be Platinum sponsors of this flagship conference,” says Katsavounidis. “At the conference, we held an industry workshop on efficient video compression and quality measurement at Facebook, where we talked about the types of problems we are facing and the solutions we are seeking.” Watch the entire workshop below.

Katsavounidis and others also announced short updates and research highlights at the recent Video@Scale event on October 22 and will be following up on their energy-efficiency research during the 2021 Picture Coding Symposium. “I am a general co-chair, and the team is working to have a number of papers submitted to that conference, which we are excited about,” says Katsavounidis.

Staying updated

The Facebook Video Quality and Research team plans to continue engagement efforts at the 2021 Picture Coding Symposium, as well as at SPIE and ICIP 2021. All updates will be posted to our blog. For all the conferences that Facebook sponsors, be sure to check our Events page.