August 17, 2022

Meta Research PhD Fellowship Spotlight: Resolving the Rashomon effect in machine learning

By: Meta Research

Each year, PhD students from around the world apply for the Meta PhD Fellowship, a program designed to encourage and support promising doctoral students who are engaged in innovative and relevant research related to computer science and engineering. Fellowship recipients receive tuition funding for up to two years to conduct their research at their respective universities, independently of Meta.

As a continuation of our Fellowship spotlight series, we’re highlighting Hsiang Hsu, a 2021 Meta PhD Fellow in applied statistics.

Hsiang is a PhD candidate in the Computer Science Department at Harvard University and is advised by Flavio P. Calmon. His research interests intersect with information theory and machine learning (ML), with applications in representation learning, privacy, fairness, and model multiplicity.

Hsiang is passionate about bringing the magic of ML to real-world applications. “To deploy machine learning in the real world, we need to understand it beyond just accuracy,” he says. “My goal is to make machine learning stable, reliable, and interpretable.” In his current research, Hsiang is working to resolve the Rashomon effect, in which models with competing performances make conflicting predictions on individual data samples. This phenomenon is named after the 1950 movie Rashômon, in which three people witness an event and share three different perspectives on what happened. Hsiang believes that, just like competing interpretations can obscure facts, the Rashomon effect can reduce the effectiveness of ML models.

“When people practice machine learning, there are random factors and classifiers that need to be trained,” Hsiang explains. He published a research paper validating that randomness exists across models in different fields, from banking to text predictions to audio detection. “One hospital may predict that a patient has cancer, while another hospital may predict they don’t,” he says. “People get hurt when this randomness is not accounted for. Right now, researchers and product developers aren’t looking at the individual-level impact of this randomness in machine learning, and I want to advocate for creating models with fairness.” To start, Hsiang recently proposed a new metric to quantify the Rashomon effect in classification problems and report the Rashomon effect in ML pipelines.

Hsiang’s aspiration to be part of an innovative research lab led him to Meta. “Not only does Meta focus on fair decision-making and privacy research, but it has data at scale to work with,” he says. “It’s one of the reasons I was drawn to this Fellowship and why I want to join Meta full-time once I complete my PhD. I’ve also enjoyed collaborating with many researchers I admire. I feel empowered to make a positive difference.”

For Fellowship candidates looking to drive impact, Hsiang says don’t give up. “I applied several times before being accepted into the Fellowship,” he says, smiling. “Don’t be discouraged if you don’t get it on your first try. This Fellowship gave me the freedom to work on what I wanted. Just be clear on your research, what you want to do, and how you’re going to achieve it.”

To learn about Hsiang and his research, visit his profile. For more information on award details and eligibility, visit the program page.