In February, Meta launched the Mephisto Dataset Collection Tool request for proposals (RFP). Today, we’re announcing the winners of this award.

Mephisto is a tool that Meta AI Research uses to standardize and codify best practices and infrastructure for research data collection and annotation. It also serves as a clear method to open-source and share our methodology, such that others can reproduce or expand upon our work. Now that we’ve hit our 1.0 release, we feel like the platform is mature and stable enough for general use. We are opening Mephisto up to the community to ensure that we’re building infrastructure that will be useful for your use cases. This effort is part of an ongoing commitment by the Fundamental AI Research (FAIR) Laboratory at Meta to generate impactful, open-sourced scientific contributions for the good of all.

The RFP attracted 13 proposals from 13 universities and institutions around the world. Thank you to everyone who took the time to submit a proposal, and congratulations to the winners.

Research award recipients

Principal investigators are listed first unless otherwise noted.

Curating a dataset for common logical fallacies in online conversations
Ting-Hao Huang (Pennsylvania State University)

Not one bit sexist: A competitive game against sexist language
Claudia Wagner, Dennis Assenmacher, Mattia Samory, Indira Sen (GESIS-Leibniz Institute for the Social Sciences)

AI-assisted soft segmentation of distant galaxies by citizen scientists
Chris Lintott (University of Oxford), Anna Scaife (University of Manchester), Michael Walmsley (University of Manchester)

Monitoring and modeling human annotator behaviors in Mephisto
Gianluca Demartini, Shazia Sadiq (University of Queensland)