Applications closed

Benchmarking City-Scale 3D Map Making with Mapillary Metropolis request for proposals


Next-generation, location-based computer vision (CV) applications like augmented reality or autonomous driving require robustly working CV algorithms. Robustness means that algorithms can cope with variability in input data like seasonal and weather-related appearance changes, low-quality data from cheap cameras, or data captured under suboptimal lighting conditions. Producing reliable predictions in such challenging data scenarios and for highly varying, city-scale environments makes a real impact for downstream applications.

To make this impact measurable, we are introducing a novel, city-scale data set called Mapillary Metropolis, available from August 31, 2021. This data set is designed with the goal of creating a completely novel and complex benchmarking paradigm for training and testing computer vision algorithms in the context of semantic 3D map making. Our new data set comprises multiple data modalities at a city-scale size, registered across different representations, and enriched with human- and machine-generated annotations for different object recognition and tracking tasks. These modalities include professional- and consumer-grade street-level images, aerial images, 3D point clouds from street-level LiDAR, aerial LiDAR, and image-based reconstruction (SfM and MVS), and CAD models. All data modalities are aligned based on manual correspondence annotations and ingestion of survey-grade ground control point data. Our data set is designed to take city-scale 3D semantic modeling to the next level by enabling researchers to study shortcomings in current methods including, but not limited to, object recognition and tracking, 3D modeling, depth estimation, relocalization, image retrieval, change detection, sensor-fusion, and so on.

We are soliciting high-quality, aspiring proposals leveraging our novel Metropolis data set to improve basic CV algorithms that use one or preferably multiple data modalities from our data set for improving semantic 3D building.

To foster further innovation in this area, and to deepen our collaboration with academia, Facebook is pleased to invite faculty to respond to this call for research proposals pertaining to the aforementioned topics. We anticipate issuing up to a total of five awards, each in the $50,000 to $100,000 range. Payment will be made to the proposer’s host university as an unrestricted gift, and researchers will be invited to join subsequent workshops and meet-ups.

Award Recipients

University of Tübingen, Germany

Andreas Geiger

Czech Technical University in Prague

Torsten Sattler

Stanford University

Jiajun Wu

Applications Are Currently CLosed

Application Timeline

Note: The deadline has been extended to Friday, August 27 at 5:00pm AOE.

Launch Date

July 27, 2021


August 27, 2021

Winners Announced

December 2021

Areas of Interest

Areas of interest include, but are not limited to, the following:

  • City-Scale 3D modeling from heterogeneous data sources
    • Image-based 3D modeling from different camera perspectives including street-level, sidewalk, egocentric, and aerial viewpoints
    • Learning of implicit 3D surface representations from images, (sparse) point clouds, and CAD model data
    • Change detection from an evolving data set
    • End-to-end image-based mapping
  • ML for object recognition, tracking, and dense labeling
    • Weakly, semi-, and self-supervised learning for 2D and 3D object recognition
    • Object tracking and re-identification
    • Depth estimation, semantic- and instance-specific segmentation from weak supervision
  • Image-based matching, relocalization, and retrieval
    • Localizing 6DOF of novel query images captured in adverse conditions
    • Image- and patch-based matching
    • Image retrieval, place and landmark recognition


Proposals should include

  • A summary of the project (1–2 pages) explaining the area of focus, a description of techniques, any relevant prior work, and a timeline with milestones and expected outcomes
  • A draft budget description (1 page), including an approximate cost of the award and explanation of how funds would be applied to Project
  • Curriculum vitae for all project participants
  • Organization details; this will include tax information and administrative contact details


  • Awards must comply with applicable U.S. and international laws, regulations, and policies.
  • Applicants must be current full-time faculty at an accredited academic institution that awards research degrees to PhD students.
  • Applicants must be the Principal Investigator on any resulting award.
  • Applicants may submit one proposal per solicitation.
  • Organizations must be a nonprofit or non-governmental organization with recognized legal status in their respective country (equal to 501(c)(3) status under the United States Internal Revenue Code).

Additional Information

Winners will be announced at the ICCV tutorial on Metropolis data set. In-person meetings (post-COVID) will happen in 2022.

You can learn more about the features of the Mapillary Metropolis data set here, starting at the 1:08:30 mark.

For questions related to this RFP, please email Peter Kontschieder

Frequently Asked Questions

Terms & Conditions

Please read these terms carefully before proceeding.

Facebook’s decisions will be final in all matters relating to Facebook RFP solicitations, including whether or not to grant an award and the interpretation of Facebook RFP Terms and Conditions. By submitting a proposal, applicants affirm that they have read and agree to these Terms and Conditions.

  • Facebook is authorized to evaluate proposals submitted under its RFPs, to consult with outside experts, as needed, in evaluating proposals, and to grant or deny awards using criteria determined by Facebook to be appropriate and at Facebook’s sole discretion. Facebook’s decisions will be final in all matters relating to its RFPs, and applicants agree not to challenge any such decisions.
  • Facebook will not be required to treat any part of a proposal as confidential or protected by copyright, and may use, edit, modify, copy, reproduce and distribute all or a portion of the proposal in any manner for the sole purposes of administering the Facebook RFP website and evaluating the contents of the proposal.
  • Personal data submitted with a proposal, including name, mailing address, phone number, and email address of the applicant and other named researchers in the proposal may be collected, processed, stored and otherwise used by Facebook for the purposes of administering Facebook’s RFP website, evaluating the contents of the proposal, and as otherwise provided under Facebook’s Privacy Policy.
  • Neither Facebook nor the applicant is obligated to enter into a business transaction as a result of the proposal submission. Facebook is under no obligation to review or consider the proposal.
  • Feedback provided in a proposal regarding Facebook products or services will not be treated as confidential or protected by copyright, and Facebook is free to use such feedback on an unrestricted basis with no compensation to the applicant. The submission of a proposal will not result in the transfer of ownership of any IP rights.
  • Applicants represent and warrant that they have authority to submit a proposal in connection with a Facebook RFP and to grant the rights set forth herein on behalf of their organization. All awards provided by Facebook in connection with this RFP shall be used only in accordance with applicable laws and shall not be used in any way, directly or indirectly, to facilitate any act that would constitute bribery or an illegal kickback, an illegal campaign contribution, or would otherwise violate any applicable anti-corruption or political activities law.
  • Awards granted in connection with RFP proposals will be subject to terms and conditions contained in the unrestricted gift agreement (or, in some cases, other mechanisms) pursuant to which the award funding will be provided. Applicants understand and acknowledge that they will need to agree to these terms and conditions to receive an award.