At CVPR 2024, Niantic researchers will present “Matching 2D Images in 3D: Metric Relative Pose from Metric Correspondences,” an oral presentation where we describe MicKey, a system to predict how two pictures taken from different locations relate to each other in 3D space.

Understanding precisely where someone is in the world is a foundational component of delivering augmented reality experiences, because when you know where someone is, you can deliver digital content that is relevant and precisely positioned.

Typically, when trying to figure out how two images are connected, existing approaches look for matching points in each image to predict the camera’s position for each picture. But these approaches don’t consider the scale of objects, making the predictions unreliable or requiring additional equipment to measure depth.

MicKey still looks for a point that matches in both photos, but understands exactly where these points are in real life, the location of each camera, and how far each camera was away from the other, without using any additional equipment. It achieved state-of-the-art performance on benchmarks such as Map-Free Relocalisation while requiring less supervision than competing approaches.

Check out the paper, our Github and code, and see you at CVPR:

Paper: https://arxiv.org/abs/2404.06337
Page: https://nianticlabs.github.io/mickey/
Code: https://github.com/nianticlabs/mickey