Nels Numan, Gabriel Brostow, Suhyun Park, Simon Julier, Anthony Steed, Jessica Van Brummelen

Have you ever wondered how amazing outdoor augmented reality (AR) experiences are created? It’s not as simple as placing virtual objects on a map. Developers need to understand the real world, but they often work remotely. That’s why Niantic Labs has built CoCreatAR – a new system designed to bridge the gap between remote developers and the real world – and will be presenting a paper at this year’s Conference on Human Factors in Computing Systems (CHI).

When a developer is working with a location they either are not at or have not been to before, it can create a confidence gap. This challenge is something that virtually all developers face when building experiences for real locations. CoCreatAR looked to specifically address the scenario when a developer, working from their office, can directly collaborate in real-time with someone at the actual location. Their on-site partner can capture the environment using their phone, sharing live video, audio, and 3D scans of the space. This can help the developer see the site’s dynamic conditions, such as moving traffic, crowds, and lighting.

This real-time feedback is essential because pre-made maps can be outdated or incomplete, leading to AR elements that don’t fit properly into the real world. The synchronous collaboration explored in CoCreatAR allows developers to place virtual objects, see how they interact with the environment, and make instant adjustments with confidence. The person on the ground can even draw on the environment to highlight issues or user paths, enabling the creation of a far more comprehensive picture.

The learnings of this study could be implemented in future Niantic tools and services for developers. This kind of real time collaborative approach means less travel for developers, more accurate AR experiences, and a more engaging design process. Our study showed that developers felt more confident in their work and found the process more engaging, and we hope this will encourage further exploration of the world through AR.

For more, read the full paper at GitHub.