Today, we’re announcing the general availability of the first set of augmented reality tools from the Niantic Lightship platform, enabling developers across the world to realize their visions for augmented reality and the Real-World Metaverse.

The Niantic Lightship Augmented Reality Developer Kit, or ARDK, is now available for all AR developers around the world at Lightship.dev. To celebrate the launch, we’re sharing a glimpse of the earliest AR applications and demo experiences from global brand partners and developer studios from across the world.

We’re also announcing the formation of Niantic Ventures to invest in and partner with companies building the future of AR. With an initial $20 million fund, Niantic Ventures will invest in companies building applications that share our vision for the Real-World Metaverse and contribute to the global ecosystem we are building. To learn more about Niantic Ventures, go to Lightship.dev.

“Transforming humanity’s relationship with technology by merging the physical and virtual worlds will require the ideas and perspectives of as many people as possible,” said John Hanke, Founder and CEO of Niantic. “That’s why we’re so excited to open the vault of technology that powers our own games, so developers, creators and brands globally can build inclusive experiences that push the boundaries of what’s possible in AR.”

The Lightship Platform

The Lightship Platform is the foundation for Niantic’s products, built on years of experience developing and running titles ranging from Ingress to Pokémon GO and Pikmin Bloom.

As part of the Lightship Platform, the ARDK is a cross-platform software development kit that runs seamlessly on billions of Android and iOS devices around the world. It brings together the tools and technologies that power the top three AR features — Real-Time Mapping, Understanding and Sharing — so developers can create unique AR experiences that seamlessly utilize some or all of the ARDK tools. Over time, we plan to make more Lightship Platform features available to developers.

Real-Time Mapping: Meshing

With Lightship, any public environment in the world can become a new experience where people explore, gather and connect. A key to making that possible is the ability to map reality.

Relying on Niantic’s advancements in using smartphone camera sensors to build depth understanding, Meshing APIs in the ARDK create real-time 3D mesh maps, understanding the topography and surfaces around the user. A mesh persists throughout the app session, expanding and improving as the user moves.

With Occlusion APIs, virtual objects appear in the right depth order relative to real-world objects, so if you kick your virtual ball over a hedge, you continue to see the hedge, but no longer see the ball.

Building on years of research, Niantic’s depth technology automatically adapts to any Android or iOS device’s camera, from standard RGB sensors to the newest LiDAR-powered devices. The result is a dynamic 3D mesh map that enables developers to create more visually realistic AR layers, taking into account both location and contextual understanding of the unique space surrounding any user within the AR application.

Understanding: Semantic Segmentation

Semantic Segmentation APIs ensure digital objects will interact correctly with different surfaces in the real world, and appear, or not appear, where they realistically should.

Semantic Segmentation APIs in the ARDK use computer vision-based capabilities to instantly identify different elements of a real-world environment, such as ground, sky, water, buildings, and more. These elements inform the way virtual content will react within a real space.

Semantic Segmentation allows developers to focus on the experiences they want to create, while the ARDK does the hard work to dynamically understand the environment, bringing creations realistically to life.

Sharing: Multiplayer Experiences

Sharing is a critical component to making all of these AR experiences feel real. ARDK’s multiplayer APIs enable developers to easily create AR sessions supporting up to five players concurrently. This allows for more interactive and collaborative applications that feel realistic and seamless, keeping virtual content, players, and their interactions in sync, all in real time.

To keep the shared experiences fast, the ARDK is backed by a P2P networking stack and managed server, so developers don’t need to write code to support multiplayer experiences. Additionally, ARDK provides some lightweight multiplayer gaming functions, including a player lobby system, a synchronized clock, and session-persistent storage.

Visual Positioning System

Since the beginning of Niantic, we have wanted to build a 3D map of the world. Building this map is one of the grand challenges of augmented reality, and it’s key to making the real world come alive with information and interactivity.

Today, Niantic CEO John Hanke also showed a demonstration of the next step on our Lightship roadmap — the Niantic Visual Positioning System, or VPS.

Our VPS enables people to place virtual objects in a specific location so those objects can persist to be discovered by other people using the same application. Using live production code, we’ve already mapped thousands of locations, and our plan is to open VPS access to a select group of developers ahead of a full launch in select cities next year.

A Spotlight For New AR Experiences

Today, we’re also sharing early work from brands, studios, and developers from around the world who participated in the private beta of the ARDK. These early explorers are diverse and span industries, audiences, and countries. We truly appreciate that they joined us in the earliest days of opening the Lightship platform, and look forward to helping them – and everyone else who joins us on this journey – build amazing apps and experiences.

• “AR enables us to engage with Coachella fans in ways unlike ever before. At the festival, attendees will be able to experience the show in new, interactive ways. Globally, fans will be able to immerse themselves in Coachella no matter where they live. Our vision is a future where anyone in the world can experience Coachella from anywhere and AR tools from Niantic give us the unique ability to make that happen.” Sam Schoonover, Coachella Innovation

• “The Lightship platform is accelerating TRIPP’s ability to bring mindful multiplayer augmented reality experiences that transform the way you see the world around you. This exciting expansion of our product would not be possible without the Lightship platform. We are honored to be a launch partner and to have the opportunity with Niantic’s support to continue to innovate in ways that encourage people to take care of their mental and emotional well being.”
  Nanea Reeves, CEO & Co-Founder, TRIPP

• “By opening up the innovative format of PGA Jr. League to the world of augmented reality, we have the opportunity to deepen interest in the game and create connections between golfers and PGA Professionals.” Arjun Chowdri, PGA of America Chief Administrative and Innovation Officer

• “AR can adapt the real world to let us inhabit the emotional landscapes we imagine when we hear our favorite music. Context has always been part of the music listening experience but, AR will enable us to drive that association even further: imagine the sky changing colors when your favorite track comes on or hearing the bassline of a song when you walk near the place where you first heard it. The use cases for AR and music are limitless and we are so pleased to be on this journey with Niantic, a true innovator leading the way into the future.” Oana Ruxandra, Chief Digital Officer and EVP, Business Development, Warner Music Group

Apps in development using the Lightship ARDK shown during the Lightship keynote include:

Historic Royal Palaces: Historic Royal Palaces is transforming the moat around the Tower of London into a new landscape, filled with flowers, starting in 2022. The group teamed up with Preloaded developer studios in London to create an AR flower experience that everyone can see – at the Tower and around the world. This is just one of many examples of how AR can bring a dose of beauty, whimsy, and history into our lives, while helping historic institutions reach completely new audiences.

Shueisha: Manga published by the Japanese company Shueisha has been entertaining comic book fans for nearly 100 years. Their works have inspired countless animated films and television series, and now, with mobile devices and AR, the wonderful characters of “One Piece” and many other manga can exist in the real world as well. We are very excited to see developer studio T&S working with Shueisha’s creative world of manga to help it come alive in AR. The app will be available from Shueisha in 2022.

The PGA of America: Playing outdoors together is at the heart of what we love at Niantic, and our multiplayer tools are core to helping the PGA team and developer Trigger Global to create new shared-AR experiences. Through AR, PGA Jr. League brings something fresh and intriguing to generations of new players. This app will be available in 2022.

Coachella: For years, the Coachella team has wanted to have its beloved butterfly land on the Spectra tower, the seven-story rainbow walkway tower that made its debut at Coachella 2018. Thanks to Lightship, that’s about to become possible. Using semantic segmentation, the Coachella app will include an experience for fans to witness a large-scale, beautiful butterfly behaving as if it is truly part of the real world. The app will be available for Coachella 2022.

TRIPP: So much more than a meditation app, TRIPP is using Lightship to connect people across the world to a global wellness community. TRIPP is building an incredibly unique experience that reaches for the potential of a real-world metaverse to create a more connected, mindful, and better world. The app will be available in 2022.

Science Museum Group: The goal of making science more engaging and interactive outside of its walls led the Science Museum to Niantic Lightship. Using the app, the museum group will connect people to the science all around them through AR. The app will be available from May 2022.

Investing in the Future - Niantic Ventures

To accelerate the growth of unique AR applications and experiences using the Lightship Platform, we’re establishing Niantic Ventures to invest in companies that are helping to build the future of AR. Niantic Ventures will identify companies doing unique work to build the AR ecosystem, and will serve as a co-investor alongside other venture capital and angel investors. The model is optimized to accelerate the growth of promising companies using AR in new ways, and across industries. To learn more, please visit Lightship.dev.

Developer Friendly Business Model

The core APIs in the Lightship ARDK, including Semantic Segmentation and Meshing, are currently free to use for developers, no matter how many people use their apps. The Multiplayer API is free for apps with fewer than 50,000 monthly active users; a fee applies for apps with more than 50,000 monthly active users. Multiplayer APIs are free for the first six months of use, regardless of the the number of people using the app.

These terms apply to all developers who sign up for the Lightship ARDK before May 1, 2022.

For more details about pricing, please visit Lightship.dev.

Niantic Lightship Developer Conference 2022

We look forward to welcoming developers to join us in person in San Francisco in May 2022 for our first Niantic Lightship Developer Conference, where we’ll provide updates on the platform and share what’s next. Until then, please visit Lightship.dev, get access to the tools, and start building AR experiences that will delight, inform, and inspire people around the world.

–The Niantic Lightship team