Jamie Wynn, Zawar Qureshi, Jakub Powierza, Jamie Watson, and Mohamed Sayed

Imagine transforming your surroundings with just a few words. Niantic’s AR Research team has unveiled Morpheus, a groundbreaking method that uses text to transform 3D scenes captured with Gaussian Splatting, a cutting-edge way to represent the world in 3D, and will be presenting their findings at this year’s Computer Vision and Pattern Recognition Conference (CVPR).

Previously, while we could change the look of these 3D scenes, altering their shape was a real challenge. Morpheus changes all that. This innovative technique gives independent control over both the shape and colour of 3D objects using a new RGBD diffusion model. Want to turn your living room into a spooky haunted house or a serene Japanese garden? Just type in your vision, and Morpheus can bring it to life.

Key to Morpheus’s magic is a clever system that ensures the stylization looks consistent from every angle. This means as you explore a reimagined 3D world, the style remains uniform and immersive.

Extensive testing, including feedback from users, has shown that Morpheus produces stylisations that not only look fantastic but also closely match the user’s text prompt. This research pushes the boundaries of what’s possible with 3D and generative AI, offering exciting new possibilities for creating immersive and expressive digital experiences.

With Morpheus, the power to reshape and recolour virtual worlds is now at your fingertips! The full paper is available on the project’s webpage if you want to learn more.