Visualization And Engineering Design Graphics With Augmented Reality App

When I asked Donald Glover (the artist currently known as Childish Gambino) what his new song "Algorythm" was about and why he wanted to create an augmented reality app around it, his reply was succinct: "Trust the Algorythm."

So I did. I launched the app, held my iPhone in front of my face, and walked through a portal into Glover's world. A cave scene began to form around me. I heard the first part of a long, extended mix of the (yet-to-be-released) Algorythm song, just beats and cave sounds (water dripping, rocks moving). I was instructed to search around the cave walls to find the "glyphs"—which when focused on, come to life and are beamed down to the cave floor. The people dance. The animals run around. Soon Childish's own avatar shows up and gets its groove on.

After I found all the glyphs, the cave disintegrated around me and I found myself floating with the cave figures in outer space. Then the song gets going in earnest, and I heard Childish sing "Everybody–move your body . . ." A wide beam of light stretches between one planet and another. The designers throw in the "monolith" motif from Kubrick's 2001, along with some visual nods to the movie's "star gate" sequence.

I asked Glover why he chose those two settings—the cave and outer space—for the AR experience. "The journey from cave to space feels like the story of humanity," he replied. Perfectly right.

From live show to AR

The visual themes, Glover explained, grew from a 2018 Childish Gambino show under a big dome at the artist's PHAROS festival in Tapapakanga Regional Park in New Zealand. Immersive 3D imagery was projected on the top of the dome over the heads of the audience. "The app is an evolution of that idea," Glover said.

Actually, the Pharos experience became a VR experience first, back in 2017. But Glover became interested in the possibilities of AR for conveying the creative world around the music. "Augmented reality removes the barrier between the physical and digital world," Glover said.

The Pharos AR app actually doesn't enable a lot of interaction between the digital content and the real world content. It does contain some interactive features—you can tap on the glyphs in the cave to activate them, for example. But the slow blending of the real world around you (as seen through the phone camera) into the world of the cave is compelling. It feels more like the gradual onset of a hallucinatory drug as opposed to the immediate occlusion and immersion you get when entering a VR experience.

Pharos AR was designed by the creative agency MediaMonks with direction from Childish Gambino and his creative management team using Unity's 3D engine. The movements of the human dancers in the cave derive from the movements of humans wearing motion capture (mo-cap) suites. The data collected from the sensors on the suits are then fed into the Unity engine where they're used to animate the avatars. The people at Unity pitched in with technical assistance.

Now Unity is out with a behind-the-scenes video showing the people and technology behind Pharos AR. The company will be showing the video this week at its beachfront Cabana at Cannes Lions. The goal, Unity says, is to show brands and creative agencies how they can use AR to engage with their audiences in a new way.

MediaMonks creative director Eric Wagliardo said Unity's deep integration with Google's ARCore framework was one of the key reasons his firm wanted to use the Unity engine. That integration allowed the app creators to build cloud anchors, which allow digital objects (like the portal into the cave) to be associated with places in the real world, explains Unity tech producer Sarah Stumbo. Those associations let users find digital objects in the same real-world place every time they point their camera there. In the app's multiplayer mode the anchors allow all the players to find and interact with objects at specific places in the real world.

The Pharos AR app already has more than 50,000 downloads at the App Store, Unity says, with a 4.9 out of 5 rating.

Some brands have begun experimenting with augmented reality to communicate their aesthetic to customers. But most of the apps we've seen to date have focused on utility. Case in point is the Ikea AR app, which lets people place digital representations of Ikea products like chairs or tables within their real-life living rooms before buying.

AR apps from brands will become more common, especially after the hardware used to experience them is no longer the smartphone, but rather a future pair of AR glasses. Childish Gambino used AR to put fans inside his creative universe. Brands may well want to do something similar for their customers and followers.

"AR technology is going to play a huge part of everyday life in the future," Glover told me. "The idea for the app was to give people a small taste of the future . . ."