Coachella 2022 - Flume 

 Coachella is a cultural landmark, occupying a vaunted place within popular arts and culture as one of the most recognizable festivals in the world.

After a 2 year long hiatus, Coachella’s triumphant return in 2022 was a welcome signal to the world of live music’s unshakable position as a crucial part of the zeitgeist.

But rather than merely revert to “business as usual”, the Coachella organizers sought to extend the creative dimension of the festival beyond the confines of the physical world.

With the help of artists at Epic Games, &Pull and Six Degrees of Freedom, All of it Now added broadcast-quality augmented reality (AR) to their main stage YouTube livestream, heralding the arrival of a new era for the festival and defining a new gold standard for live festival broadcasts.

The scope for this project included injecting augmented reality elements into Flume’s mainstage performance, as well as infusing moments between performances with some extra surprise and delight in the form of AR interstitials.

AOIN Involvement:

While initially approached to provide technical integration services, All of it Now’s involvement quickly expanded to include the entirety of Coachella’s augmented reality content pipeline.

For Flume’s performance, All of it Now developed creative AR elements in collaboration with artists from Epic Games and &Pull as well as the artist’s creative team led by TrasK House’s John McGuire, with AOIN acting as the final integration point for these assets.

All of it Now also partnered with the artist’s creative team to provide visualization services and advise on aspects including camera placement, lens selection, system engineering and playback protocols to ensure faithful delivery of the show’s visual concept.

For technical deployment, AOIN composited and transmitted three camera feeds in real time to be incorporated into the show’s live broadcast. We accomplished this through the use of Stype’s camera tracking hardware and Stypeland plugin for Unreal 4.27.2. Stypeland is one of the most technically transparent AR compositing tools available, and allowed the team to use as much of the native Unreal Engine toolset as possible. AOIN further bolstered integration of AR elements in the show’s stage by utilizing custom 3D AR holdout tools developed in-house, including a custom plugin to read incoming timecode and dynamically adjust active sequences live.

Check out the Unreal Engine Spotlight

Technical Challenges:

One of the most significant challenges of the project was the both the creative and onsite integration timeline. The creative teams from Epic, &Pull and All of it Now had only a few weeks to develop creative for the artist performance, and only a few days for the onsite integration. This project required an immense level of planning, coordination and cooperation from multiple teams, with a small margin for error. 

To maintain cross-team alignment, Gitlab was used for version control in managing and seamlessly merging changes and updates into project files, allowing the artists to collaborate from across the globe. While on site, the AOIN team’s experience in integrating augmented reality into broadcast systems proved crucial for the rapid deployment and integration of the AR package.

Adding both reflections and refractions from the real world onto AR objects proved a significant technical challenge. For some of the festival's interstitial moments, the creative direction called for reflective objects that effectively acted as mirrors, reflecting the world around them, and colliding with the stage itself. The 3D AR holdouts tool also served as a collision mechanism for these 3D objects, creating an illusion of visual elements bouncing and interacting with the physical stage itself as well as other 3D objects in the scene.

The Result:

Coachella 2022 marked an important cultural, creative and technical achievement within the history of live events. This effort established a new standard for music festivals, heralding their safe return after a two-year hiatus with the use of brand new technologies and storytelling tools to elevate the experience for millions of worldwide audience members unable to attend in person.

The broadcast is truly what makes Coachella a global cultural event, connecting fans of music from across the globe to a single event at a single point in time. The ability to augment the broadcast with creative elements that would otherwise be impossible to build, install or develop adds an extra level of depth to the creative possibilities, expanding the vision of artists to tell their stories in new ways. 

Fans and brands alike responded positively to the event, with some viewers wondering if the AR elements were real inflatables or other manner of practical stage props, a testament to the quality of the assets. These elements were only visible via broadcast, and as such brought at-home viewers closer to the festival through an experience exclusively available to them on their devices.

Services Rendered
Unreal Previsualization
Content Creation
Content Integration and Optimization
AR System Design
AR playback via Stypeland
Tracked Camera Data Recording
Custom 3D AR Occlusion Tools

AOIN Team Credits
VP Supervisor - Berto Mora
Senior AR Engineer - Neil Carman
Lead UE4 Technical Artist - Jeffrey Hepburn
UE Operator - Brady Villadsen
UE Operator - Vishal Sharma
UE Artist - Yanjun Chen
UE Artist - Kevin Peter He
Production Manager - Nicole Plaza

Associated Parties
Coachella
Epic Games
Six Degrees
Flume
&Pull