Gorillaz & Bad Bunny AR at Coachella 2023

After augmented reality’s (AR) incredible success during Coachella’s 2022 livestream, All of it Now made a triumphant return in 2023. This year, the creative scope expanded to multiple headlining artists including Bad Bunny, Blackpink, and Gorillaz as well as festival bumpers for Coachella’s innovation team.

AOIN Involvement:

AOIN worked alongside multiple creative studios on this project, acting as technical consultant and creative integrator in collaborating with each musical artist’s creative team. As a team experienced in both live stage production and virtual production, AOIN was uniquely positioned to integrate stage production workflows into the live stream.

In order to work with so many different teams on this project, AOIN developed a custom content onboarding and ingest pipeline along with concise project documentation that outlined the creative and technical features of broadcast augmented reality as a platform. This effort resulted in a unified project file structure for each artist while maintaining confidentiality between each creative team.

AOIN worked with Nexus and Swear for Gorillaz, Sturdy and Aura for Bad Bunny, along with the Forward Team for Blackpink. Nexus was also involved in the interstitial content, with AOIN developing additional creative elements for the “Mirage” effect as well as creative sequencing.

Technical Challenges:

Coachella is a place to showcase the apex of entertainment technology on a global scale by pushing the boundaries of possibility on a technical and creative level. This ethos extends into the main stage broadcast, creating a new platform for artists and the Coachella brand alike.

A lynchpin of Coachella's 2023 marketing campaign was an interstitial element which came to be known as the "mirage effect". This effect refracted real-world objects within digital elements, a historically difficult outcome to achieve within Unreal Engine's real-time compositing pipeline. AOIN accomplished this feat by working directly with stYpe's team to develop a custom compositing solution.

AOIN constantly works to maximize time spent on site for creative endeavors by minimizing technical challenges. At this year’s edition of Coachella, one such hurdle to success was creating a single environment wherein the work of multiple technical artist teams - each with their own workflows and creative methods - could be operated from a single control schema by our operators. We achieved this through custom Unreal Engine editor scripting and deploying tools like Stage Precision to switch between scenes, monitor engine health and manage operations from a single source. As a result, our work enabled six engines to run concurrently, seamlessly integrating into the rest of the festival’s tech stack.

.

For this show, AOIN developed a universal playback system that could be applied to content provided by all the teams with their various control methods. Certain artists were using timecode and needed an automated playback method that would sync to their show live, other artists did not utilize timecode and required their sequences to be manually triggered, and yet other content was heavily physics and gameplay driven and required direct communication to the actors in the scene to control playback. All of these were brought together under a simple unified system, with Stage Precision used as a control surface. The benefits of Stage Precision were its ease to develop interfaces, macros, and the ability to selectively target which machines to control if we needed to isolate certain machines for camera blocking and calibration. 

.

The Result:

Coachella’s 2023 livestream proved that broadcast-grade augmented reality for large scale music festivals is truly scalable, with AOIN helping to develop an inclusive creative platform for artists at the highest level. AOIN recorded the raw camera plates as well as the camera tracking data, giving future creative teams crucial material for testing additional creative concepts for years to come. This data enables reliable, repeatable camera moves that reflect a truly accurate representation of the live stream environment, and is extremely useful as a pre-visualization tool for creative teams using augmented reality for the first time.

For the first time ever in the history of the festival, multiple artists were given an opportunity to use broadcast-grade augmented reality to enable a new creative dimension of their performance for millions of viewers from around the world.

Services Rendered
Unreal Previsualization
Content Creation
Content Integration and Optimization
AR System Design
AR playback via Stypeland
Tracked Camera Data Recording
Custom 3D AR Occlusion Tools

AOIN Team Credits                              
AR Supervisor - Berto Mora
AR Producer - Nicole Plaza
Technical Director - Neil Carman
Lead UE5 Technical Artist - Jeffrey Hepburn
UE Operator - Vishal Sharma
System Technician - Mykel Booker

Associated Parties
Coachella
Epic Games
Six Degrees
Forward
Sturdy
Swear Studio
Nexus Studios
Aura Studios