There’s no doubt that Virtual Production (VP) is revolutionising the way we make movies, but where’s it headed next? We think the tech behind VP will have a huge impact beyond pure film production; it’s set to transform immersive experiences, museums, venues, theme parks and more. Below we set out some of the ways we see this happening.
What’s the big deal with VP anyway?
VP film studios today consist of huge three-dimensional LED stages called ‘volumes’ that wrap around actors, projecting real-time ‘virtual twin’ rendered backgrounds that mimic a real environment. The film camera is tracked to the motion of the backdrop, creating a compelling sense of spatial movement and realistic depth to the filmed shots.
Entire scenes are captured straight in-camera, avoiding the need for layers of VFX and post-production. It’s as if the actors are ‘on location’, inhabiting the scenes without ever leaving the studio. We can use the backdrop to recreate any place you can imagine, from ancient Rome to a futuristic city or even galaxies far, far away.
It’s a new magic ingredient for film content generation, removing many of the pain points that currently exist in the greenscreen film process. So quite rightly, it’s been a fast take up by the industry. As volume stage hire becomes more cost-effective and motion tracking sensor tech develops, VP has a secured future in the film business. There aren’t many productions happening now that don’t integrate at least some VP techniques. Large live-action productions like the Netflix original series ‘1899’ use VP Volumes to supercharge the filmmaking process, drastically reducing shoot times and costs.
Increasingly we are using VP techniques across our work at Squint. Our Empire State Building visitor experience transported actors back to 1920s New York to see what it was like when the groundbreaking skyscraper was constructed, and more recently, our film about Trojena fast-forwards a decade to place tourists of the future inside the extraordinary yet-to-be-built destination in the mountains of KSA.
Recap complete, onto our main point – where is this technology heading next?
# 1 Windows into other worlds
Rather than using the Volume as a way to generate film content, what if we use it as the heart of an immersive experience? What if the actors become the audience? What if we inhabit virtual environments, immersing ourselves deeper into compelling narratives?
By removing the camera, and placing an audience at the heart of the action, surrounded by the volume, we give them an experience which literally transports them into a film-like alternate reality. The fact it takes place in a physical space, like a gallery or attraction, also means that real interactions take place. It becomes a shared virtual experience.
At the ESB Visitor Attraction, we transported visitors back in time, projecting onto the ‘windows’ on all sides of the room; we essentially wrapped screens physically around the audience and virtually recreated panoramic views of 1920s Manhattan. In another gallery, aeroplanes dogfight around the building as King Kong suddenly appears at the window to peer into the room of visitors, surprising and terrifying simultaneously.
We make the audience feel like they are really there. The popularity of this kind of experience is indisputable; the Empire State Building Observatory was named the #1 attraction in the U.S. by Tripadvisor for the second consecutive year in the 2023 Travelers’ Choice, Best of the Best award.
2# – Bigger and Closer
Stepping up in scale from immersive rooms, huge purpose-built volume “experience” venues are popping up all over the world, places like MSG Sphere in Vegas, the Lightroom in London, and the New Murabba in Riyadh to name just a few.
What do these venues share in common?
They’re social spaces. They firmly exist in the physical world. They are destinations that people travel to and socialise at. They constitute an event, an excursion, a family outing, or a meet-up spot. They mark an occasion, an interaction. In our post-covid world, this physical interaction that had long been missed is now more than ever, valued and celebrated. We’re all yearning for a bit more IRL.
It will be within these types of venues where the power of virtual production really becomes astonishing, further blurring the boundaries between the virtual and the physical.
These immense, tech-ready spaces are exciting in terms of what they might become in the near future, even if they don’t know exactly what that will be yet. We’re building volumes filled with possibilities for technologies still in development.
They will allow for new and intimate ways of exploring places that are difficult or impossible to access; what would it be like to live in a city on the moon?
The transformative potential of immersive storytelling in attractions offers a compelling way to boost visitor engagement and connectivity. It enables the recreation of delicate or distant landscapes and incredibly lifelike historical landmarks for audiences to explore collectively. It will educate and mesmerise visitors in equal measure by granting access to places and spaces they could otherwise not reach, a magical experience that, crucially, is shared. One of the clearest current expressions of the trend is the ABBA Voyage show, in which the members of Abba are brought to life as virtual Avatars but within the context of a live gig.
3# – What do ABBA, Fortnite and Travis Scott have in common?
The Abba Voyage show places virtual avatars in the midst of an immersive VP backdrop to create a hybrid physical/virtual experience – it’s hard to tell what is real and what is not.
The band (hidden) are humans playing live, but the performers (original members of ABBA) are pre-recorded in 3D using a virtual production technique called volumetric video.
Volumetric video, combined with motion capture, plays a crucial role in driving new types of live experiences within virtual worlds. Entirely virtual shows like Travis Scott in Fortnight or Lil Nas in Roblox utilise capture technology where the artists don motion-tracking gear and perform live. Simultaneously, their avatars mirror their every move within the virtual space to gargantuan audiences nearing 50 million.
This ‘virtual twinning’ of people is being taken to unbelievable new heights through the development of hyper-realistic human avatars; check out Unreal’s Metahuman. You can create a hyperreal 3D rig of your face (or anyone else’s) using nothing but an iPhone.
The implications of this technology for video games, filmmaking and immersive experiences are phenomenal.
#4 – Give us some space!
Virtual Production also has huge implications for the world of spatial design.
It will bring about increasingly enhanced ways of engaging with locations and environments that haven’t yet been built. And invite more immersive ways to explore optionality in the design of those places.
We foresee that there will soon be interactive VP areas in architects’ and designers’ offices where you could explore and evaluate the design of projects, from an urban macro level down to eye-level views. In a non-professional context, you could walk into an interactive Volume in IKEA and see how a kitchen fits into your home on a 1:1 scale.
The potential here is nearly unfathomable, from visions of future smart cities and new mobility systems to ambitious architectural interventions or environmental objectives; these technologies can help us immerse in imagined places in ways we’d never thought we’d see.
Your imagination is the only limit
Like every good science fiction story, we invent the future by imagining the possibilities of what the future can be. We’re designing for a world of experiences that don’t exist — yet. But we know they’re just around the corner.
Plug generative AI into this story, and you quickly get to three-dimensional environments conjured up on a voice command. Your imagination is the only limit.
Envisioning and experiencing a world that doesn’t exist yet is what Squint has been doing for over 20 years now. We navigate through technologies to find innovative ways to connect people to big ideas through the engaging and the extraordinary, to work with pioneers in ushering in new realities, to help drive innovation, and to build a new future.