What is Virtual Production?
Virtual production is a filmmaking technique that blends real-world filming with computer generated imagery (CGI). It allows filmmakers to shoot scenes that combine live-action with digital backgrounds and effects in real-time.
Virtual production makes use of LED screens, game engine software, motion capture, and other technologies to create immersive virtual environments on set. This allows actors to interact with CGI elements as if they were really there, providing more realistic performances.
Some key benefits of virtual production include:
-
Real-time visual effects – CGI backgrounds and effects can be rendered in real-time and composited with the live action. This saves huge amounts of time and money in post-production.
-
Immersive environments – Actors can see and interact with virtual environments and characters right on set, improving performances.
-
Dynamic camera angles – The virtual environment moves and adjusts perspective in real-time as the camera operator moves the camera. This unlocks creative freedom.
-
Iterative workflows – Shots can be changed and effects tweaked iteratively on the fly, saving time and money.
The Technology Behind Virtual Production
Several key technologies have enabled the rise of virtual production in filmmaking:
LED Screens
-
LED video walls provide high-resolution background visuals in real-time that actors can interact with. These are known as in-camera VFX.
-
Flexible LED panels can be configured into different shapes and sizes to surround environments and actors with visuals.
-
LED screens provide far higher brightness, contrast and resolution compared to older green screens. This improves realism.
Game Engines
-
Game engines like Unreal Engine are used to render detailed 3D virtual environments that update in real-time on the LED screens as the camera moves.
-
They provide realistic lighting, textures, physics and perspective adjustments in real-time.
-
Many effects like smoke, fire and explosions can be added dynamically.
Motion Capture
-
Motion capture technology is used to track the camera and actors movements and relay it to the game engine to update the virtual environment’s perspective and lighting accordingly.
-
Performers may also wear mocap suits to animate CGI characters and creatures they interact with.
Notable Examples
Some groundbreaking projects that have utilized virtual production:
-
The Mandalorian (2019-) – This Star Wars series was shot on cutting-edge LED stages called StageCraft. Entire detailed worlds were rendered in real-time for actors to interact with.
-
The Lion King (2019) – The realistic CG animals were partially created using VR and real-time rendering to allow the voice actors to interact with virtual environments and characters.
-
Avatar sequels (2022-) – James Cameron built massive underwater virtual production stages to blend underwater live-action with CG creatures and environments.
Benefits Over Green Screen
Virtual production provides major improvements over traditional chroma key green screen compositing:
-
More realistic lighting – Actors are lit naturally by the LED screens instead of harsh green screen lighting.
-
Real-time previews – Shots can be iterated on immediately instead of waiting for post-production.
-
Realistic environments – Actors can feel fully immersed in CG environments instead of just green walls.
-
Interactivity – Actors can interact with and influence CG elements like characters and effects.
The Future
-
More affordable LED screens and game engines are making virtual production accessible to more content creators.
-
Real-time rendering quality continues to improve rapidly, enabling more complex virtual worlds.
-
VR and AR tools may allow for collaborative real-time previsualization and scouting.
Virtual production represents an exciting shift towards in-camera VFX across film, TV, commercials and other fields. Blending live-action with CGI in-camera is revolutionizing visual effects and opening new creative possibilities. This emerging field will likely become a standard filmmaking process in the near future.
