Introduction
Volumetric video capturing is an exciting new display technology that is poised to transform how we interact with visual content. In recent years, volumetric video has emerged from research labs as a practical method for capturing real-world scenes and people in three dimensions. While volumetric video is still in its early stages, it has the potential to enable incredibly immersive visual experiences in fields like entertainment, communication, education, and more.
In this article, I will provide an in-depth look at volumetric video capturing and how it works. I will cover the key benefits and use cases of volumetric video, current techniques for capture and display, major industry players, and the future outlook for this nascent display trend. My goal is to give readers a comprehensive understanding of the state of volumetric video today and where it may take us tomorrow.
What is Volumetric Video Capturing?
Volumetric video is a technique for capturing real-world scenes and objects as three-dimensional models that can be viewed from any angle. It goes beyond traditional video by capturing the full volume, depth, and texture of a subject. The result is photorealistic 3D media that can be manipulated and explored spatially.
While traditional video only captures the outward appearance of a subject from one viewpoint, volumetric video fully recreates it as a dimensional digital object. This allows users to change their perspective and examine the subject from all angles as if it were really there. Volumetric video is sometimes called “holographic video” or “holoportation” for its realistic 3D effects.
How Volumetric Video Capturing Works
Volumetric video capture typically relies on an array of synchronized cameras and depth sensors to scan subjects from multiple viewpoints. There are a few main approaches used today:
-
Multi-camera rigs – These rigs contain upward of 100 calibrated cameras to surround the subject and capture views from all angles. Advanced computer vision techniques are used to stitch the imagery into a 3D model.
-
Depth sensor arrays – Sensors like LiDAR can accurately measure the depth and surface details of subjects in a scene. This depth data gets integrated with camera textures.
-
AI-powered photogrammetry – With machine learning algorithms, 3D models can be constructed from standard 2D photos and videos. This allows volumetric capture using just a smartphone camera in some cases.
The full process involves capturing footage from varied perspectives, processing the data into a unified 3D scene representation, and then encoding this model for storage, transmission, and rendering. Advanced compression algorithms shrink the large 3D video files for practical use.
Key Benefits and Use Cases
Some major benefits and emerging use cases for volumetric video include:
-
Immersive entertainment – Volumetric video will enable holographic displays and immersive media experiences including interactive storytelling, virtual concerts, holographic cinema, VR/AR, and more.
-
Presence sharing – Capturing people volumetrically allows realistic telepresence for remote communication, collaboration, or family interactions over distances.
-
Digital human preservation – Organizations are exploring uses for archiving important figures, actors, and public leaders using volumetric video.
-
Medical imaging – Capturing anatomical scans volumetrically can enable detailed analysis and 3D visualizations for improved medical diagnosis.
-
Athlete tracking – Sports analytics leveraging volumetric video of motion and biomechanics can provide coaches/trainers new performance insights.
-
Education and training – Volumetric lessons, demonstrations, and simulations can immerse students in learning experiences unachievable with 2D video.
-
Digital fashion – For online fashion shopping, volumetric video lets customers realistically visualize clothing on personalized 3D avatars.
-
Gaming and VR – 3D-captured scenes, objects, and humans can massively enhance realism for next-gen games and virtual experiences.
Current Industry Landscape
While volumetric video is still maturing, some companies making progress include:
-
Microsoft – With Azure Depth Video Camera APIs, Kinect depth sensors, and Mixed Reality Capture studios, Microsoft is a leader in democratizing volumetric video.
-
Google – Google’s Light Field Lab and Welcome to Light Fields project explore realistic holographic effects via volumetric capture.
-
Intel – Intel’s depth-sensing RealSense cameras and software support simplified 3D scanning for volumetric video uses.
-
Volumetric Video – This startup offers end-to-end volumetric capture studios and displays for telepresence and other applications.
-
8i – 8i develops proprietary Holo human hologram technology using volumetric video capture.
-
Facebook – Facebook’s Reality Lab research on VR, AR and realistic human telepresence involves volumetric video capture.
-
Raytrix – Raytrix offers 3D light field cameras used in professional volumetric video rigs with dozens of sensors.
In addition, various research labs and startups continue pioneering new approaches to volumetric capture and reconstruction using AI-powered photogrammetry, single lens solutions, advanced depth sensors, and more.
The Future Outlook
Volumetric video still faces some technical hurdles on the capture end like expensive multi-camera rigs required, processing massive datasets, and seamlessly compressing 3D video. However, rapid improvements in camera miniaturization, computer vision, depth sensing, display quality, and 5G connectivity point to volumetric technology heading towards mainstream adoption within 5-10 years.
I believe three key developments will shape volumetric video’s future:
-
Cheaper, smarter capture – Smaller, faster sensors plus AI will enable high-quality volumetric sequences from simpler capture setups.
-
New display devices – Holographic displays, immersive glasses, and other spatial viewing devices will arrive for both consumers and enterprises.
-
Enhanced 5G and WiFi – Ultra-fast 5G and WiFi 6/7 networks will enable streaming of live volumetric video feeds to devices.
As volumetric video transitions from visual novelty to mass market practicality, it promises to profoundly expand our capabilities for digitally representing, communicating, and interacting with both real and virtual worlds. Its emergence underscores an accelerating wave of spatial computing that seeks to break visuals free from flat screens into surrounding space. Volumetric video will likely unlock new applications we can’t yet envision, just as text, images, video and 3D graphics expanded human expression and understanding when first introduced. We are only glimpsing the start of this exciting new display trend.
