A new system combines video of a scene from multiple cameras to created 4D visualizations. This makes it possible to view the scene from a variety of angles, place people into new scenes or, in this case, remove someone from a scene.
Carnegie Mellon Approach Requires Neither Studio nor Specialized Cameras
Researchers at Carnegie Mellon University have demonstrated that they can combine iPhone videos shot “in the wild” by separate cameras to create 4D visualizations that allow viewers to watch action from various angles, or even erase people or objects that temporarily block sight lines.
Imagine a visualization of a wedding reception, where dancers can be seen from as many angles as there were cameras, and the tipsy guest who walked in front of the bridal party is nowhere to be seen.
The videos can be shot independently from variety of vantage points, as might occur at a wedding or birthday celebration, said Aayush Bansal, a Ph.D. student in CMU’s Robotics Institute. It also is possible to record actors in one setting and then insert them into another, he added.
Bansal and his colleagues worked around that limitation by using convolutional neural nets (CNNs), a type of deep learning program that has proven adept at analyzing visual data. They found that scene-specific CNNs could be used to compose different parts of the scene.
The CMU researchers demonstrated their method using up to 15 iPhones to capture a variety of scenes — dances, martial arts demonstrations and even flamingos at the National Aviary in Pittsburgh.
The Latest Updates from Bing News & Google News
Go deeper with Bing News on:
- Tech: New system combines smartphone videos to create 4D visualizationson July 1, 2020 at 11:00 am
Researchers at Carnegie Mellon University have demonstrated that they can combine iPhone videos shot “in the wild” by separate cameras to create 4D visualizations that allow viewers to watch action ...
- New system combines smartphone videos to create 4D visualizationson July 1, 2020 at 10:17 am
Researchers at Carnegie Mellon University have demonstrated that they can combine iPhone videos shot 'in the wild' by separate cameras to create 4D visualizations that allow viewers to watch action ...
- ‘Augmented reality’ comes to the job siteon June 29, 2020 at 5:00 pm
This led to intense study of the means and methods to solve the problem. We used 4D visualization and augmented reality tools to plan and communicate the work with the labor crews. While the 4D ...
- 3D ultrafast ultrasound quantifies coronary blood flowon June 23, 2020 at 2:54 am
Three-dimensional ultrafast Doppler coronary angiography enables quantitative imaging of coronary blood flow in a single heartbeat ...
- Barco showcases new presentation switcher for live events and attractionson June 23, 2020 at 2:40 am
Barco, provider of innovative visualization solutions, has launched a new presentation switcher to support live events and attractions.
Go deeper with Google Headlines on:
Go deeper with Bing News on:
Convolutional neural nets
- Facebook advances VR rendering quality with neural 4×4 supersamplingon July 1, 2020 at 11:51 am
A research team has come up with a way to teach computers how to transform low-res 3D renders into high-resolution versions suitable for live VR viewing.
- Xilinx Selects Mipsology Zebra Software to Accelerate Alveo U50 FPGAon June 24, 2020 at 1:14 pm
AI software innovator Mipsology today announced that its Zebra neural network accelerating software has been integrated into the latest build of Xilinx's Alveo U50 data center accelerator card, the ...
- Challenges In Building Smarter Systemson June 23, 2020 at 12:11 am
Until recently, almost all of the intelligence in AI systems was in the cloud. Now it is moving down to the edge. The problem is that AI algorithms are changing almost daily. So now we have these ...
- Hypertension, diabetes affect age prediction from retinal fundus imagingon June 16, 2020 at 9:04 am
A convolutional neural network model accurately and reliably predicted age and sex using retinal fundus images; however, accuracy dropped among patients with underlying systemic vascular-related ...
- Five steps for building and deploying a deep learning neural networkon June 14, 2020 at 5:00 pm
The neural network then figures out properties of each grade ... One of the oldest frameworks, Caffe2 has widely supported libraries for convolutional neural networks and computer vision applications ...