When science fiction heroes communicate, they don‘t use landlines or cell phones. The caller simply appears in virtual form in the middle of the room; full sized and three dimensional. For researchers at Fraunhofer, this vision is already within reach: At the trade fairs IFA in Berlin (Hall 11.1, Booth 3) and IBC in Amsterdam (Hall 8, Booth 8.B80) they will show a new technology technology called “3D Human Body Reconstruction”.
Scientists at the Fraunhofer Institute for Telecommunications, Heinrich Hertz Institute, HHI in Berlin have developed a method by which the realistic image of a person can be transmitted in a virtual world; and just like in science fiction movies, the image appears full sized and three dimensional. The image can be viewed from different directions and the viewer can even walk around it – just like in the movie. Until now, this was not possible; even virtual reality (VR) still has its limits. People can be represented by artificial three-dimensional models (so-called avatars) that can be seen when the viewer puts on VR data goggles. Nevertheless, these artificial figures do not have a lifelike appearance or natural movement. Another option is to play the video image of a person in frontal view in the VR data goggles. However, the viewer cannot walk around the image. As a result, the whole scene looks artificial as one moves through the virtual world. The person always turns his two-dimensional front to the viewer.
20 stereo cameras
In contrast, the HHI researchers have perfected the three-dimensional impression. To do so, they have developed a camera system that films the person. The core of this system is a stereo camera: Just as people do with their two eyes, the camera records the person with two lenses. This stereoscopic vision results in distances being estimated well, because both eyes look at an object from a slightly different angle. The result is a three-dimensional impression. Recording a person in detail from all directions takes more than one camera. “We are currently using more than 20 stereo cameras to map a human,” says Oliver Schreer, Head of the Research Group “Immersive Media & Communication” at HHI. Each camera only captures a part of the person. The challenge is to merge the individual camera images together so that a realistic overall picture is produced.
The system includes more than just the camera technology. The researchers have developed algorithms that can quickly extract depth information from the stereoscopic camera images. This is necessary in order to calculate the 3D form of a captured person. The computer calculates a virtual model of the human, which is then transferred into the virtual scene. The cameras perceive the surface shape with many details. In this way even small wrinkles, e.g. on the clothes of the person, can be shown. The model has a natural and realistic appearance.
“In developing these algorithms, special care has been taken to ensure they work efficiently and fast, so the movements of dialogue partners can very quickly be converted into a dynamic model,” Schreer says, since this is the only way that the movements will look natural. The images from a single camera pair can be processed in real time.
Showing every gesture
The fusing of the 3D information from the various camera images takes a few seconds. The illusion has already been perfected, though. The system transmits the three-dimensional dynamic model of a person rapidly in virtual reality. A person can move freely in a dedicated capture area. The virtual image portrays every gesture and movement realistically. “Our goal is that in the future a realistic image copy of a human is able to directly interact with the virtual world – for example, to let it grab virtual objects,” says Schreer.
In the future, the new camera system is planned to be used for other application fields too. For example, the researchers work on a virtual video conferencing application. It could as well be used for infotainment applications. Instead of a passive, frontal viewing experience, a television viewer could be directly involved in a movie scene by means of VR goggles. The viewer would not only see a three-dimensional image of the scene on the television, but could virtually walk around inside it, and, for example be a part of the adventures of his science fiction heroes.
An alternative to motion tracking
“We can also imagine installing the camera system at different locations in small studios,” says Schreer. “Film producers could use it to transfer the movement of actors into scenes more easily than ever before.” That has been very costly so far. In general, the actor’s movements are recorded using the motion tracking method. As part of this process, the face and body of the actor are marked with small dots. The computer tracks the movement of the points and transfers it to the computer-generated artificial image of the actor – for example, an action star jumping from skyscraper to skyscraper. However, with individual marker points, motion tracking methods can only detect movements and, especially, fine facial expressions very inaccurately or with very high technical effort. That means a lot of post processing for the computer-graphic artists until the scene looks more realistic. “With our camera system, however, our goal is to break down and represent a person and it’s movement in the future with much more details,” says Schreer.
The researchers are currently improving their camera system and the accompanying analysis software. Whether they will make both of these available as a service or license them out to production companies is not yet known.
Learn more: Beam me up to the video conference
The Latest on: 3D Human Body Reconstruction
via Google News
The Latest on: 3D Human Body Reconstruction
- UMN researchers 3D bio-print a model that could lead to improved anticancer drugs and treatments on February 11, 2019 at 1:37 pm
"This model is more consistent with what the body is like," said Panoskaltsis-Mortari, "and, therefore, studying the effects of drugs with human cells at ... by the NIBIB (#1R21EB022830 "3D Bioprintin... […]
- World's first full-body medical scanner generates astonishing 3D images on November 19, 2018 at 4:33 pm
The groundbreaking imaging device is almost 40 times faster than current PET scans and can capture a 3D picture of the entire human body in one instant scan ... especially once we got the reconstructi... […]
- Researchers Create 3D Model of Neanderthal Rib Cage on October 31, 2018 at 12:07 pm
Our close evolutionary cousins may have had a subtle, but somewhat different breathing mechanism compared to anatomically modern humans, according to a team of scientists who completed the first 3D vi... […]
- Study reconstructs Neandertal ribcage, offers new clues to ancient human anatomy on October 30, 2018 at 9:07 am
An international team of scientists has completed the first 3D virtual reconstruction of the ribcage of the most complete Neandertal skeleton unearthed to date, potentially shedding new light on how t... […]
- Researchers Compare Human Cadavers and 3D Printed Anatomical Models to Determine Print Accuracy on September 1, 2018 at 1:30 pm
Surgeons often turn to innovative technology, like 3D printed anatomical models, to get a closer, more detailed look inside a patient’s body ahead of complex procedures ... when it comes to matching t... […]
- Fraunhofer HHI receives AIS Technology Innovation Award 2018 for 3D Human Body Reconstruction on January 17, 2018 at 4:50 am
Volumetric video is regarded worldwide as the next important development step in the field of media production. Especially in the context of the extremely rapid development of the Virtual Reality (VR) ... […]
- Non-destructive 3D visualization of whole-body live insects on July 30, 2017 at 5:00 pm
We successfully employed CO 2 gas as an anesthetic to provide temporary paralysis to keep the insect fully immobilized during micro-CT scanning (similar to a human CT scan where the patient must remai... […]
- 3D printed facial reconstruction aids investigation into body found in Ohio on December 19, 2016 at 4:00 pm
A forensic artist with the Ohio Bureau of Criminal Investigation has made a partially 3D printed facial reconstruction ... team who worked on the reconstruction, creating the lifelike skull by copying ... […]
- Virtual Reality: 3D Human Body Reconstruction from Fraunhofer HHI digitizes Human Beings on August 31, 2016 at 1:35 am
Scientists at the Fraunhofer Institute for Telecommunications, Heinrich Hertz Institute, HHI have developed a method by which the realistic image of a person can be transmitted into a virtual world. T... […]
via Bing News