New video editing technique could save time and reduce costs for the film industry.
The technique was developed by an international team led by a group from the Max Planck Institute for Informatics and including researchers from the University of Bath, Technicolor, TU Munich and Stanford University. The work, called Deep Video Portraits, will be presented for the first time at the SIGGRAPH 2018 conference in Vancouver on 16th August.
Unlike previous methods that are focused on movements of the face interior only, Deep Video Portraits can also animate the whole face including eyes, eyebrows, and head position in videos, using controls known from computer graphics face animation. It can even synthesise a plausible static video background if the head is moved around.
Hyeongwoo Kim from the Max Planck Institute for Informatics explains:
It works by using model-based 3D face performance capture to record the detailed movements of the eyebrows, mouth, nose, and head position of the dubbing actor in a video. It then transposes these movements onto the ‘target’ actor in the film to accurately sync the lips and facial movements with the new audio.
The research is currently at the proof-of-concept stage and is yet to work at real time, however the researchers anticipate the approach could make a real difference to the visual entertainment industry.
Professor Christian Theobalt, from the Max Planck Institute for Informatics, said:
Despite extensive post-production manipulation, dubbing films into foreign languages always presents a mismatch between the actor on screen and the dubbed voice.
Our new Deep Video Portrait approach enables us to modify the appearance of a target actor by transferring head pose, facial expressions, and eye motion with a high level of realism.
Co-author of the paper, Dr Christian Richardt, from the University of Bath’s motion capture research centre CAMERA, adds:
This technique could also be used for post-production in the film industry where computer graphics editing of faces is already widely used in today’s feature films.
A great example is ‘The Curious Case of Benjamin Button’ where the face of Brad Pitt was replaced with a modified computer graphics version in nearly every frame of the movie. This work remains a very time-consuming process, often requiring many weeks of work by trained artists.
Deep Video Portraits shows how such a visual effect could be created with less effort in the future. With our approach even the positioning of an actor’s head and their facial expression could be easily edited to change camera angles or subtly change the framing of a scene to tell the story better.
In addition, this new approach can also be used in other applications, which the authors show on their project website, for instance in video and VR teleconferencing, where it can be used to correct gaze and head pose such that a more natural conversation setting is achieved. The software enables many new creative applications in visual media production, but the authors are also aware of the potential of misuse of modern video editing technology.
Dr Michael Zollhöfer, from Stanford University, explains:
The media industry has been touching up photos with photo-editing software for many years, meaning most of us have learned to take what we see in photos with a pinch of salt. With ever improving video editing technology, we must also start being more critical about the video content we consume every day, especially if there is no proof of origin. We believe that the field of digital forensics should and will receive a lot more attention in the future to develop approaches that can automatically prove the authenticity of a video clip. This will lead to ever better approaches that can spot such modifications even if we humans might not be able to spot them with our own eyes.
To address this, the research team is using the same technology to develop in tandem neural networks trained to detect synthetically generated or edited video at high precision to make it easier to spot forgeries. The authors have no plans to make the software publicly available but state that any software implementing the many creative use cases should include watermarking schemes to clearly mark modifications.
The Latest on: Artificial intelligence
via Google News
The Latest on: Artificial intelligence
- LexinFintech CEO Speaks at World Economic Forum on Artificial Intelligence on September 19, 2018 at 2:03 am
TIANJIN, China, Sept. 19, 2018 /PRNewswire/ -- LexinFintech Holdings Ltd. ("Lexin") (LX), a leading online consumer finance platform for educated young adults in China, attended the World Economic ... […]
- Artificial Intelligence and its Dramatic Effect on Our Lives [Video] on September 19, 2018 at 12:22 am
Jeremy Walker is an entrepreneur and software developer who specializes in building businesses that utilize artificial intelligence in the advancement of medicine and education. Jeremy Walker is ... […]
- Key weapon for closing IoT-era cybersecurity gaps? Artificial intelligence on September 18, 2018 at 11:00 pm
As businesses struggle to combat increasingly sophisticated cybersecurity attacks, the severity of which is exacerbated by both the vanishing IT perimeters in today’s mobile and IoT era, and an acute ... […]
- IBM Debuts Tools to Help Prevent Bias In Artificial Intelligence on September 18, 2018 at 9:02 pm
IBM wants to help companies mitigate the chances that their artificial intelligence technologies unintentionally discriminate against certain groups like women and minorities. The technology giant ... […]
- Do the Benefits of Artificial Intelligence Outweigh the Risks? on September 18, 2018 at 12:52 pm
Towards the end of the second world war, a group of scientists in America working to develop an atomic bomb for the Manhattan Project warned that using the weapon would inevitably lead to a ... […]
- UB launches Artificial Intelligence Institute; hires inaugural director on September 18, 2018 at 11:34 am
The University at Buffalo is launching a new institute dedicated to the study of artificial intelligence. UB's Artificial Intelligence Institute (or UBuffalo.AI) will "explore how to combine machines’ ... […]
- You Might Want Artificial Intelligence Reading Your Next Mammogram on September 18, 2018 at 7:36 am
These are perhaps the most powerful and important four words a woman can hear after a breast-screening visit. X-ray based mammography is an effective screening tool for detecting cancer, but what many ... […]
- Artificial intelligence can transform the economy on September 18, 2018 at 6:39 am
Erik Brynjolfsson is the director of the MIT Initiative on the Digital Economy and co-author, with Andrew McAfee, of “Machine/Platform/Crowd.” Xiang Hui is an assistant professor of marketing at Washi... […]
- Artificial Intelligence To Create 58 Million New Jobs By 2022, Says Report on September 18, 2018 at 5:05 am
Machines and algorithms in the workplace are expected to create 133 million new roles, but cause 75 million jobs to be displaced by 2022 according to a new report from the World Economic Forum (WEF) c... […]
via Bing News