A global team of computer scientists, from Tsinghua and Beihang Universities in China, Harvard University in the US and IDC Herzliya in Israel, have developed “Write-A-Video”, a new tool that generates videos from themed text.
In a world of novice photographers and videographers, capturing a deluge of content via their smartphones and handheld devices, there is a need for an intelligent, easy-to-use tool for automating the creation of movies and video montages. To date, many quality videos still rely on professional frame-based editing tools to manipulate raw footage and produce a coherent video with a captivating storyline.
A global team of computer scientists, from Tsinghua and Beihang Universities in China, Harvard University in the US and IDC Herzliya in Israel, have developed “Write-A-Video”, a new tool that generates videos from themed text. Using words and text editing, the tool automatically determines which scenes or shots are chosen from a repository to illustrate the desired storyline. The tool enables novice users to produce quality video montages in a simple and user-friendly manner that doesn’t require professional video production and editing skills.
The team is set to present their work at ACM SIGGRAPH Asia, held Nov. 17 to 20 in Brisbane, Australia. SIGGRAPH Asia, now in its 12th year, attracts the most respected technical and creative people from around the world in computer graphics, animation, interactivity, gaming, and emerging technologies.
While existing video editing tools still demand knowledge in video processing and editing, the researchers’ new method allows novices to create videos that tell stories more naturally. Write-A-Video, say the researchers, allows users to create a video montage by simply editing the text that accompanies the video. For example, adding or deleting text, and moving sentences around convert to video-editing operations, such as finding corresponding shots, cutting and rearranging shots, and creating a final video montage result.
“Write-A-Video uses current advances in automatic video understanding and a unique user interface to allow more natural and simpler video creation,” says Professor Ariel Shamir, Dean of the Efi Arazi School of Computer Science at IDC Herzliya. “With our tool, the user provides input mostly in the form of editing of text. The tool automatically searches for semantically matching candidate shots from a video repository, and then uses an optimization method to assemble the video montage by cutting and reordering the shots automatically.”
“Write-A-Video also allows users to explore visual styles for each scene using cinematographic idioms generating, for example, faster or slower paced movies, less or more content movements, etc.” explains Dr. Miao Wang from Beihang University.
When selecting candidate shots from the video repository, the method also considers the aesthetic appeal of the shots, choosing those that are ideally lit, that are well focused and are not blurry or unstable. “At any point, the user can render the movie and preview the video montage result with an accompanying voice-over narration.” says Professor Shi-Min Hu from Tsinghua University.
The team’s research shows that intelligent digital tools combining the abilities of humans and algorithms together can assist users in the creative process. “Our work demonstrates the potential of automatic visual-semantic matching in idiom-based computational editing, offering an intelligent way to make video creation more accessible to non-professionals,” says Shamir.
For the study, the approach was tested on various pieces of themed text and video repositories, with quantitative evaluation and user studies. Users without any video editing experience could produce satisfactory videos using the Write-A-Video tool, sometimes even faster than professionals utilizing frame-based editing software. At SIGGRAPH Asia, the team will demonstrate the Write-A-Video application and showcase a variety of examples of text-to-video productions.
The Latest on: Write-A-Video
- Lady Gaga's '911' director wanted to use concept for a different projecton September 23, 2020 at 5:00 am
The best music video directors hear a song and they write a video for the song... "I'll wait for the right song to fit [my ideas]. But I loved the song. I knew what to do with it, but I usually ...
via Bing News
The Latest on: Videos from themed text
- NHL Draft 2020: 5 takeaways from Devils’ 3 1st-round picks | Dawson Mercer a Patrice Bergeron clone?on October 7, 2020 at 9:45 am
The New Jersey Devils selected three players in the first round of the NHL Draft: left wing Alexander Holtz at No. 7l, center Dawson Mercer at No. 18 and defenseman Shakir Mukhamadullin at No. 20.
- Jane Fonda joined by breast-pumping Katy Perry and Orlando Bloom to rally up voters in throwback exercise videoon October 6, 2020 at 7:31 am
Jane Fonda was back in her old gym gear as she headed up a star-studded workout themed video to encourage fans to register to vote in the upcoming US presidential election. Taking inspiration from her ...
- North Carolina Democrat seeking Senate seat apologizes over sex-themed textson October 3, 2020 at 11:30 am
The Democratic challenger for a U.S. Senate seat for North Carolina has issued a public apology after sending sexually themed text messages to a woman who is not his wife. “I have hurt my family, ...
- 5 big picture takeaways from the most unusual Stanley Cup Playoffson September 29, 2020 at 12:13 pm
With the very unusual and unique 2020 Stanley Cup Playoffs behind us, Justin Bourne presents five big-picture takeaways from the unforgettable tournament.
- Video games should cover mature and evocative themes – but they have to do it righton September 29, 2020 at 2:39 am
This article discusses ‘The Suicide Of Rachel Foster’, which includes themes of self-harm, suicide, suicide ideation and sexual abuse. It also contains unmarked spoilers for the game.] The Suicide Of ...
- Life far from sunny in Southern California where jobless hospitality workers face bleak futureon September 28, 2020 at 5:05 am
Autumn is coming to Southern California, where crowds of visitors normally would have been heading to arts festivals and celebrity events, and chef Nigel Henderson would have been making his living as ...
- VIDEO: IMO Secretary-General calls for sustainable shipping for a sustainable planeton September 24, 2020 at 9:48 am
In a World Maritime Day message today, International Maritime Organization Secretary-General Kitack Lim said that IMO’s theme for 2020, “Sustainable shipping for a sustainable planet,” couldn’t be mor ...
- There’s no stopping Kareena Kapoor’s birthday celebrations as actor grooves to Queen song in unseen video. Watchon September 22, 2020 at 10:09 pm
The text on the video reads, “40 and Fabulous ... Karisma shared a picture of Kareena cutting her Wonder Woman themed birthday cake, on Tuesday. “Precious Moments with our birthday girl ...
- Yes, live from New York: ‘Saturday Night Live’ to return Oct. 3on September 10, 2020 at 8:52 pm
“Saturday Night Live” teased the episode on Twitter with a video of the stage, text that reads “October ... special compilation of Mother’s Day-themed sketches for Mother’s Day on ...
- Stormzy pays tribute to Chadwick Boseman with Superheroes music videoon September 9, 2020 at 6:16 am
The end of the video features a black screen with white text reading ... Serena and Venus Williams and includes quotes from the theme tune to British children’s show Tracy Beaker.
via Bing News