Compositions created using database of well-known pop, classical and jazz artists
A marimba-playing robot with four arms and eight sticks is writing and playing its own compositions in a lab at the Georgia Institute of Technology. The pieces are generated using artificial intelligence and deep learning.
Researchers fed the robot nearly 5,000 complete songs — from Beethoven to the Beatles to Lady Gaga to Miles Davis — and more than 2 million motifs, riffs and licks of music. Aside from giving the machine a seed, or the first four measures to use as a starting point, no humans are involved in either the composition or the performance of the music.
Ph.D. student Mason Bretan is the man behind the machine. He’s worked with Shimon for seven years, enabling it to “listen” to music played by humans and improvise over pre-composed chord progressions. Now Shimon is a solo composer for the first time, generating the melody and harmonic structure on its own.
“Once Shimon learns the four measures we provide, it creates its own sequence of concepts and composes its own piece,” said Bretan, who will receive his doctorate in music technology this summer at Georgia Tech. “Shimon’s compositions represent how music sounds and looks when a robot uses deep neural networks to learn everything it knows about music from millions of human-made segments.”
Bretan says this is the first time a robot has used deep learning to create music. And unlike its days of improvising, when it played monophonically, Shimon is able to play harmonies and chords. It’s also thinking much more like a human musician, focusing less on the next note, as it did before, and more on the overall structure of the composition.
“When we play or listen to music, we don’t think about the next note and only that next note,” said Bretan. “An artist has a bigger idea of what he or she is trying to achieve within the next few measures or later in the piece. Shimon is now coming up with higher-level musical semantics. Rather than thinking note by note, it has a larger idea of what it wants to play as a whole.”
Shimon was created by Bretan’s advisor, Gil Weinberg, director of Georgia Tech’s Center for Music Technology.
“This is a leap in Shimon’s musical quality because it’s using deep learning to create a more structured and coherent composition,” said Weinberg, a professor in the School of Music. “We want to explore whether robots could become musically creative and generate new music that we humans could find beautiful, inspiring and strange.”
Shimon will create more pieces in the future. As long as the researchers feed it a different seed, the robot will produce something different each time — music that the researchers can’t predict. In the first piece, Bretan fed Shimon a melody comprised of eighth notes. It received a sixteenth note melody the second time, which influenced it to generate faster note sequences.
Bretan acknowledges that he can’t pick out individual songs that Shimon is referencing. He is able to recognize classical chord progression and influences of artists, such as Mozart, for example.
“They sound like a fusion of jazz and classical,” said Bretan, who plays the keyboards and guitar in his free time. “I definitely hear more classical, especially in the harmony. But then I hear chromatic moving steps in the first piece — that’s definitely something you hear in jazz.”
Shimon’s debut as a solo composer was featured in a video clip in the Consumer Electronic Show (CES) keynote and will have its first live performance at the Aspen Ideas Festival at the end of June. It’s the latest project within Weinberg’s lab. He and his students have also created a robotic prosthesis for a drummer, a robotic third arm for all drummers, and an interactive robotic companion that plays music from a phone and dances to the beat.
The Latest on: Music robot
- Dance Music at the 2020 Grammys: A Billboard Dance Contributor Roundtableon January 23, 2020 at 12:24 pm
Billboard Dance contributors discuss the 2020 dance and electronic Grammy nominations and what they say about the state of the scene.
- Tech Daily: AAPL Bug, AMZN JEDI Motion, Music Subs, UK Digital Tax, Moreon January 23, 2020 at 12:13 pm
The top stories here are Apple's ITP vulnerability, Amazon's motion to stop work under the JEDI contract, Amazon's soaring music subs and the UK digital tax.
- India is sending a creepy robot lady to spaceon January 23, 2020 at 9:08 am
great works of music from across international cultures, and greetings in a variety of different languages. The idea is that encountering this collection would help aliens understand who we are and ...
- Bear Robotics, a company making robot waiters, just raised a $32 million round led by SoftBankon January 22, 2020 at 9:58 am
JH: Penny can speak and play sound, but it’s not conversational yet. It can say, ‘Please take your food,’ or play music while it’s moving. That’s where customers may want to personalize the robot for ...
- ‘The Crow’, ‘I, Robot’ & ‘Gods Of Egypt’ Director Alex Proyas Signs With Gershon January 21, 2020 at 5:07 pm
EXCLUSIVE: The Crow, I, Robot and Gods Of Egypt director Alex Proyas has signed with Gersh. Sydney-based Proyas, also well known for movies Dark City, Garage Days and Knowing, started out as a music ...
- Society for New Music presents ‘Minimalism & the Sound of Our Times’on January 21, 2020 at 7:36 am
Utilizing drums, gongs, mallet instruments and shakers, “Play the Robot” explores the “classic sound worlds” of wood, skin and metal. “I characterize it as true chamber music because there is constant ...
- The performance line-up for Lahore Music Meet 2020 revealed!on January 20, 2020 at 10:28 am
Gentle Robot, Madlock, Mahak Qayyum, Hassan Sheikh & Roshaan Sherwani, Towers, Maanu, Karakoram, Fake Shamans, Farheen Raza and Iqbal. The outdoor showcase will also feature Pepsi Battle of the Bands ...
- Holly Herndon hails robot collaboration on Proto albumon January 20, 2020 at 3:04 am
Mathew Dryhurst — which acted as a new type of robot songwriting collaborator. Spawn looks like a souped-up gaming computer, and was trained with Herndon’s own voice and others, as well as percussion ...
- Watch: Trailer debuts for Oklahoma-made family film 'The Adventures of A.R.I. My Robot Friend'on January 17, 2020 at 6:35 pm
Original titled "A.R.I.," the production used the Oklahoma Film Enhancement Rebate Program administered by Oklahoma Film + Music Office. With an almost entirely Oklahoma-based ... Jude Manley and ...
- Would a robot conductor change orchestra music?on January 17, 2020 at 5:38 pm
The Kunitachi College of Music in Tachikawa, Tokyo, and the University of Tokyo have launched a unique research project to determine whether a humanoid robot can conduct an orchestra and elicit a ...
via Google News and Bing News