Need a Soundtrack for Your YouTube Video? Ask an AI Composer

In a recent demo conducted over Zoom, I watched as the music composition app Dynascore transformed the entire emotional tenor of a short video multiple times in under a minute, all without altering a single frame of the visuals. What began in my short briefing as a very serious workout ad with a very serious soundtrack—something where you’d expect to see neon sweat pouring out of the athlete’s head behind a Gatorade logo—quickly changed in tone to something a bit funnier. The machine intelligence engine inside Dynascore swapped out the action film theme music for Beethoven’s somber Moonlight Sonata, suddenly transforming the video into a dark comedy.

A few taps of the mouse on the other end of the Zoom window, a few seconds of rendering, and I was watching the same video with a modern pop song now layered over it, equally form-fitting to the burly close-ups on screen. This time, the result felt more like a music video.

If you’ve ever produced a video, whether for a paying client or for your own YouTube channel, you’re probably drooling right now. All video producers know that stock music tracks—the prefabricated, often wordless jingles purchased from an online library—fit like a borrowed suit. But when the alternative has long been to spend countless hours editing down better-known songs to fit the rhythm and length of your video, many video editors have settled for the easy option of selecting a canned piece of audio off the virtual shelf.

Dynascore’s AI–powered software is able to take a composed piece of music, then slice it up automatically and “recompose” the song to fit whatever video you’re working with. Producers are able to try out new songs immediately. A user can also re-render any of hundreds of classics and house-composed tunes to perfectly fit the length and pace of the same video, either inside an Adobe Premiere Pro plug-in or via the company’s own app. Need a musical transition or an ending bump for a logo? No problem. Like the song but need it to build more slowly? Got it. Because it’s written by an AI composer and the music is either out of copyright or written in-house at Dynascore, each tune comes with a license that allows unlimited global use of the music.

“I think it’s incredible,” says professional videographer Joseph DiGiovanna, who spent months testing the software before its release. “The truth is, the playing field has 100 percent been leveled,” he says, “The kid with the cell phone is up for the same job at Vogue that I am.”

Socially Distanced Lemonade

Dynascore wouldn’t exist if 2020 had gone according to plan. Unfortunately for New York musician Greg Jarrett and for Yunus Saatchi, who was then a researcher at Uber AI Labs, people stopped taking ride-shares en masse around the same time Broadway theaters closed their doors. For the duo, and for many on the team that would help create Dynascore, life under Covid quickly metastasized to full-blown uncertainty.

Uber shuttered the industry-leading AI division Saatchi worked for, and the off-Broadway show that Jarrett was working on as a music director went on indefinite hiatus.

“We left for the weekend thinking we’d be back on Tuesday,” he says, “And then two weeks. And then four weeks. And then six months.”

Saatchi found new work quickly, thanks to a meeting with composer Peter Lerman during his time at Uber. Peter’s brother Howard is the tech entrepreneur who founded Yext, an AI-powered search platform. An idea for an AI-based music tool landed Saatchi a job as part of Wonder Inventions, a new company backed by the Yext founder.

They didn’t have much of a problem finding musicians to compose tunes; Peter had deep connections to the New York music scene, and folks like Jarret were in need of something to occupy their time.

With Jarrett and a small group of furloughed Broadway composers, arrangers, and orchestrators to write the music, Dynascore’s technical team stumbled into a best-case scenario when it came to in-house talent. Who better to write pieces that evoke visual drama than those who are already paid to do it nightly?

The Human Problem

Human composers’ golden ears are really the key to Dynascore’s ultimate success. The historic problem with AI-based, or “algorithmically composed” music, according to Saatchi, was that it mostly attempted to teach software instruments to write music from scratch, rather than reinterpreting pieces that had already been written.

“Making music that actually resonates with people is a human problem, so you have to start with the people,” Saatchi says. “AI is the supercharger for the humans.”

In the early days of Dynascore, Saatchi and the team worked to develop a way of breaking down original music and out-of-copyright classics (think Grieg’s “Hall of the Mountain King”) into segments they call “morphones.” They’d teach the AI a song, then ask it to recompose something similar using the original song’s morphones as a guide. Afterwards, they’d ask the musicians to critique the AI’s composition.

Getting music to fit perfectly in a video isn’t as simple as chopping up existing songs in predictable ways. Organic transitions require a more thorough understanding of key, rhythm, and intensity, among other musical markers. As such, morphones don’t just indicate the speed and key of a song. They also indicate various other tonal and musical characteristics, all of which allow the AI to know which types of musical Lego blocks snap together in which ways.

After they developed the morphone system, the team would feed the AI songs and have it recompose them. It took a while before it was musically literate enough to make good choices.

“The AI would compose a piece, and the musicians would go, ‘That was bad,’” Saatchi says, chuckling at the simplicity of the test. “It gets that feedback and it learns from it, and it gets to the point where, suddenly, it creates coherent compositions.”

The AI soon became smart enough to fit transitions, fades, breaks, and other user-dictated timing shifts into each tune it wrote. The version of Dynascore that I witnessed reshaping the Moonlight Sonata was born.

Take a Load Off

Dynascore’s achievement represents a dramatic improvement to what’s historically been a cumbersome workflow.

“When you’re working as an editor or filmmaker, you spend so much time with the music, because you have to make it fit frame by frame,” says DiGiovanna, who has worked on everything from feature films to TV ads, “With Dynascore you can do it on the fly.”

See a demo of how Dynascore handles scene transitions.

A tool that enables dynamic musical composition is particularly useful when working on commercial projects where certain items might need to be cut down the line. DiGiovanna gives an example of a director who needs a purse removed from an ad he created.

“You have to remove five seconds of video, and now the ending of the song doesn’t work, the transition to the next song doesn’t work,” he says, “That’s when Dynascore is going to save me a lot of time.”

The new software offers him a luxury that most new products don’t provide in such abundance: time. When asked how much it could realistically save him, he’s enthusiastic. “Hours, sometimes a whole day.”

A New Dynamic

In terms of the art itself, the most exciting development Dynascore brings to aspiring and established filmmakers is how many new musical options they can try for a given piece. From hip-hoppy beats to reinterpretations of out-of-copyright classical works, users can pick specific musical vibes to try, all without the same out-of-pocket costs they might have incurred otherwise.

In my initial experience with Dynascore, I got to see the differences in the results when several different songs were placed over the same video. From classical music to a hip-hop beat to a folk song, I was able to get three distinct versions of that workout ad rendered in front of me in minutes, each form-fitting to the video.

Previously, I would have had to select stock music, and the songs wouldn’t have been form-fitting. The bespoke nature of the sounds really makes each video feel more professional and less like a demo for a stock music website.

“It’s hard to believe how much a piece of music can change how you feel about what you’re watching,” says composer Jarrett, “You can make something, and it can be like, ‘Who knew it could be this too?’”

Buy Once, Use Anywhere

The fact that an artificial intelligence engine is assembling the music does raise some questions about how Dynascore’s internal composers will be paid if, for example, one of the songs they originally composed ends up being used for a Super Bowl commercial or in a trailer for a blockbuster film. Typically, big projects like these earn big payouts for composers. Dynascore’s leaders say they are committed to examining these issues as they arise. For now, bigger budget projects will probably continue to use human composers, but it’s not hard to see how AI could get more and more involved in videos like those that are showing up on Instagram, TikTok, and YouTube.

In fact, the unlimited global license is a key component to what may make the software a success, at least in terms of the platform agnosticism of the music it offers.

Having one single global license that you own, in perpetuity, means you can use the songs from Dynascore anywhere you want, whenever you want.

“Generally, if you’re dealing with American composers, and you’re dealing with songs that are in the public domain, you should be able to buy out all the necessary rights,” says industry expert and attorney Donald S. Passman, the author of All You Need to Know About the Music Business, one of the most iconic books about licensing.

In Tune

Saatchi has a clear idea of how good he wants the AI to be. “My metric as a listener is that you shouldn’t be able to tell whether a machine did it or it was in a studio,” he says. At least as far as I’ve seen and heard, Dynascore is meeting that standard.

Given the quality of the music and the plug-in’s affordability—it now costs $19 per month for unlimited tracks—it’s tough to imagine how Dynascore won’t gain ground among creators starved for a more bespoke music experience. The company has also released a developer API—a set of tools that will let third-party software developers use the music engine in their apps—making content creation even easier for those of us without an Adobe subscription.

Even DiGiovanna, who actually does have the musical chops to compose his own music for the videos he creates, sees Dynascore as a tool that undoes the creative limits of the medium. “It’s like heaven,” he says.