Machine Learning and Cover Songs
There’s nothing like a great cover.
You’re rekindling angst at a Pearl Jam show and without any warning they go right into a Beatles song. You recognize some David Bowie lyrics on Spotify, and you discover it’s an unrecognizable version of Let’s Dance by M. Ward. You listen to Tiny Cities by Sun Kil Moon several times before you even realize it’s an entire album of beautifully fermented Modest Mouse songs.
How often have you thought to yourself, I would love to hear this person sing this other band’s song in their own style? For instance, I wish I could listen to Mike Doughty sing just about anything.
Over the past year or two, we’ve started to see artificial intelligence begin to approximate that dream (or nightmare, depending on your perspective). First it was eye-opening deep fake videos of past presidents appearing to say things they never said, but now it’s moved on to much more creative and cool endeavors like OpenAI Jukebox. You should read the full description on the site, but essentially they are training models to identify everything that goes into a song: instruments, lyrics, musical style, and a whole lot more. The models are primitive for now, but even at this early stage, they can start recombining things in interesting ways like having Ella Fitzgerald sing a Prince song but in the style of folk rock.
I spent a good part of the weekend messing around in Jukebox, and it’s mesmerizing. It really feels like the beginning of something big, and just as excitingly, something that could get orders of magnitude better within only a few years.
When you listen to it, it almost feels like the first words of a child… or if you prefer, the first song from Jimmy Page.
A lot of the stuff in the library is pretty rough, but here are some of the most interesting ones I found:
- Frank Sinatra introducing Jukebox
- The Greatest Love of All by Whitney Houston re-arranged and sung as Bob Dylan
- U Can’t Touch This by M.C. Hammer re-arranged and sung as Earth, Wind, and Fire
- Stopping by Woods on a Snowy Evening. A poem by Robert Frost. As read by Dolly Parton, Aretha Franklin, Neil Diamond, or Billie Holiday.
- Some of them aren’t so much “good” as they are fascinating, like this attempt at Brain Damage by Pink Floyd which sounds like a singer who doesn’t know English trying to just sound out the words as best he can.
- Others come pretty close to the real thing, like these Ray LaMontagne and Ryan Adams songs, which sound like they’ve just had a few too many whiskeys.
Everything feels very Frankensteiny right now, but imagine a few years from now when these techniques are improved and expanded. We may reach a point where there is a virtually unlimited universe of concert-quality covers you can create with just a few taps. As a music lover, this is super intriguing, but on the other hand, I wonder how musicians will feel about it. And will their opinions change based on whether we can find a way to monetize it generously for them? I could see some artists rejecting this sort of thing outright because it’s not real music in the traditional sense, and I wouldn’t blame them. But what if you told them that every time their voice was mixed into another song, they made a royalty off of it? That might change some opinions.
This is going to be a really fun space to watch closely over the next few years. Until then, I leave you with another great cover: Metallica’s Orion — by Rodrigo y Gabriela. Incidentally, the header image for this page is from their Masonic Auditorium show in 2015. Pure luck but probably the best photo I’ve ever taken.
