Become a Creator today!Start creating today - Share your story with the world!
Start for free
00:00:00
00:00:01
Behind the Beat: How AI is Changing Music w/Maya Ackerman PhD
110 Plays1 year ago
In this episode of the AI-Driven Marketer, Dan Sanchez delves into the intriguing intersection of AI and music with Dr. Maya Ackerman, CEO and co-founder of WaveAI. They explore how AI is not just infiltrating every aspect of our lives but is also revolutionizing the music industry by transforming the songwriting process. From foundational machine learning to computational creativity, Dr. Ackerman shares her journey from academia to entrepreneurship, highlighting how her personal experiences and challenges in songwriting led to the creation of WaveAI's groundbreaking tools, LyricStudio and MelodyStudio. Discover how these AI-driven platforms empower musicians, enabling them to create music more efficiently and expressively. This conversation sheds light on the ethical considerations, the future of AI in music, and the balance between technology and human creativity.
Episode Timestamps:
00:00 - Introduction to the transformative role of AI in music with Dr. Maya Ackerman.
01:01 - Dr. Ackerman's journey from academic research to pioneering AI in music.
04:32 - Understanding how LyricStudio and MelodyStudio work and their unique approach to aiding musicians.
06:22 - The non-invasive, artist-centric design philosophy behind WaveAI's products.
08:05 - The training processes and creative foundation of WaveAI's AI models.
10:18 - The role of 'hallucinations' or creative freedom in AI's music composition.
13:20 - Addressing the challenges and limitations musicians face with AI.
17:22 - Success stories and the transformative impact of WaveAI's tools on musicians.
19:57 - Advice for musicians skeptical about integrating AI into their workflow.
22:12 - How to get started with WaveAI's tools, and the accessibility of LyricStudio and MelodyStudio.
Recommended
Transcript
Introduction to AI in Music
00:00:00
Speaker
Welcome back to AI Micro Skills, where we're trying to empower marketers to master AI and revolutionize their practices. Today, I'm excited because we're venturing a little bit off the beaten path of marketing to talk about music with Dr. Maya Ackerman, who's the CEO and co-founder of Wave AI around how AI is transforming the music space. So Maya, thank you so much for joining me on the show. It's a real pleasure. Thank you for having me.
00:00:28
Speaker
So I'd love to dive into like your backstory, like how this even became a thing for you. I didn't even realize AI was in music until a few weeks ago when a mutual friend of ours introduced me to it. And I was like, Oh my gosh, like, of course, of course it is because I mean, first lyric writing, but then even like music and being able to just improve the melody or the beat or whatever is going on. I'm like, it just makes a lot of sense. AI, it just seems like AI is creeping into everything, you know, and it's helping everything.
00:00:56
Speaker
But how did it get started for you? What inspired you to mix these two things together?
Dr. Ackerman's Journey into AI and Music
00:01:00
Speaker
Yeah, it's really quite an unusual story. I come from an academic background. I got my PhD at the University of Waterloo focusing on foundational machine learning.
00:01:12
Speaker
and then continued down the academic path, did my postdocs at Caltech, then UC San Diego. And in 2014, discovered a little known research field known as computational creativity group of a couple of hundred researchers, mostly out of Europe who were making machines that made music and art and poetry and dance choreography and recipes.
00:01:37
Speaker
since about the 1980s, even dating as far as the 1950s. It was just a really, really small number of people who were focusing on it, and there really wasn't any kind of spotlight on it. That's before IBM, before Microsoft, before Google, before any of these big companies even started dabbling in it.
Challenges and Inspirations in Songwriting
00:01:55
Speaker
What helped me recognize the opportunity specifically in music was that during my PhD,
00:02:01
Speaker
I took voice lessons from an opera singer and started performing semi-professionally within a year. And that's what led me to try to write my own songs and the frustrations that come with not being able to write the songs that I feel in my heart. And it was just sometimes your life experiences prepare you to see something that's there, but that other people just haven't had those experiences yet to be able to recognize the opportunity.
00:02:30
Speaker
And so knowing a little bit about computational creativity and the possibilities there and knowing what I knew about songwriting and the struggles I was having, it was right there. It's right there. So funny hearing that because I came into marketing through a design background and obviously like
00:02:45
Speaker
Most people get into design because they were a visual artist first, you know, we all start with like pen and paper and drawing. But what you just said, I'm like, Oh, that maps so well to like where I mean, with as a visual artist, you're like, you have this grand vision of like what something could be, yet your ability and skill to be able to bring it to life is sometimes it's just not there. It takes more practice, more reps. And I have been finding with AI on the visual side,
00:03:11
Speaker
that it's giving me tools to be able to unlock my creativity and maybe even from lack of skill of being able to get it there or just time fiddling with something. I'm being able to find and make things faster and actually get to where I'd kind of dreamed of something might be sooner because I can go to mid-journey and create something quickly and find and select something. So it's interesting. It sounds like it's kind of taking place in a music place as well.
Wave AI's Role in Music Creation
00:03:37
Speaker
In its own way, what we're doing is quite different from the work of Mid-Journey. I love Mid-Journey. I think it's so fun. I'm not a visual artist. I dabbled in it as a kid. That does not count. So I'm okay to some degree to have something that
00:03:54
Speaker
driving the process for me in art, even though there too I get frustrated sometimes that I can't really convey all of my vision. It understands what it understands and it fills it in how it wants to fill it in. It's marvelous, it's magical, but I'm not really driving everything that my company does in music, Wave AI. The user is not only getting help, but they're never getting hindered. That's just as important. When you think about a human collaborator, if I'm helping you with something,
00:04:24
Speaker
It's important that at the same time, I don't also hinder you just as much. So that's on some level similar, but in very important details, very different perspective on what the role of AI is. Well, let's dive into it. How do Lyric Studio and Melody Studio work and what makes them unique? How is it guiding the process? So our products are made entirely for musicians and people who want to be musicians.
00:04:50
Speaker
They're not for people who want music. They're for people who want to make music. And that attitude is kind of in the DNA of the company. We build our machine learning models, our generative AI models, completely from scratch, completely from scratch. So that respect for artists, that need for the model to lean into the artist's style, the need to be infinitely creative to the same suggestions are never given to two different users.
00:05:20
Speaker
All these ideas that make RAI really helpful to the artist is built completely from scratch, completely from scratch. We don't rely on chat CPT or any of the other large language models. We don't rely on the large music models. We build things from scratch so that it really helps you
00:05:41
Speaker
express your vision rather than replacing you or doing so much work for you that there is no opportunity for you to express yourself or no opportunity for you to grow as an artist. So it's really the difference between the AI being the star and the AI being a background support actor. And for us, it's really about that support role so that our users should really be the only stars in this show.
00:06:09
Speaker
So is it something where you kind of have to get it started where you're, do you bring it a rough draft first and then it helps you fill in the gaps or is it coming in and like maybe helping you master something?
00:06:20
Speaker
Well, it's specifically for lyrics and melodies in chords, so it helps you create those lead sheets, that foundational element of music. But it can work in many different ways. And that's part of the non-invasiveness of the process, that you can get started in a lot of different ways. So for example, somebody brand new can come in and
00:06:40
Speaker
you know, pick a few topics and just start clicking on the suggestions and quickly build something out. We generate these suggestions one line at a time for their studio. And, you know, that can be exactly what they needed. They wanted a lot of support, so they lean heavily into the product. On the other hand, we've had an artist who had a song laying in her drawers for three years. She pulled out the song, typed it into their studio and just wanted help with this second verse.
00:07:09
Speaker
And it helped her with that second verse. And then she was done. So help how she needs help. I use it a little differently. I sometimes have one line idea. And I generate a lot of thoughts from it. And it helps me lay out my song. It helps me kind of come up with the emphasis for each verse. I don't know if anyone else uses it like this. That's one of my use cases. And then there's a whole bunch of different stuff for Melody Studio that's just as flexible.
00:07:35
Speaker
We had one person who like real time generates the lyrics and sings them in real time as they get generated. And it's beautiful. It just like really fits his workflow. And there are so many more examples with how Melody Studio gets used as well. What are some of the processes you've trained the AI on in order for it to be better at coming up with lyrics than something like a chat GPT?
Creative Potential of AI in Music
00:07:58
Speaker
Well, you know, chat GPT is
00:08:01
Speaker
I mean, I'm in such awe of these large models. I really, really am. I don't want anybody to think otherwise. I think they're marvelous. I think they're brilliant. They obviously have problems as well. That's not without that. But I think, you know, the researcher in the space as an entrepreneur and generative AI, I think it's, you know, credit where credit is due. It's there's really a lot of amazing, amazing stuff about chat GPT. The system is what's called convergent. So really, the brilliance of open AI was that
00:08:28
Speaker
they decided to make a chatbot. So these are generative models, which until chatGPT called it a chatbot, until OpenAI decided to call it a chatbot, people thought about it as a generative model, which is really conceptually quite different. When we think of a chatbot, we think of something that should have a clue about what it's talking about. For example, you could ask it some simple questions, you know, some really basic math,
00:08:50
Speaker
or maybe questions about when different countries were founded. And you expect answers that are correct. And maybe this seems trivial, but it's really not. Because when you take a generative model that's inherently hallucinatory, inherently imaginative, and you want it to start giving you the right answer to some stuff,
00:09:09
Speaker
You have to make a lot of decisions in the construction of the model to end up in that space. We are not constrained by that. Lyric Studio does not need to know anything about, you know, how to cook, what ingredients go well together, what the weather is like in different parts of the world. It just needs to know how to write lyrics. And we give it permission to be as imaginative as possible because it's not constrained. Again, there is no notion of right or wrong. And that's one of the core kind of foundational principles that make our models
00:09:39
Speaker
really orders of magnitude more creative than any LLM ever could be. That's interesting. So hallucinations actually are a benefit in your case because in a way it can be creative if it comes up with a weird way of saying something. You're kind of like, actually that kind of works because lyrics kind of always were that way. Hallucinations are in the DNA of
00:10:00
Speaker
of generative AI. Calling it hallucination is actually kind of funny. The word choice is funny. Hallucinations are associated with something negative, right? Like when I generally say, oh, he's hallucinating, he's hallucinating. It's like a bad thing. But when a human being is being imaginative, oh, no, that's positive, imaginative, creative, right? So we have chosen a negative word.
00:10:21
Speaker
to talk about the capabilities of these systems. Because we are thinking about, because we want the machine to be convergent. We want the machine to do boring things and be right. Meanwhile, these machines are better suited for the imaginative, wild, crazy, imaginative applications. And if we don't try to shove them into the convergent box, then suddenly the fact that they're hallucinating is obviously an advantage. That's an interesting way to frame it. What word did you use to describe it?
00:10:49
Speaker
I mean, I've been in the field for 10 years before the rise of generative AI, and these are creative machines. That's a word I use. They're creative. And maybe chat GPT is not creative enough for some people, but my models definitely are creative. On the melody studio side, how is it helping musicians craft better music?
Navigating Musicians' Concerns with AI
00:11:11
Speaker
The core there is to help create a lead sheet. I'm going to talk first about our most recent use case because I'm so excited about it. So we just added the capability of our musician users uploading a backing track. So any beat that they already have.
00:11:29
Speaker
They get to upload it to the system and then the system can help them write lyrics as well, or they can bring their own. And then the system comes up with top line melodies, with the vocal melodies that fit the lyrics and fit the backing track. Wow.
00:11:44
Speaker
And that's such a game changer. I'm so excited about this. And so that's already out. It's already rolled out. People are already using it. We're just so, so thrilled to be able to offer that. The default use case used to be just the same thing, but without the backing track. So that's when a musician wants to build a song from scratch, kind of starting with the lyrics and melodies and then later bringing in
00:12:05
Speaker
the music production, we also have people, you know, guitar players, piano players, who create, who use AI to ultimately make instrumental music. Because for me, it's really not about the new replacing the old, I'm an opera singer and a piano player, right? Like, does that census it? Like when people use our AI tools to go back to instrumental music, which is what I do very often,
00:12:28
Speaker
I think that's really cool, in addition to our very many users who essentially use our tools to create modern music that includes digital audio workstations in some part of the process. Can you discuss any challenges or limitations musicians face when using AI? In general, or our tools specifically? We can start in general and then zoom into your tools. There's a lot of...
00:12:53
Speaker
It's a lot of challenges. And you know, the longer I'm in the space, the more I'm on the musician side, partially because I am a musician and partially because they're right.
00:13:03
Speaker
They have a lot of concerns, a lot of concerns. How can they not have a lot of concerns? Like, look what streaming did to them. It used to be if you were, you know, you could be a small scale musician and make a living. It's almost impossible these days to make a living through your music because you can't sell CDs, right? And streaming just pays so, so little to what CDs used to pay. It's really, the gap is wild and this all happened sort of
00:13:29
Speaker
Just without the cooperation agreement or anything of musicians, it just sort of happened to them as opposed to even, you know, not even with them. And I don't want the same thing to happen with AI. They don't want the same thing to happen with AI. I don't want the same thing to happen with AI. You know, and sometimes when companies say we're helping artists, you go and you try their tool and it's just not true. It's just not true. You pick a few parameters and it writes a whole song for you.
00:13:55
Speaker
in what universe is that helping? That's replacing, right? And musicians are smart people that can tell the difference between helping and replacing. So, and sometimes, I mean, I should give credit where credit is due. You know, you have very, very creative people who take these tools that are not really designed to work with them, and they still find ways to use these kind of replace of tools inside of their creative process.
00:14:18
Speaker
creative people. So sometimes they find a way anyways, but they're also feeling the threat as a result. And so I actually feel a lot of personal responsibility here in being on the right side of history. So in a universe where
00:14:33
Speaker
automated music generation gets progressively better. So music that really doesn't need a person very much or doesn't need a musician. In a world where this kind of music exists, we definitely need the best possible tools to help artists. So that's where my work comes in. It's fascinating.
00:14:52
Speaker
I don't know. I definitely thought for a while, especially on more of the visual side, that these tools are going to replace them. The more I play with it, and as good as they are, I'm like, you know what? This is only going to be enhancing what they do. I guess you could go in the far future and just be like, write me 100 songs, and it makes 100 songs, and you pick a few that works better, and then submit them to the streaming services or whatever is available then.
00:15:19
Speaker
But at the same time, musicians are going to be able to do it better than everybody else because they just have the ear for what is good or what should be or what could be. I mean, it's a problem with mid-journey for most people is they don't have the vocabulary to even describe what they want, right?
00:15:35
Speaker
very, very true. It's an interesting landscape, but it's not entirely clear yet where it's going to land. I think like yourself, I tend to be optimistic. I tend to believe that human creativity will survive and will definitely survive, but will sort of like remain dominant for a variety of reasons. But at the same time, I like to
00:15:56
Speaker
Just if I'm being really sincere, I think there is some threat. And I think artists and musicians don't want any threat to their livelihood. So they're not okay with a small percent of it being carved out, because there's already not enough. There's already so many talented people that can't make a living on their art and craft and musicianship. So is some damage going to happen? That's possible. And that's, we need to be responsible, do our best, and to just take their concerns seriously. That's my take.
00:16:26
Speaker
Can you share a success story of somebody using your tool in order to make a masterpiece that maybe they would have had a hard time doing or taken a much longer time or have been impossible before? Oh, there's so much of that. Oh my goodness. That's why we do what we do, right? So, I mean, one of my favorite stories is Curtis King about a year ago. About a year and a half ago now. He made an album called DIY 2.
00:16:54
Speaker
that, you know, Lyric Studio he used to aid in his Lyric creation. And it ended up being number one on the iTunes hip-hop charts. Wow. So that was just so awesome to hear. Just totally made my week when I heard that.
00:17:10
Speaker
But then there's so much more. There are just these beautiful pieces of music that we get sent sometimes, and then we find out that somebody won a contest, got money on it. I know that some artists who write a lot of music and they make a little bit of money on each song, that they use our system to be more efficient, which is also a valid use case. They're just able to write faster and more, which as a result, they make more money, which is excellent.
00:17:38
Speaker
But then for me, it doesn't have to be a monetary or an award for it to be a success story. There was this one, several really incredible songs that I heard. I was like, wow, this is just so good. And we only hear the songs when our users choose to send them to us. Otherwise, it's complete privacy. So in case of those few songs that were shared with us, the people who made them, they said that they
00:18:04
Speaker
wouldn't have been able to make them without our tools. They just were missing certain skills like lyrics, writing, or melody creation. It's like, wow, this song would not have existed without our tools, and this person would not have been able to express it. And sometimes it's such personal stories that get expressed in these songs. So really, we've helped millions of people write music.
00:18:24
Speaker
But even one song that we helped get out there and help the artists share what's in their hearts, I think, I think it's kind of priceless for me. Absolutely.
00:18:35
Speaker
And hopefully these stories that you're sharing now convince more people. But for those who are skeptical, for those who are, and many musicians are, right? Like musicians are kind of a skeptical bunch when it comes to sometimes technology or business or a lot of other things. What advice would you give for those who are kind of on the edge and not sure they want to try out an AI tool as part of their workflow? What advice would you give to them? I think the first thing that I'd say is that I get it. Given the tools that are popular,
00:19:02
Speaker
Nowadays, I can understand why people are, you know, some people find it hard to believe that there are tools that are genuinely helpful for artists. What I can say is that within the first few minutes, musicians get our tools, like it's not a long process. They try, they're like, oh!
00:19:17
Speaker
Oh, that is actually helping me. And then they get really excited. So I do hope you check it out, but we are here to be part of your creative journey if this is something that you'd like. And I'm perfectly happy to respect it if musicians, some people are purists in different ways and that's also good. If some people feel strongly about not using AI, I can respect that as well.
00:19:42
Speaker
And I hope there will be some people who never do and go all analog, everything, because there's something beautiful about that too, right? There is. We really need artists of all kinds. I really don't think there's any tool that's for everybody. And I think that's okay. I think as people who make music tools, I think we need to be comfortable with that and give people the invitation. One way that we give people the invitation is that you can try our tools for free. And I think that's in part because we want to respect that.
00:20:12
Speaker
It's a very new tool, so people should be able to play around with it for free. It's not credit card backed. You just go in, play with it a little bit, see if you like it before deciding if you want to add it to your repertoire.
Access and Future of Wave AI Tools
00:20:25
Speaker
And then also like most people don't tell anybody that they use our tools. So it's just, and there's no way to tell. You'll never be able to tell. It's not like chat GPT where it's detectable. Yeah. Yeah.
00:20:35
Speaker
So for those listening and want to get started with it, where's the best place to start? Would you start with just one or one tool or the other, or would you kind of dive into both? Well, it depends. We have a lot of rappers, for example, who just use Lyric Studio because that's kind of a perfect, very, very simple interface, just focusing on lyrics. But if you think you could, if you're curious how AI could be a little bit helpful to brainstorm melodies for your top line, if you are a singer,
00:21:05
Speaker
And, you know, you have your beats that you get somewhere or your producer, you create your own beats with Melody Studio, you could see kind of like explore the space of possibilities for what type of top line melodies can work on top of these beats that you have. Yeah. So it depends on who you are. You could test out one of the other products. Fantastic.
00:21:25
Speaker
Thank you so much for joining me on the show and helping us understand where AI can really help musicians succeed and fill in some gaps or just compliment their current skill sets. My absolute pleasure. Thanks so much for having me.