Wish you could sing like Charli XCX or possess any musical talent? YouTube AI might make that happen
(2023/11/17)
- Reference: 1700182035
- News link: https://www.theregister.co.uk/2023/11/17/youtube_generative_ai/
- Source link:
YouTube is experimenting with software that generates music using an AI model called Lyria built by Google DeepMind.
The Google-owned home-video giant announced on Thursday two features it's pushing out to a small group of testers: Dream Track and the Music AI Tool.
Dream Track converts a text-based prompt into a short audio snippet that mimics the voice and style of various pop stars – namely Alec Benjamin, Charlie Puth, Charli XCX, Demi Lovato, John Legend, Sia, T-Pain, Troye Sivan, and Papoose. The choice of artist is limited to these performers so far because Google has had to negotiate licenses to train Lyria on their music to avoid a copyright war.
[1]
You can listen to what Dream Track generates for the prompt "a sunny morning in Florida, R&B" in the style of T-Pain – an artist well known for altering his voice with autotune – below.
[2]
[3]
[4]Youtube Video
It certainly sounds like T-Pain, and the lyrics are appropriate and match the prompt, too. Dream Track is currently only available to "a limited set of creators" who can generate 30-second clips of AI-made tracks that can be posted as YouTube Shorts – typically minute-long videos.
[5]
Music AI Tool seems more interesting and useful. It allows folks, particularly those without any or many instruments, to transform an audio clip – such as a chord or someone humming a tune – into something that retains the original sound but played in the form of another instrument.
The most impressive demo, perhaps, converts a bunch of "na-na-na" singing sounds into a orchestral score, complete with string instruments that seems like it could be a somewhat convincing soundtrack for a film. You can hear it below.
[6]Youtube Video
[7]
YouTube is only sharing Music AI Tool with select artists, songwriters, and producers that are part of its Music AI Incubator program right now.
Music AI Tool allows people to create music in new forms without having to know how to sing or play musical instruments particularly well – much like how anyone can use text-to-image models to generate artwork without having to know how to draw or paint.
It kinda reminds us of years ago, in the rise of electronic music, when cynics moaned that synthesizers and computer-aided sequencing let anyone churn out tracks like so-called real musicians.
"These experiments explore the potential of AI features to help artists and creators stretch their imaginations and augment their creative processes," [8]explained YouTube's Lyor Cohen and Toni Reid, global head of music, and VP emerging experiences & community products, respectively.
"And in turn, fans will be able to connect to the creatives they love in new ways, bringing them closer together through interactive tools and experiences. All of this will help us iterate and enhance the technology, informing applications for the future."
[9]AI copyright row deepens: Stability VP quits in protest over 'fair use' excuse
[10]YouTubers kindly asked to mark their deepfake vids as Fake Fakey McFake Fakes
[11]Bad Vibrations: Music publishers sue Anthropic AI for using copyrighted lyrics
[12]Spotify now using AI to clone podcaster's voice into Spanish
Generative AI and music, however, is particularly tricky. Not only is it difficult to build models capable of creating audio that actually sounds good, but securing the data to train the systems is difficult. Record labels are notoriously litigious when it comes to protecting their copyrights – as YouTube knows very well. The video site has said it's working around these issues, and is trying to enter licensing agreements to compensate artists for their music.
"Despite the tremendous opportunity AI presents, we also recognize it's a quickly evolving space that presents complex challenges. One of YouTube's greatest strengths is our strong relationships with music industry partners. We're committed to collaborating with them as we enter this new era, critically exploring together new opportunities and developing sensible and sustainable controls, monetization and attribution frameworks," Cohen and Reid added.
Meanwhile, researchers at Google DeepMind are tackling the problem of fake AI-generated audio that could be used to manipulate or mislead listeners. Tracks created using its Lyria model will carry imperceptible watermarks from its SynthID tool used to identify synthetic content. SynthID [13]apparently works by converting audio data into a two-dimensional spectrogram, applying a digital watermark to that representation, and converting it back into audio.
"The watermark is designed to maintain detectability even when the audio content undergoes many common modifications such as noise additions, MP3 compression, or speeding up and slowing down the track. SynthID can also detect the presence of a watermark throughout a track to help determine if parts of a song were generated by Lyria," DeepMind [14]explained . ®
Get our [15]Tech Resources
[1] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2ZVbzd091fb8cciDR9a8CJQAAAE4&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[2] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZVbzd091fb8cciDR9a8CJQAAAE4&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33ZVbzd091fb8cciDR9a8CJQAAAE4&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[4] https://www.youtube.com/watch?v=F4Ngs4mZ5cY
[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZVbzd091fb8cciDR9a8CJQAAAE4&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[6] https://www.youtube.com/watch?v=aC8I2YvL6Uo&t=20s
[7] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33ZVbzd091fb8cciDR9a8CJQAAAE4&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[8] https://blog.youtube/inside-youtube/ai-and-music-experiment/
[9] https://www.theregister.com/2023/11/16/stability_ai_vp_quits/
[10] https://www.theregister.com/2023/11/14/youtube_ai_label/
[11] https://www.theregister.com/2023/10/20/music_publishers_sue_anthropic/
[12] https://www.theregister.com/2023/09/27/spotify_now_using_ai_to/
[13] https://deepmind.google/technologies/synthid/
[14] https://deepmind.google/discover/blog/transforming-the-future-of-music-creation/
[15] https://whitepapers.theregister.com/
The Google-owned home-video giant announced on Thursday two features it's pushing out to a small group of testers: Dream Track and the Music AI Tool.
Dream Track converts a text-based prompt into a short audio snippet that mimics the voice and style of various pop stars – namely Alec Benjamin, Charlie Puth, Charli XCX, Demi Lovato, John Legend, Sia, T-Pain, Troye Sivan, and Papoose. The choice of artist is limited to these performers so far because Google has had to negotiate licenses to train Lyria on their music to avoid a copyright war.
[1]
You can listen to what Dream Track generates for the prompt "a sunny morning in Florida, R&B" in the style of T-Pain – an artist well known for altering his voice with autotune – below.
[2]
[3]
[4]Youtube Video
It certainly sounds like T-Pain, and the lyrics are appropriate and match the prompt, too. Dream Track is currently only available to "a limited set of creators" who can generate 30-second clips of AI-made tracks that can be posted as YouTube Shorts – typically minute-long videos.
[5]
Music AI Tool seems more interesting and useful. It allows folks, particularly those without any or many instruments, to transform an audio clip – such as a chord or someone humming a tune – into something that retains the original sound but played in the form of another instrument.
The most impressive demo, perhaps, converts a bunch of "na-na-na" singing sounds into a orchestral score, complete with string instruments that seems like it could be a somewhat convincing soundtrack for a film. You can hear it below.
[6]Youtube Video
[7]
YouTube is only sharing Music AI Tool with select artists, songwriters, and producers that are part of its Music AI Incubator program right now.
Music AI Tool allows people to create music in new forms without having to know how to sing or play musical instruments particularly well – much like how anyone can use text-to-image models to generate artwork without having to know how to draw or paint.
It kinda reminds us of years ago, in the rise of electronic music, when cynics moaned that synthesizers and computer-aided sequencing let anyone churn out tracks like so-called real musicians.
"These experiments explore the potential of AI features to help artists and creators stretch their imaginations and augment their creative processes," [8]explained YouTube's Lyor Cohen and Toni Reid, global head of music, and VP emerging experiences & community products, respectively.
"And in turn, fans will be able to connect to the creatives they love in new ways, bringing them closer together through interactive tools and experiences. All of this will help us iterate and enhance the technology, informing applications for the future."
[9]AI copyright row deepens: Stability VP quits in protest over 'fair use' excuse
[10]YouTubers kindly asked to mark their deepfake vids as Fake Fakey McFake Fakes
[11]Bad Vibrations: Music publishers sue Anthropic AI for using copyrighted lyrics
[12]Spotify now using AI to clone podcaster's voice into Spanish
Generative AI and music, however, is particularly tricky. Not only is it difficult to build models capable of creating audio that actually sounds good, but securing the data to train the systems is difficult. Record labels are notoriously litigious when it comes to protecting their copyrights – as YouTube knows very well. The video site has said it's working around these issues, and is trying to enter licensing agreements to compensate artists for their music.
"Despite the tremendous opportunity AI presents, we also recognize it's a quickly evolving space that presents complex challenges. One of YouTube's greatest strengths is our strong relationships with music industry partners. We're committed to collaborating with them as we enter this new era, critically exploring together new opportunities and developing sensible and sustainable controls, monetization and attribution frameworks," Cohen and Reid added.
Meanwhile, researchers at Google DeepMind are tackling the problem of fake AI-generated audio that could be used to manipulate or mislead listeners. Tracks created using its Lyria model will carry imperceptible watermarks from its SynthID tool used to identify synthetic content. SynthID [13]apparently works by converting audio data into a two-dimensional spectrogram, applying a digital watermark to that representation, and converting it back into audio.
"The watermark is designed to maintain detectability even when the audio content undergoes many common modifications such as noise additions, MP3 compression, or speeding up and slowing down the track. SynthID can also detect the presence of a watermark throughout a track to help determine if parts of a song were generated by Lyria," DeepMind [14]explained . ®
Get our [15]Tech Resources
[1] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2ZVbzd091fb8cciDR9a8CJQAAAE4&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[2] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZVbzd091fb8cciDR9a8CJQAAAE4&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33ZVbzd091fb8cciDR9a8CJQAAAE4&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[4] https://www.youtube.com/watch?v=F4Ngs4mZ5cY
[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZVbzd091fb8cciDR9a8CJQAAAE4&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[6] https://www.youtube.com/watch?v=aC8I2YvL6Uo&t=20s
[7] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33ZVbzd091fb8cciDR9a8CJQAAAE4&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[8] https://blog.youtube/inside-youtube/ai-and-music-experiment/
[9] https://www.theregister.com/2023/11/16/stability_ai_vp_quits/
[10] https://www.theregister.com/2023/11/14/youtube_ai_label/
[11] https://www.theregister.com/2023/10/20/music_publishers_sue_anthropic/
[12] https://www.theregister.com/2023/09/27/spotify_now_using_ai_to/
[13] https://deepmind.google/technologies/synthid/
[14] https://deepmind.google/discover/blog/transforming-the-future-of-music-creation/
[15] https://whitepapers.theregister.com/
Who?
jake
"namely Alec Benjamin, Charlie Puth, Charli XCX, Demi Lovato, John Legend, Sia, T-Pain, Troye Sivan, and Papoose."
I've never heard of any of 'em.
Why do I fear that this state of affairs will not continue for long ... and I'll sadly be the worse for it.
So basically, it's autotune stepped up a notch.
jake
One wonders if it'll give me worse headaches, faster, than the original ... and how many wannabe "musicians" are going to inflict their lack of talent on the rest of us using it.
Copyright war
"The choice of artist is limited to these performers so far because Google has had to negotiate licenses to train Lyria on their music to avoid a copyright war."
and
"but securing the data to train the systems is difficult. Record labels are notoriously litigious when it comes to protecting their copyrights – as YouTube knows very well. The video site has said it's working around these issues, and is trying to enter licensing agreements to compensate artists for their music. "
What wrong with scraping the internet for data as they have for all their other LLMs? I thought it was 'fair use' to train AI on copyrighted material. It seems it is only 'fair use' when it's the little guy's copyrighted material. 'Fair use' does not apply when the copyrighted material is owned by a huge corporation.
As always there is one rule for us and no fucking rules for big business.