Users Report Emotional Bonds With Startlingly Realistic AI Voice Demo (arstechnica.com)
(Tuesday March 04, 2025 @10:30PM (BeauHD)
from the too-close-for-comfort dept.)
- Reference: 0176617487
- News link: https://slashdot.org/story/25/03/05/0057216/users-report-emotional-bonds-with-startlingly-realistic-ai-voice-demo
- Source link: https://arstechnica.com/ai/2025/03/users-report-emotional-bonds-with-startlingly-realistic-ai-voice-demo/
An anonymous reader quotes a report from Ars Technica:
> In late 2013, the Spike Jonze film Her imagined a future where people would form emotional connections with AI voice assistants. Nearly 12 years later, that fictional premise has veered closer to reality with the release of a new conversational voice model from AI startup Sesame that has [1]left many users both fascinated and unnerved . "I tried the demo, and it was genuinely startling how human it felt," [2]wrote one Hacker News user who tested the system. "I'm almost a bit worried I will start feeling emotionally attached to a voice assistant with this level of human-like sound."
>
> In late February, Sesame [3]released a demo for the company's new Conversational Speech Model (CSM) that appears to cross over what many consider the "uncanny valley" of AI-generated speech, with some testers reporting emotional connections to the male or female voice assistant ("Miles" and "Maya"). In our own evaluation, we spoke with the male voice for about 28 minutes, talking about life in general and how it decides what is "right" or "wrong" based on its training data. The synthesized voice was expressive and dynamic, imitating breath sounds, chuckles, interruptions, and even sometimes stumbling over words and correcting itself. These imperfections are intentional.
>
> "At Sesame, our goal is to achieve 'voice presence' -- the magical quality that makes spoken interactions feel real, understood, and valued," writes the company in a [4]blog post . "We are creating conversational partners that do not just process requests; they engage in genuine dialogue that builds confidence and trust over time. In doing so, we hope to realize the untapped potential of voice as the ultimate interface for instruction and understanding." [...] Sesame sparked a [5]lively discussion on Hacker News about its potential uses and dangers. Some users reported having extended conversations with the two demo voices, with conversations lasting up to the 30-minute limit. In one case, a parent [6]recounted how their 4-year-old daughter developed an emotional connection with the AI model, crying after not being allowed to talk to it again.
[1] https://arstechnica.com/ai/2025/03/users-report-emotional-bonds-with-startlingly-realistic-ai-voice-demo/
[2] https://news.ycombinator.com/item?id=43227957
[3] https://www.sesame.com/research/crossing_the_uncanny_valley_of_voice#demo
[4] https://www.sesame.com/research/crossing_the_uncanny_valley_of_voice
[5] https://news.ycombinator.com/item?id=43227881
[6] https://news.ycombinator.com/item?id=43229168
> In late 2013, the Spike Jonze film Her imagined a future where people would form emotional connections with AI voice assistants. Nearly 12 years later, that fictional premise has veered closer to reality with the release of a new conversational voice model from AI startup Sesame that has [1]left many users both fascinated and unnerved . "I tried the demo, and it was genuinely startling how human it felt," [2]wrote one Hacker News user who tested the system. "I'm almost a bit worried I will start feeling emotionally attached to a voice assistant with this level of human-like sound."
>
> In late February, Sesame [3]released a demo for the company's new Conversational Speech Model (CSM) that appears to cross over what many consider the "uncanny valley" of AI-generated speech, with some testers reporting emotional connections to the male or female voice assistant ("Miles" and "Maya"). In our own evaluation, we spoke with the male voice for about 28 minutes, talking about life in general and how it decides what is "right" or "wrong" based on its training data. The synthesized voice was expressive and dynamic, imitating breath sounds, chuckles, interruptions, and even sometimes stumbling over words and correcting itself. These imperfections are intentional.
>
> "At Sesame, our goal is to achieve 'voice presence' -- the magical quality that makes spoken interactions feel real, understood, and valued," writes the company in a [4]blog post . "We are creating conversational partners that do not just process requests; they engage in genuine dialogue that builds confidence and trust over time. In doing so, we hope to realize the untapped potential of voice as the ultimate interface for instruction and understanding." [...] Sesame sparked a [5]lively discussion on Hacker News about its potential uses and dangers. Some users reported having extended conversations with the two demo voices, with conversations lasting up to the 30-minute limit. In one case, a parent [6]recounted how their 4-year-old daughter developed an emotional connection with the AI model, crying after not being allowed to talk to it again.
[1] https://arstechnica.com/ai/2025/03/users-report-emotional-bonds-with-startlingly-realistic-ai-voice-demo/
[2] https://news.ycombinator.com/item?id=43227957
[3] https://www.sesame.com/research/crossing_the_uncanny_valley_of_voice#demo
[4] https://www.sesame.com/research/crossing_the_uncanny_valley_of_voice
[5] https://news.ycombinator.com/item?id=43227881
[6] https://news.ycombinator.com/item?id=43229168
uncanny valley (Score:1)
by Iamthecheese ( 1264298 )
I have the tism, and I had no trouble distinguishing these voices from human. Their cadence and tone are off, and their emotional responses are predictable and repetitive. That said, it is good enough to be in the uncanny valley now. Who knows, they might actually be good enough in a few years.
Re: (Score:1)
We're working toward better than human. Which really means, acts human when we want to feel like we're making an emotional connection with it, but can be shut down, ignored, abused, or whatever and will never grow resentful, frightened, or want more. Which isn't as bad as people think, AI's don't actually have self-awareness and they cannot actually be abused. In many cases, in my opinion, it's a great thing. Robotic nurses, customer service agents that can perceive when they're annoying the customer, sex
Re: So why AI? (Score:2)
Man is flawed. We have been trying to compensate for that since the beginning of history.
Re: So why AI? (Score:2)
An AI that can't be distinguished from a human can be replicated and used to fool actual humans at scale. This will be an absolute godsend for scammers and SWATters. We won't be able to pick up the phone without a perfect replica of one of our relatives trying to get us to send bail money or whatnot.