Claude’s voice chat isn’t “native” though, is it? It feels like it’s speech-to-text-to-LLM and back.

You can test it by asking it to: change the pitch of its voice, make specific sounds (like laughter), differentiate between words that are spelled the same but pronounced differently (record and record), etc.

▲

lxgr 2 days ago | parent | next [-]

Good idea, but an external “bolted on” LLM-based TTS would still pass that in many cases, right?

▲

sosodev 2 days ago | parent | next [-]

Yes, a sufficiently advanced marrying of TTS and LLM could pass a lot of these tests. That kind of blurs the line between native voice model and not though.

You would need:

* A STT (ASR) model that outputs phonetics not just words

* An LLM fine-tuned to understand that and also output the proper tokens for prosody control, non-speech vocalizations, etc

* A TTS model that understands those tokens and properly generate the matching voice

At that point I would probably argue that you've created a native voice model even if it's still less nuanced than the proper voice to voice of something like 4o. The latency would likely be quite high though. I'm pretty sure I've seen a couple of open source projects that have done this type of setup but I've not tried testing them.

	▲	BoxOfRain a day ago \| parent [-]
		I've been experimenting with something similar to this approach recently. IndexTTS2 gives you emotion vectors as an input, I used an external emotion classification model on the LLM output to modulate the TTS emotion vectors. You need to manage the state of the current affect with a bit of care or it sounds unhinged, but it's worked surprisingly well so far. I wired it together using Cats Effect. As you'd expect latency isn't great, but I think it can be improved.

▲

barrkel 2 days ago | parent | prev | next [-]

The model giving it text to speak would have to annotate the text in order for the TTS to add the affect. The TTS wouldn't "remember" such instructions from a speech to text stage previously.

▲

jablongo 2 days ago | parent | prev [-]

I tried to make ChatGPT sing Mary had a little lamb recently and it's atonal but vaguely resembles the melody, which is interesting.

▲

2 days ago | parent | prev [-]

[deleted]

▲

causalmodels 2 days ago | parent | prev [-]

I just asked it and it said that it uses the on device TTS capabilities.

▲

furyofantares 2 days ago | parent [-]

I find it very unlikely that it would be trained on that information or that anthropic would put that in its context window, so it's very likely that it just made that answer up.

▲

causalmodels 2 days ago | parent [-]

No, it did not make it up. I was curious so I asked it asked it to imitate a posh British accent imitating a South Brooklyn accent while having a head cold and it explained that it didn't have have fine grained control over the audio output because it was using a TTS. I asked it how it knew that and it pointed me towards [1] and highlighted the following.

> As of May 29th, 2025, we have added ElevenLabs, which supports text to speech functionality in Claude for Work mobile apps.

Tracked down the original source [2] and looked for additional updates but couldn't find anything.

[1] https://simonwillison.net/2025/May/31/using-voice-mode-on-cl...

[2] https://trust.anthropic.com/updates

	▲	furyofantares 2 days ago \| parent [-]
		If it does a web search that's fine, I assumed it hadn't since you hadn't linked to anything. Also it being right doesn't mean it didn't just make up the answer.