moby dick didn't align properly, there's a confidence gate it didn't pass.

i know, it makes me sad too, i'll keep working on it

this is the list of everything: https://tale.fyi/audiobooks

librivox is all human volunteers, so i think it makes a difference from eleven labs, but i might add a fallback to ai readings for folks who want that

reset feedback noted!

▲

p-e-w 2 days ago | parent [-]

> moby dick didn't align properly, there's a confidence gate it didn't pass.

What does this mean?

▲

samuelcole 2 days ago | parent [-]

well! (i've been waiting for someone to ask ha)

when i first knew i wanted to align audio to text, i reached for aeneas, which was built for this exact problem/product (https://www.readbeyond.it/aeneas/)

however, it drifted by hundreds of seconds deep into chapters and emits no confidence signal, so i couldn't eval it cleanly

so i switched to a neural acoustic model trained with ctc (meta's massively multilingual speech aligner, it ships inside torchaudio): it outputs a probability for each character at every frame of audio, and viterbi decoding finds the most probable path that spells exactly our text, with a per-word probability

that probability is the gate: a book ships read-along only if its median confidence and coverage clear a bar. moby dick's didn't, so it's honestly text-only

▲

p-e-w a day ago | parent [-]

Could the problem be Moby Dick’s infamously large vocabulary, which contains words used essentially nowhere else?

▲

samuelcole a day ago | parent [-]

i actually had an out-of-memory bug, which i've since fixed. chewing through it now to see if we can get it

	▲	samuelcole a day ago \| parent [-]
		got it working! https://tale.fyi/moby-dick it got confused by some of the more uhh complex passages ;-)