I took a week of from work recently to reimplement Storyteller's forced alignment algorithm. Storyteller[1] is an open source, self hosted platform for creating, managing, and reading/listening to "readaloud" books — books that have audiobook narration built in and can highlight each sentence (and/or word, with this new algorithm!) as it's read aloud. Forced alignment is the process of determining where each piece of text starts and ends in the audiobook.

Anyway, I am really pleased with how the new algorithm turned out! Hopefully someone else finds it interesting as well.

[1] https://storyteller-platform.dev

▲

chrisweekly 14 hours ago | parent | next [-]

Cool! S an avid reader, Whispersync is one of the things that keeps me paying Amazon/Audible. Competition and viable OSS alternatives would be great!

	▲	smoores 14 hours ago \| parent [-]
		I'll post again when we release the v3 apps — even in alpha, they're really awesome, I think they blow the Kindle app out of the water!

▲

zackify a day ago | parent | prev | next [-]

This is great. Recently got an xteink x3 and jailbroke my kindle, making custom stuff for it.

I couldn't find it exactly in the docs. Sorry if I missed it.

If you align these and then pull onto koreader via opds or other way, will audio work over Bluetooth there too? Along with the highlighting.

▲

smoores a day ago | parent [-]

You can use a KOReader plugin, https://github.com/stradichenko/audiobook.koplugin, which has work-in-progress support for Media Overlays (the EPUB spec that Storyteller uses for readaloud)!

▲

zackify a day ago | parent [-]

Wow I didn't even think about just using on device AI.

I'm going to set this one up and try it with kokoro which has the most natural for small size that I've seen. Wonder if the paperwhite 12th Gen can handle it.

And also setup the main repo for when I have the real audio book. Thanks!

▲

leobg 20 hours ago | parent [-]

I tried running Kokoro on my iPhone XS (several years old), and it was slower than real-time. So, i wouldn’t expect it to be usable on a Kindle. But if you find a solution, please let me know.

	▲	deivid 9 hours ago \| parent [-]
		Kokoro can be hacked a bit (see script https://github.com/DavidVentura/translator-rs/blob/master/sc... ) which makes it about 3x faster. My phone went from slightly slower than realtime to very comfortable (also lower latency on first utterance!)

▲

orsenthil a day ago | parent | prev | next [-]

This is excellent! I wanted this. I had noticed that Kindle had limited feature like this for certain books that support audiobooks. I had tried this style of reading with audiobook and ebook without a any automated sync, but hadn't pursued it. This looks cool. :)

	▲	smoores a day ago \| parent \| next [-]
		Thanks! We're improving it all the time — we're working on big new releases ("v3") for the web app and mobile apps. There's a Discord server linked in the docs if you ever need any help out want to chat!
	▲	neves 16 hours ago \| parent \| prev [-]
		In Kindle you must pay for the ebook and audiobook to make it work

▲

zackify 16 hours ago | parent | prev [-]

Any plans to offer a way to generate audio books on demand. like if i set kokoro, a book auto imports, it can make the file for me for the whole book after some time of local gpu work?