Hi HN! I've been working on BookWith, an open-source e-book reader that integrates AI as your reading companion.

The problem: Traditional e-readers are passive. When you encounter something unclear, you have to context-switch to search for it. Your highlights and notes remain isolated, and you can't easily connect ideas across different books.

My solution: BookWith embeds an AI that maintains full context of what you're reading. It features:

- Context-aware AI chat: Ask questions about the current page/chapter and get instant answers

- AI podcast generation: Automatically converts book content into conversational podcasts using Google Cloud TTS

- Multi-layer memory system: Short-term (last 5 conversations), mid-term (summarized every 20), and long-term (vector search) memory that maintains continuity across reading sessions

- Smart annotations: 5-color highlighting system that AI can reference and analyze

Technical stack: Built as a fork of Flow (epub reader), with added LLM integration and vector database for semantic search. Supports multiple LLMs and languages (EN/JA/ZH).

If we are purely talking about textbooks, I can see the value in a tool like this... assuming we still have yet to actually solve the problem of AI being able to tell the truth and could just lead to more issues if we can't even read a textbook without it and then learn the wrong thing.

Not to necessarily diss the work that was done on this, but the idea of actually wanting this for reading feels like it is a continuation of the lack of attention span that has seemed to get worse and worse. We already saw this with the oversimplification of television shows and movies. Many of them leaning more towards slapping you in the face with something instead of subtly.

I know way too many people that struggle to sit still for a half hour episode of some show now (like my partner, frustratingly) and have to be doing something else.

If you are struggling with absorbing the information you are reading that is likely a sign you should put down the book and come back to it later, obviously your mind wants to be doing something else. If it is a continued issue than practice reading something that you know you would like. Personally my "in" for my love of reading was reading video game books that expanded the lore and it grew from there, but I was already invested in the story so the book was easier to read.

Using this for a book feels more like a crutch than anything else. That is obviously before you get into whether or not the LLM is actually going to tell you the truth.

There is however one possible use case I could get into, but this is something that could be solved by just finding a video or something online. A refresher when it has been a long time between books coming out in a series.

When I was trying to extract as much meaning out of George Eliot's Middlemarch as I could, I would read a chapter, and then upload that chapter's text file (thank you Gutenberg) to NotebookLM. I'd then have it extract unusual vocabulary words, Latin phrases, and cultural/historical references and compare that list to my handwritten notes from my close reading session. It was fun to have a dialog with the LLM about the chapter, and I felt like I got a lot out of it.

At some point I'll work on better integrating Emacs's nov.el EPUB reader with gptel to approximate something like this. Books are text, and I already have the ultimate text processing environment that I've invested quite a lot of time in.

"By conversing with an AI that fully understands the book's content in real time,"

That seems like a maybe a wee bit of an overstatement of possibilities.

It's interesting that, if this became commonplace, it could be much easier to get value out of poorly written books...

Some people have deep knowledge, but don't have the skills to untangle context and lay out the right learning path for a reader. These people likely bell-curve around certain neurotypes, which perhaps know certain sorts of knowledge more strongly.

Right now, those people shouldn't publish. But if LLMs could augment poorly structured content (not incorrect content, just poorly structured), that perhaps open up more people to share their wisdom.

Anyhow, just thinking out loud here. I'm sure there are some massive downsides that are coming to mind for ppl reading :)

If this looks hard to install (you need to host a server, supabase, docker, lots of python dependancies) - instead, you could just install this to Edge https://microsoftedge.microsoft.com/addons/detail/epubreader... to use it's built in Copilot sidebar to chat about the book. It would work identical to this project AFAICT, except the back end would be Microsoft.

Sounds like a cool project. Not sure if it's one that I would personally use, but I think that LLMs can be used wisely.

What I've found interesting when doing similar experiments (feeding things like books to an LLM and asking questions) is that the output is almost always more bland than one would hope for. I suspect this may both be a result of LLMs being biased for the material they've been trained on and a reality I've suspected which is that the majority of books are mostly filler and aren't making points that are particularly profound. Most books, when you distill them down, fundamentally communicate ideas that are rather obvious, but the language around those points makes them sound a lot more profound than they really are. It's a kind of hypnosis, I think. In a sense, LLMs may be able to reveal how bereft a piece of written material is.

I disagree with the OP's statement that traditional e-readers being passive is actually a "problem". It's kind of like saying that cars are a problem because they can't fly. Maybe I'm being pedantic, but being alone with a book and one's own thoughts is hardly a problem; if anything, the problem is fewer and fewer people are comfortable without a constant barrage of thoughts other than their own.

Just seeing the introduction, I can see this tool hardly adds any value.

The introduction video shows how easy it is to import an epub, and then "asks the ebook" to give them the Table of Contents. While the ToC was already available... no real added value compared to RAG

Haven’t tried the app and not to be disrespectful but I would prefer something like system wide app that can explain the highlighted text anywhere after pressing a shortcut in a tooltip. That way anywhere if you find any word or sentence you want explained you just highlight press shortcut and find out and continue reading, was looking for such app but didn’t find any lightweight single purpose app

Can you look into integrating with private book repositories like Calibre? Also, many books on Archive.org are scanned PDFs (usually as images). This would need OCR probably. Would this work with them ? I feel that this would be most useful with old books where the text is not easily searchable, citations are not convenient, or definitions need lookups.

Nice! I was also thinking about e-reader with LLM support to discuss topics of the book I'm reading with, explain words or phrases (I'm not a native English reader) and so on. But this seems too troublesome to install and does not have a mobile app - I want to read books on the go, not really in front of my computer. Do you think it's possible to make it phone friendly and easier to install?

I built https://readboost.io earlier this year. It adds Q&A to quiz yourself to the end of each chapter and lets you download the annotated ePub. Unfortunately, motivation left me after 3 months and it can't handle all books yet. Anyway, it's free to try if anyone is interested.

Is there any mechanism to prevent the AI from answering with spoilers? For example, when I ask about a character I don't want information coming from pages/chapters I still didn't read.

Disclaimer:

I am not fluent in English, so I have included my comments via a translation tool. Therefore, I may offend you with inappropriate language expressions. I apologize in advance.

wow. I've been entertaining this idea for some time now (Emphasis on "entertaining"). Seeing that already somebody has actually made this makes me very happy.

Will definitely give it a go.

"Too much thought into whether they could, and not enough thought on whether they should" could be the motto of the vast majority of AI products.

Algorithmic social media has already destroyed our attention spans. ChatGPT is in the process of destroying the the rest. People read less than ever and have difficulty engaging with anything that takes more effort than "grok is this real?". Do we really need to put AI into the """reading experience"""?

mega based project, i was thinking about the same. LLMs are very helpful for reading difficult books, or just books requiring previous context. i would add a feature of adding "auto preface" to the book so the reader know what he should know.

Interesting project. I've been thinking about a tool like this; I might be following a multi-volume book series, but it's been years since the last book. When I pick up the latest volume, sometimes there are details that I just can't remember (small details that may turn out important, relationships between minor characters, etc.)

I would just consult a fan wiki, but that doesn't work if the title isn't popular or if the book is too new. This seems like the perfect tool if it can somehow maintain coherency across multiple books.

That said, I do understand (and share) a lot of the frustration and hesitancy that people here have around AI tools; I don't want an app that takes away the act of thinking (like that post recently about teachers using AI to make banal lesson plans, and students in turn using AI to write essays -- what is the point then?). I hope you don't take it too much to heart, and try to showcase use cases where your app can actually provide value.

Another piece of feedback is it would be great if this could be all packaged up into a docker image that would make it easy to deploy on a local machine (or like on a home server/NAS). Right now it seems there are still a lot of manual steps and scaffolding.

Wow literally had this thought yesterday, while reading a book in Britain in the 1920s, so much slang and references to things no longer existing had me searching for things on my laptop more than I’d like. How useful!

I recently finished a book I was looking forward to but wound up really disliking [0]. Throughout the book I found myself saying "wtf?" and I occasionally would complain to my spouse about it. After I finished reading, I went to Goodreads and enjoyed the schadenfreud of seeing how many other reviewers had the same reaction and reactions as me. I do wonder if a tool like this could help me experience something like this DURING the reading of the book rather than after, but I also worry about it lessening the book reading experience, contributing to attention drain, or lessening the experience of finishing reading a book and talking with others about it.

0 - Brandon Sanderson's Wind and Truth

Thank you for sharing this!

I'm disappointed by some of the negative comments. I'm pretty excited about this and plan to try and get this running locally this week. I end discussing a lot of the books I read with ChatGPT, e.g. "hey, this made me think of $idea" or "uh... this is familiar, what am I thinking of?" and to be able to get back a "yeah, check out Ulam's paper on $topic or $book by $author" is really really valuable to me.

I don't read a lot of ePubs (they tend to be mostly PDFs or dead trees) so I might need to adjust a bit, but I'd definitely gonna give something a spin with this.

This feels incredibly dystopian

> The problem: Traditional e-readers are passive.

Why is that a problem? Your statement is a bit like saying "traditional avocados are too delicious. We at YuckCo are aiming to change that!"

You can't just define something as a problem merely to help you sell a solution.

> When you encounter something unclear, you have to context-switch to search for it.

Literally every eReader I've used has a built in dictionary. I tap the word and it tells me what it means.

How is that context switching but "Hey, Siri, what does the word avocado mean?" isn't?