| ▲ | refactor_master 4 hours ago |
| Here’s an even simpler take: just embed everything the first time, then track what was changed. Use a cheap model to summarize and clean up the documents/chats with summary and keywords. Unless you have entire libraries of books to embed it’s going to be a few hundred dollars of API calls. Then, throw it all in BigQuery. Handles all the vector stuff natively. Sprinkle an agentic bot UI thing on top to make it appear all-knowing and magical. I assume other vendors than Google have a similar batteries-included approach you can just plug in. |
|
| ▲ | usernametaken29 3 hours ago | parent | next [-] |
| > embed everything the first time This assumes your text is small. Try embedding pdf reports - though luck. It surely won’t fit into most embeddings. I can think of many more examples: books, news articles, medical reports, insurance claims etc. they’re all too big to “index it all at once” |
| |
| ▲ | khalic 2 hours ago | parent | next [-] | | you won't get anything out of a whole book embedding anyway, even a structured page is too much | |
| ▲ | robrorcroptrer 2 hours ago | parent | prev [-] | | What about splitting bigger content into chunks before embedding? | | |
| ▲ | freakynit 2 hours ago | parent | next [-] | | How are you gonna handle the relations that span across individual chunks... if a later chunk refers something from 2 chunks before using `it`, rather than proper name, how will you handle that? Because at query time, that later chunk would not match. | | |
| ▲ | gf000 3 minutes ago | parent | next [-] | | Absolutely a novice in this topic, but I would imagine that by simply having sufficiently big chunks it's simply not a problem? You surely have enough information in like a couple of paragraphs to denote in vector space roughly what it is about. So that both chunks would get found by a vector search, and then whatever is the logic it may put the whole original text of those chunks into context, but in any case enough so that an LLM can "reason" about the references in-between the two. | |
| ▲ | refactor_master 14 minutes ago | parent | prev [-] | | Humans usually have ways around that in longer documents eg page numbers, paragraphs, links. If someone gave me a report, in my hands, that said “see ‘it’” I’d also be confused. |
| |
| ▲ | mdp2021 39 minutes ago | parent | prev [-] | | What member freakynit said nearby about chunks and relations between chunks, plus the storage and information efficiency problem: make some calculations about storing vectors - for paragraphs and for collections of paragraphs -, then compare the needed space with the original data... Because you could have clever ideas about vectors related to more paragraphs related in the document structure - but that would multiply the vectors. The index can become much bigger than the corpus. |
|
|
|
| ▲ | cpursley 4 hours ago | parent | prev [-] |
| Yep, lock into some vendor from day 1. Great idea! |
| |
| ▲ | orisho 3 hours ago | parent [-] | | Vendor lock in is 2025. Porting became trivial with LLMs advancing like they have. | | |
| ▲ | cpursley 2 hours ago | parent [-] | | What I'm saying is pick transportable tech from day 1 so you can easily move if they shut down, hike prices, decide they don't like you, etc. |
|
|
