How many proprietary use cases truly need pre-training or even fine-tuning as opposed to RAG approach? And at what point does it make sense to pre-train/fine tune? Curious.

▲

mirekrusin 8 hours ago | parent | next [-]

You can fine tune small, very fast and cheap to run specialized models ie. to react to logs, tool use and domain knowledge, possibly removing network llm comms altogether etc.

▲

Shitty-kitty 9 hours ago | parent | prev | next [-]

rag basically gives the llm a bunch of documents to search thru for the answer. What it doesn't do is make the algorithm any better. pre-training and fine-tunning improve the llm abaility to reason about your task.

▲

baby 11 hours ago | parent | prev [-]

RAG is dead

▲

charcircuit 11 hours ago | parent | next [-]

Using tools and skills to retrieve data or files is anything but dead.

▲

nathanappere 5 hours ago | parent [-]

I think people just mean "using vector databases to enable RAG".

▲

menaerus 5 hours ago | parent [-]

Even that doesn't make sense. Why would you not build a vector database to complement your RAG engine?

▲

charcircuit 5 hours ago | parent [-]

For coding use cases you may want a way to search for symbols themselves or do a plain text exact match for the name of a symbol to find the relevant documents to include. There is more to searching than building a basic similarity search.

	▲	menaerus 4 hours ago \| parent [-]
		Sorry but who mentioned coding as a use-case? My comment was general and not specific to the coding use-case, and I don't understand where did you get the idea from that I am arguing that building a similarity search engine would be a substitute to the symbol-search engine or that symbol-search is inferior to the similarity-search? Please don't put words into my mouth. My question was genuine without making any presumptions. Even with the coding use-case you would still likely want to build a similarity search engine because searching through plain symbols isn't enough to build a contextual understanding of higher-level concepts in the code.

▲

CharlesW 11 hours ago | parent | prev | next [-]

And yet your blog says you think NFTs are alive. Curious.

But seriously, RAG/retrieval is thriving. It'll be part of the mix alongside long context, reranking, and tool-based context assembly for the forseeable future.

▲

prophesi 5 hours ago | parent | next [-]

Not OP, but...

> Of course you would have to set a temperature of 0 to prevent abuse from the operator, and also assume that an operator has access to the pre-prompt

Doesn't the fact that LLM's are still non-deterministic with a 0 temperature render all of this moot? And why was I compelled to read a random blog post on the unsolved issue of validating natural language? It's a SQL injection except without a predetermined syntax to validate against, and thus a NP problem we've yet to solve.

▲

nl 9 hours ago | parent | prev | next [-]

I don't think RAG is dead, and I don't think NFTs have any use and think that they are completely dead.

But the OP's blog is more about ZK than about NFTs, and crypto is the only place funding work on ZK. It's kind of a devil's bargain, but I've taken crypto money to work on privacy preserving tech before and would again.

▲

elicash 10 hours ago | parent | prev | next [-]

I have no interest in anything crypto, but they are making a proposal about NFTs tied to AI (LLMs and verifiable machine learning) so they can make ownership decisions.

So it'd be alive in the making decisions sense, not in a "the technology is thriving" sense.

▲

strongly-typed 11 hours ago | parent | prev [-]

Wait, what does NFTs have to do with RAG?

	▲	panarky 11 hours ago \| parent \| next [-]
		I, for one, find NFT-shilling to be a strong signal that I should downgrade my trust in everything else a person says.
	▲	LoganDark 11 hours ago \| parent \| prev [-]
		Nothing, I think they're just pointing out a seeming lack of awareness of what really is or isn't dead.

▲

bigyabai 11 hours ago | parent | prev | next [-]

In what, X's hype circles? Embeddings are used in production constantly.

▲

loeg 11 hours ago | parent | prev [-]

Is it??