Isn't search engines quicker than calling a LLM ? It might have a huge impact between a 20ms search engine call and a 2s LLM call for the end user.

▲

quixoticaxolotl 2 hours ago | parent | next [-]

They are already solving the problem with search engines, they're just using an LLM as a first pass to create better embeddings to run a similarity match on first. The difference in latency is likely made up for in accuracy.

▲

fastball 2 hours ago | parent | prev | next [-]

A 2s LLM call is pretty slow.

	▲	gadflyinyoureye 2 hours ago \| parent [-]
		Try using Digital Ocean. Minutes spent on inference.

▲

nullsanity 2 hours ago | parent | prev [-]

[dead]