Can anyone explain why LLMs are so bad at finding products (their webpages) with given specifications?

You'd think they would have solved it by now.

LLMs aren't architected to handle filter-style comprehensive search without setting them up with additional tools.

Asking an LLM for a list of every county in the USA for example, or every county with a population of more than 100,000 people.

Even if those county names and their populations are mixed up in their weights, the nature of next-token-prediction does not lend them to effectively answering comprehensive, detailed questions like that.

An agent system build on top of an LLM can do it, if it has access to tools which can help access eg a table of counties and then filter them with SQL or Pandas or similar.

▲

amelius an hour ago | parent [-]

Yes, I was assuming they'd use external tools. Using only the raw LLM doesn't sound like a good strategy.

Considering that agents are not a new concept, why isn't this a solved problem by now?

	▲	ACCount37 an hour ago \| parent [-]
		Agents are a very new concept. We've got the early LLM-based AI agents in 2023, and it only became a popular, mainstream thing in 2025 - with Claude Code.

▲

braiamp 2 hours ago | parent | prev | next [-]

Because that's structured data and structured data is usually hidden away from users _and_ machines. Product rarely want to be honest, unless it's B2B in a very competitive market (and even then!). So, yeah, it's not that they are bad, it's that there are few good sources of information.

(Lets ignore for now that no one seems to agree to what should be the spec sheets)

▲

ashu1461 2 hours ago | parent | next [-]

With agentic commerce protocol / unified commerce protocol open ai and gemini are trying to solve this problem.

The idea is to make structured queries using these protocols which can be used to fetch top products matching the user needs instead of just relying on semantic search.

https://developers.openai.com/commerce/specs/file-upload/pro...

▲

Zigurd 2 hours ago | parent | prev | next [-]

It gets worse: shopping agents are hostile adversaries to Amazon unless they're paying Amazon and they've agreed to be friendly agents. No agent that won't betray you to an Amazon pricing strategy is going to be allowed access to Amazon structured data. They might even be fed poisoned data to discredit them.

But you'll be amazed by the abundance.

▲

amelius 2 hours ago | parent | prev [-]

An LLM can read websites, right? And turn them into structured data.

▲

ashu1461 2 hours ago | parent [-]

It can do that on run time, but it does not store data like that. The data is typically stored as embeddings in which it is hard to query data in a structured form. Example give me all products whose price is less than 200$ vs suggest me products for my spouse's birthday.

	▲	amelius 2 hours ago \| parent [-]
		Then they shouldn't store the data as embeddings. Instead: use an LLM to build a large (old-school) database of products with all their specifications. The LLM can also build the schema for that database as it finds more data. Then use an LLM to query that database based on the user's specifications (+ add some intelligence to find nice suggestions for a birthday if wanted, but I'd consider that an extra).

▲

_flux 2 hours ago | parent | prev | next [-]

Amazon Rufus has been mildly successful for me. I think the failures I've experienced with it are mostly because the product I'm looking for doesn't exist in the catalog.

▲

pydry 2 hours ago | parent [-]

It's been an absolute fucking disaster for me. It hallucinates endlessly and its searches are terrible. It even managed to confidently gaslight me about there being a VAT invoice available for a specific product.

I noticed yesterday when browsing on mobile that there used to be a box where I could search reviews and it got swapped with a Rufus box. I guess somebody needs to juice their engagement numbers for an investor briefing.

honestly, Amazon doesnt even need AI it just needs a better UI, more metadata for its products and to make reviews less scammy.

	▲	sgc 2 hours ago \| parent [-]
		I asked a question once and now there is a effing alexa for shopping toolbar that takes a quarter of the screen that will not go away no matter how many times I close it, and the space remains taken even if I adblock it. Absolutely hostile implementation. I have words for this I cannot type out.

▲

wslh 2 hours ago | parent | prev [-]

Because the data, in general, is not included in the LLM model and it needs to search/browse for external information. It cannot look indefinitely so it get the top results from lists, not "evrything".