Remix.run Logo
K0balt 11 hours ago

The training data is not automatically in the context scope, and on list tasks LLMs have nearly no way to ensure completeness due to their fundamental characteristics.

To do a task like this with LLMs, you need to use a document for your source lists or bring them directly into context, then a smart model with good prompting might zero-shot it.

But if you want any confidence in the answer, you need to use tools: “here is two lists, write a python script to find the exact matches, and return a new list with only the exact matches. Write a test dataset and verify that there are no errors, omissions, or duplicates.”

LLMs plus tools / code are amazing. LLMs on their own are a professor with an intermittent heroin problem.