| ▲ | aabdi a day ago | |
Evals are tests of model accuracy. For example here are 100 people's sample houses, go test if the agent can turn off the lightbulbs correctly. Right job sure. But, If you're replacing the LLM with a jev like as in the article, or switching with a smaller classifier, then you better have good data/evals otherwise you're reducing accuracy without you even knowing. | ||