Remix.run Logo
▲ aabdi 3 hours ago

Two things, my problem isn't with the probabilistic nature, its:

1. with the quality of the data that customers use to eval.

2. with the lack of rigor/ops/testing around accuracy for deployment and regression.

I'll give you examples for failures IRL:

Offline:

- sample distribution for test data didn't have p99 customers in it, failed in production when it launched for those customers.

- test data sets were all synthetic, launch to real customers fails because data set was not representative of real.

Online:

- yeah we used LLM as a judge in prod/online eval, but the judge sampling failed to measured against our japanese customers, our bilingual customers, customers with media devices etc.

- the LLM as a judge was used, but there was no measure of human accept rate measures/alarms for various use cases (lightbulbs turning off, music playing, etc) so the lightbulbs started failing.

- there was no consistent human sampling of online data, so for example samsung TVs stopped working, but no one caught it until 3 months later when some VP was watching his TV and it broke and was accelerated despite it affecting 2-3% of customers.