> It's really hard to trust anything public

Just feed it into an LLM, unintentionally hint at your bias, and voila, it will use research and the latest or generated metrics to prove whatever you’d like.

> The only true tests are the ones you write yourself, never publish, and only work 100% on open models.

This may be good enough, and that’s fine if it is.

But, if you do it in-house in a closet with open models, you will have your own biases.

No tests are valid if all that ever mattered was the argument and perhaps curated evidence.

All tests, private and public tests have proved flawed theories historically.

Truth has always been elusive and under siege.

People will always just believe things. Data is just foundation for pre-existing or fabricated beliefs. It’s the best rationale for faith, because in the end, faith is everything. Without it, there is nothing.