Remix.run Logo
▲ maherbeg 2 hours ago

Has anyone actually eval'd the other open source options against Jev on real world tasks rather than looking at benchmarks?

I see a lot of people parroting the quick open source alternatives as being better on the benchmarks, but it's such a new category that I'm not convinced we have solid benchmarks.

I'm hoping a company releases an internal eval benchmark for these options. I'm sure some of the open source ones are solid in some cases, but would love to see more reliable data.

▲santiago-pl 2 hours ago | parent [-]

I found this benchmark helpful: https://benchmarkheaven.com/jev-models

▲TN1ck an hour ago | parent [-]

I also benchmarked them here [1] for content moderation. Jev is better than any of them.

[1]: https://tn1ck.com/blog/jevdit

▲maherbeg an hour ago | parent [-]

This is awesome! Please submit updates when a new model comes out that people are saying are better than Jev!