Remix.run Logo
nl 4 hours ago

Trinity Large Preview managed 17/25 on my agentic SQL benchmark: https://sql-benchmark.nicklothian.com/?#all-data which is a fairly mediocre score for a large model (Qwen 27B managed 23/25)

This non-Preview release scored 16/25. Probably the same model as the preview, or at least not particularly improved if you want agentic performance.

Good to see more options for large open models though!

It's hard to point definitively to a reason it underperforms but generally models that perform well at agentic tasks were trained on very large numbers of tokens (Qwen, frontier models) or were heavily post trained for reasoning (see eg Nemotron-Cascade-2-30B-A3B at 21/25 vs the base model Nemotron-3-Nano-30B-A3B-Base at 12/25 )

anon373839 2 hours ago | parent [-]

Bit of a tangent, but I'm pleased to see that Qwen 3.5 35B is tied with GPT-5.4 and just 2 points behind 4.6 Opus. That little model is so impressively capable and fast! I'm frequently still surprised that I have that level of capability and speed running locally on my laptop.

nl 2 hours ago | parent [-]

Nemotron-Cascade-2-30B-A3B is worth checking out too at the size - I found it even better than Qwen 3.5 35B! I ran it slowly but successfully on a 8GB 1070GTX with CPU offload.

anon373839 2 hours ago | parent [-]

Thanks for the tip! Hadn't seen that one.