We have run it on DeepSWE, check out our blog post for details of how we leverage the intelligence of the smart models https://usetokenless.com/blog/building-tokenless/

Ah I didn’t see the tabs in the benchmark chart. Performance is worse on the two coding benchmarks and cheaper which makes sense.

I did read the blog post and I’m not sure what you mean by your reply. I don’t think the blog post addresses my comment.