Artificial Analysis recently released a dedicated 3D benchmark, where Astra lags behind both Opus and Sonnet 5.5 (and yes, open weight models seem to be quite far behind).

https://artificialanalysis.ai/articles/aa-3d

Interesting!

For me it's large files in Blender (in some cases >400MB), mostly, where Astra excels. I gave it a look, but I don't know if the benchmark encompasses that sort of thing.

.step files are simpler pieces of straightforward code, so it makes sense that Opus and Sonnet 5.5 do very well...

its unfortunate they didn't include deepseek in the initial batch of benchmarks