Remix.run Logo
▲ SketchAgentBench – evaluate LLMs on 3D spatial reasoning(sketchagent.dev)
9 points by wsxiaoys 3 days ago | 7 comments
▲anilgulecha 2 days ago | parent | next [-]

It's only benchmarks where I hear about Google's models being frontier. Never really get to use it because they don't have a usable subscription model on par with Claude/Openai ones in terms of usage.

▲dogma1138 a day ago | parent [-]

Yes they do https://gemini.google/subscriptions/

▲anilgulecha a day ago | parent [-]

These are not equal to the other 2 in usage in amount of inference, like I mentioned. Pain to get work with Pi as well.

▲gorpy7 2 days ago | parent | prev | next [-]

i had astra route some ducting thru joists and beams and other crap and i was so surprised and rejoicing because it knocked it out of the park! this is sketchup pro. i’m not especially good at sketchup. i also had it do some plumbing and again it was good. i wanted proper slope on all the joints and pipes and if i needed to adjust a route- boy was that aggravating! but then asked astra and just sipped my pop.

▲vivzkestrel 2 days ago | parent | prev | next [-]

- can we get a benchmark on game decompilation / figuring out?

▲ 3 days ago | parent | prev | next [-]
[deleted]
▲ 3 days ago | parent | prev [-]
[deleted]