Remix.run Logo
▲ ramoz an hour ago

The sentiment on x right now is interesting.

The influencers who get early access are using words like "worst", "terrible", "disappointing"

  - https://x.com/danshipper/status/2122754729012777052
  - https://x.com/mattshumer_/status/2122752192226406421
  - https://x.com/theo/status/2122757647170703550
▲RobinL an hour ago | parent | next [-]

Ethan Mollick too: https://bsky.app/profile/emollick.bsky.social/post/3n2joens5...

▲suddenlybananas an hour ago | parent | prev [-]

>`ts-rust` (rewrite of ts in rust) is a project I started to bench new OpenAI models during early access (ty for the unlimited tokens btw).

>No OpenAI model could even break ground on this, even after $300k+ in tokens were spent on it.

>I decided to throw Opus at it, and it was done in under 2 weeks for ~$20k in tokens.

>I was really, really excited to throw the new Astra at this to see if it could burn down some of the gap between ts-rust and other rust rewrites.

>It ran for 7 days and landed 0 improvements. It got super fixated on the "also make it smaller" part of the request, and ended up making the compiler 56% SLOWER.

I don't understand this at all. Is he just throwing different agents at vague open ended problems without any oversight and then getting upset when they don't do much?

I'm not exactly llm-pilled but surely even with a pretty mediocre model you could get something out of 300k spent.