I find this as well for models that make it to production.

On a more fun note, I also find just trying some hair brained idea is now extremely cheap - so why not? I used to have to weigh up spending a week of effort on something likely not to work and now I can now get spit out test in 20 minutes.

The incredible thing to me is I can say "I have a theory that X is a problem."

The agent then works for 20 minutes and comes back with, "Yes this is a problem. Here is the evidence. Here is a golden data set. Finally I built a new evaluation metric and benchmark where we can back test or use for new model experiments to see if we are closing the gap."

I converted an LSD experience into a 4k demo.

I always wanted to build a specific event/sequence in one LSD trip into a small game to give that experience some kind of playability.

Then I saw some 4k demo, wanted to see how good it is in doing this.

Its really fun yeah :)