Remix.run Logo
▲ semiquaver a day ago

Take this as genuine advice, not criticism: The fact that you felt the need to “characterize” the jank and verbally describe it is evidence of your mistake. The only way for these new development models to function is to create a tight machine-only validation loop. Every aspect of an LLM-generated text editor must be drivable fully by the LLM itself so that it can independently observe the problems and fix them (and so those observations can be mechanically translated to regression test cases)

LLMs won’t always guide you to do this correctly so you need to understand how important it is from the start and always enforce it.

▲qlte a day ago | parent | next [-]

I'm not seeing any indication that OP didn't do that. The jank they're describing sounds exactly like something that could slip past the harness' validation loop (weird intermittent graphical anomalies).

▲semiquaver 20 hours ago | parent [-]

Then the equivalence I refer to is not in place.

The software’s behavior that a human can observe must be observable by the agent. Any interaction the human can perform should also be performable by the agent.

If the human can notice jank that the bot cannot or does not for the “same” interaction, that directly points to the next thing that should be fixed before continuing to add additional features to the software under development.

▲jaykru a day ago | parent | prev | next [-]

I appreciate the friendly tone :-) and I mean this not as a jab: I think maybe you missed the point of my comment above, which wasn't that one can't build functional software with LLMs, but rather that it requires the diligent involvement of a technical worker.

> Barring an architectural paradigm shift we will still need people with strong engineering and design instincts, and even more importantly the ability to write reference code and formal specifications codifying desired behaviors.

The people building those specifications and the tight validation loops you describe are the technical software workers of tomorrow.

▲PhilipRoman a day ago | parent | next [-]

I generally agree with your idea, however you can also say "Hey Claude, I want to vibe code XYZ, set up an AI friendly automation environment" and it will have no problem throwing something together with qemu, etc. And of course it doesn't have to be you saying that, it could be a N+1 level agent, who loaded his /ai-project-management-best-practices skill.

▲semiquaver 20 hours ago | parent [-]

Right. Draw the line out from where it’s pointing right now and there are only so many heirarchical supervision levels until you reach “all”.

Human brains are just going to seem incredibly expensive for their relative capability level. It only takes a few doublings for the comparison to become overwhelmingly against us.

▲semiquaver 20 hours ago | parent | prev [-]

With absolute, genuine sincerity: I hope with all my heart and soul that you are correct.

Best, friend.

▲titzer a day ago | parent | prev | next [-]

Yeah, this is really important. Having a tight test loop is already great for humans, but doubly-so for agents. A clunky manual process sucks for humans and agents alike.

▲asdfman123 a day ago | parent | prev | next [-]

I find LLMs produce very good results if you just prompt it into good software engineering practices. Write thorough unit tests, minimize diff noise, try to re-evaluate all work to see if it can be done more elegantly and simply. Then you can run an adversarial review agent to catch any bugs it missed.

Same goes for AI slop comments. I can just prompt AI into more or less writing like me.

▲groestl a day ago | parent | prev | next [-]

> always enforce it.

Here be lions, since these things tend to lie to you or themselves when skipping the enforcement part.

▲semiquaver 18 hours ago | parent [-]

Sure. Step outwards: slap an adversarial agent on top of the loop, tell it to actively search for the gap you describe and fix it. The only problem then is it will find so many gaps

▲the_real_cher a day ago | parent | prev [-]

I'm not sure you're correct.

If the software was capable of creating high complexity software I, or anyone, should be able to replace and build anything I want with just some prompts and guidance.

Anyone who has worked with LLM heavily knows this is far from the truth.

▲dboreham a day ago | parent | next [-]

What thing that you want have you failed to build?

▲the_real_cher a day ago | parent [-]

A new operating system?

▲ 17 hours ago | parent [-]
[deleted]
▲semiquaver 17 hours ago | parent | prev [-]

  > I should be able to […] build anything I want with just some prompts and guidance.
…Yes? You are able to do this. That’s why a lot of people are excited. The fact that agents can do a thing does not mean that you will never need skill and taste to guide it in the right direction.

No one ever claimed it was a magic button without fault. Few things are. This is “only” a new form of cognition.

And god knows I don’t find all the human cognition I encounter in my life to be the image of perfection…