That may be true now, but think about how far we've come in a year alone! This is really impressive, and even if the models don't improve, someone will build skills to attack these specific scenarios.

Over time, I imagine even cloud providers, app stores etc can start doing automated security scanning for these types of failure modes, or give a more restricted version of the experience to ensure safety too.

▲

afavour 3 days ago | parent | next [-]

There's a fallacy in here that is often repeated. We've made it from 0 to 5, so we'll be at 10 any day now! But in reality there are any number of roadblocks that might mean progress halts at 7 for years, if not forever.

	▲	christophilus 3 days ago \| parent [-]
		Even if progress halts here at 5, I think the programming profession is forever changed. That’s not hyperbole. Claude Code— if it doesn’t improve at all— has changed how I approach my job. I don’t know that I like this new world, but I don’t think there’s any going back.

▲

usefulposter 3 days ago | parent | prev [-]

This comment addresses none of the concerns raised. It writes off entire fields of research (accessibility, UX, application security) as Just train the models more bro. Accelerate.

	▲	maherbeg 3 days ago \| parent [-]
		Both accessibility, and application security are easier to build rules + improved models for because they have pretty solid constraints and outcomes. UX on the other hand is definitely more challenging given how much of it isn't quite codified into simple rules. I didn't write off an entire field of research, but rather want to highlight that these aren't intractable problems for AI research, and that we can actually start codifying many of these things today using the skills framework to close up edges in the model training. It may not be 100% but it's not 0%.