I'm curious about that too.

I'm building a language with AI, and when I leave the AI running on its own, it keeps generating code in strange directions. I wonder what kind of structure AI-only code would have.

▲

lonelyasacloud 4 days ago | parent | next [-]

> I'm building a language with AI, and when I leave the AI running on its own, it keeps generating code in strange directions.

In the linked paper the researchers state "AI solutions must match the original program’s output exactly on end-to-end tests, including held-out tests. " And a bit further on "The AI agent is given execute-only access to the original program and a set of visible test cases, but does not have access to the original source code. "

At a guess, their agents are not going off into the weeds (like mine do occasionally) is because they have both access to a reference runtime implementation to use as a definitive end to end test to guide development. And also a relatively large token budget to run enough characterisation tests to allow them to home in on solutions.

▲

onlyrealcuzzo 4 days ago | parent | prev | next [-]

Also building a language with AI.

It's decent at design and implementation, but strangely terrible at language consistency - and expectedly architecture.

Steering it on language consistency is easy. Steering it on architecture has required me to learn a lot more about compilers than I wanted to do for this project (since that's a lot more than I wanted to take on for something that was supposed to be a "fun" "side" project).

	▲	jdw64 4 days ago \| parent [-]
		I have a similar problem. I'm not sure about architecture. When I first implemented the slot system in my language, I designed it around that implementation, and in the end, the architecture suffered. There were quite a few cases where consistency was simply unavoidable. It's not easy to create a completely new architecture with something that doesn't exist in existing languages. Consistency doesn't come together as easily as you'd think. In that sense, I really admire the people who first introduced something new. When I started implementing it, I focused on one killer feature and just went for it—with the feeling that if this succeeds, my language will succeed. But once I started building around it, it turned out to be very different from other compiler architectures. With CRUD or other well-established domains, it's easier to maintain consistency. But achieving internal consistency in a program seems to be a different kind of challenge.

▲

ivan888 4 days ago | parent | prev [-]

Is it like self driving cars, where 100% non-human code would be arguably safer?

But then it always comes down to having some human who is accountable for verifying the behavior. I think ultimately we just need a better set of languages and frameworks to minimally encode the types of applications we want to build today, so that human comprehension and verification of the source code becomes easier.

▲

jdw64 4 days ago | parent | next [-]

I've noticed that when only AI writes the code, there are cases where it ends up being unusable by humans. For small code snippets or function-level units, AI is often much better than me. But when I look at the whole picture, AI tends to perform poorly.

▲

LetsGetTechnicl 4 days ago | parent | prev | next [-]

How would 100% non-human code possibly be safer?

▲

micromacrofoot 4 days ago | parent [-]

well if driving is supposedly safer when automated, why not code

▲

dd8601fn 4 days ago | parent | next [-]

The qualities of computation that make autopilots safer than human drivers aren’t the same qualities that produce better designed software products.

	▲	micromacrofoot 2 days ago \| parent [-]
		to err is human

▲

ModernMech 4 days ago | parent | prev | next [-]

It’s supposedly safer than the average driver, not safer than professional drivers.

	▲	micromacrofoot 2 days ago \| parent [-]
		the average american spends 60 minutes a day driving, that's thousands of hours by the time you reach 25... at what point is that not a professional level of experience?

▲

LetsGetTechnicl 4 days ago | parent | prev [-]

Is it actually safer? If you can't understand the code you can't verify it's safety, and AI code often has bugs in it. You can't trust implicitly.

	▲	skeledrew 3 days ago \| parent \| next [-]
		You don't need to understand code to verify its safety if you have a good QA framework.
	▲	micromacrofoot 2 days ago \| parent \| prev [-]
		Same applies to driving cars

▲

roncesvalles 4 days ago | parent | prev [-]

Category error. Self-driving cars are still meticulously coded by humans.

	▲	fragmede 3 days ago \| parent [-]
		They use trained models in the system though, so I don't know if you can really claim that.