There’s been a meme circulating about how AI doesn’t help because “code was never the hard part.” I think that’s true in some domains, but in others, writing the code absolutely was the hard part. JIT compilers are a great example of that.

▲

asdfsa32 3 hours ago | parent [-]

Anyone who thinks AI is good with writing code that is hard to write for the operator, not due to lack of basic software engineering know how but complexity of the domain, either has access to models beyond what is available to the public or is completely lost.

I believe this because every time I use AI for domains that I consider myself above competent, if it is anything beyond UI components or a simple CRUD endpoints, I cringe at the quality of what it generates.

This has made me to be extremely cautious of starting working in a new domain with AI if I want anything beyond throw away quick hacks or junk, shy of quick bug fixes perhaps.

▲

vouwfietsman an hour ago | parent | next [-]

> I cringe at the quality of what it generates.

Besides all the other mentioned points, I think a good remaining less-discussed point is that quality in software has always been in the eye of the beholder. You may very well see the AI output as low quality, I may not, and its not necessarily clear who is right or wrong, because there was always little precedent in objectively evaluating code quality.

This is a long standing issue, and was never resolved before AI happened, and the coming of AI has not really changed things, except that AI is under a magnifying glass obviously. How do we objectively measure the quality of code? There's some general consensus on things, but surprisingly little is true professional agreed upon consensus.

If I can make up a figure, I would guess 95% of software engineering quality rhetoric, and craftmanship advice, is just strongly held opinions.

This is not something I can prove, but if I look at the (still ongoing.....) debates on very basic ideas like clean code, and the reactions from also-great programmers like Carmack, Blow & Muratori, it is clear to me that there is little consensus on even the fundamentals of software design.

If all these people can produce excellent working software while disagreeing on these fundamentals (of quality), it means we do not yet understand what the fundamentals are.

	▲	asdfsa32 29 minutes ago \| parent [-]
		https://en.wikipedia.org/wiki/Relativist_fallacy

▲

mgaunard 2 hours ago | parent | prev | next [-]

Fable (and even Opus if kept tightly under reigns) does generate high-quality code even for highly complex tasks.

It generally performs better if the tasks are broken done into small manageable pieces, and the person is actually reviewing and calling out problems, which usually requires the person to be a competent engineer in the problem domain to begin with.

But yes, I have personally used it to build what the OP calls a JIT. I would usually write that by hand and it would take me one week. The AI does it in an hour.

▲

asdfsa32 2 hours ago | parent | next [-]

You can't make assertions about "quality" of code that was generated under an hour while it would have taken a human 40 hours, unless you put substantial amount of work into reviewing it.

I used Fable on a Zephyr project with time sensitive code for LR-WPAN and it broke everything. Literally made the code worst to the point that the devices stopped connecting.

▲

Mawr an hour ago | parent | prev [-]

Your second paragraph invalidates your first.

If I need to be a domain expert anyway, the value of the tool goes down by orders of magnitude. Same if I need to first break the task down into pieces and keep reviewing all the output. That sounds to me like >80% of the work I'd need to do anyway.

If I need to design and understand all of the code anyway, I might as well skip the whole process of repeatedly fixing the subpar-at-every-level LLM output and write it all myself.

Personally, I've found the greatest value in asking for simple tasks, like wiring up APIs, generating boilerplate, bug finding etc. Anything that requires effort to do but results in either very little or very simple output, so that I can easily verify its correctness.

But give the LLM anything remotely complex to generate and it cakes its pants.

	▲	cseleborg 32 minutes ago \| parent [-]
		That is my experience as well. I get a (subjective) speedup between 1 and 3 for parts of the code I'd consider critical and where I check the output tightly, and 5-20 for menial work OR for important code that's well isolated into its own module such that its quality doesn't matter because I can have it rewritten easily if it doesn't work as expected.

▲

grebc 2 hours ago | parent | prev | next [-]

Three quarters of my CS class at university could barely code and/or understand code.

That’s not a joke. A lot went on to be programmers professionally. And judging by the quality of closed & open source code I witness daily those figures from university accurately depict people’s capabilities.

Now that said, if you can’t really code then using AI will be a godsend to said individuals.

	▲	mgaunard 2 hours ago \| parent \| next [-]
		I'd say that is the problem we're observing. A lot of decent code is being written with weird inconsistencies, because it's actually written by AI driven by people who don't really understand what they're doing. the tell-tale mark of AI code is highly over-engineered local solutions to trivial problems that don't matter, or that were already solved better elsewhere and that no sane human would ever duplicate.
	▲	asdfsa32 2 hours ago \| parent \| prev [-]
		Fair, but I fear that now even more people who can't code will code, and code that is not any better than what people who could barely code write. Growing cabbages starting to look more and more interesting.

▲

IshKebab an hour ago | parent | prev [-]

When did you last try? They've improved a lot.

Also often the difficulty with writing code is simply knowing where to start - getting past the blank page. AI can help a lot with that. Often there's a task where I've got kind of writers block, but you can ask AI to do it and suddenly it's like "ah yeah, sort of but actually that's not quite right we should do it this way".