"It’s about attention and understanding. To keep my attention, I must go beyond ‘read code’ like a passive observer of agents from afar. To really connect with the architecture of the system, it helps to truly experience the code"

I guess the funny answer that is behind this sentence is: You have to train your own mental model. We always argue about code in a very abstract and logical manner. But when coding the subconsciousness makes most of the decision ("this just feels right"). But for this to work you have to train it. And this does only work in a very limited way with code reviews or reading documentation. It requires repetition and deep focus.

When there is an issue in production with this mental model you will be able to point to the cause of an error message instantly. With generated code you'll search for a long time with your slow, conscious part of the brain.

For LLMs to be really helpful, they have to take over complete maintenance of the code. So you can treat them like an external library: Just assume it works. Otherwise this will always be problematic.

▲

jurgenburgen 3 hours ago | parent | next [-]

> For LLMs to be really helpful, they have to take over complete maintenance of the code. So you can treat them like an external library: Just assume it works.

We already tried this with humans. It works so poorly that it got the derogatory name “ivory tower architect”. It usually results in theoretical designs that are unworkable in the actual system, implementation teams (or LLMs) that work around the architecture and a lot of slowing down of velocity as the architect and implementers argue past each other.

▲

zelphirkalt 2 hours ago | parent [-]

This happens when the architect is out of touch. If the architect themselves works on the code, writes code, deals with the imposed restrictions, then the chances of that happening is much lower. Assuming, that they are a good architect.

	▲	jurgenburgen 2 hours ago \| parent [-]
		I agree, if the architect participates in the implementation then they avoid this anti-pattern. That’s not compatible with hands-off autonomous agents where you treat implementation as a black box.

▲

kqr 4 hours ago | parent | prev [-]

> With generated code you'll search for a long time

The observability people will claim that if the dynamic runtime behaviour of your system makes it hard to find the source of a behaviour, your system must be made more transparent and observable. They would also claim this was always the case -- we should never have relied on people's mental models being amazing because people move around.

(I don't know yet where I stand on this but I'm trying to learn more.)

▲

nmehner 3 hours ago | parent | next [-]

If it was only "my" system without any integrations, I might agree.

But currently e.g. I am working on an MES/Scada layer that integrates data from a load of different machines in a factory. These machines are from China, Korea, Germany, Sweden ... Upwards there is an ERP integration (and some other systems).

Sometimes machines are updated and suddenly behave differently. Giving error messages in Chinese.

The ERP has the nasty behavior of returning error messages where it is not clear whether the actual processing actually happened or not. There are some heuristics on parsing the error messages, but these also change with new versions.

Sometimes one machine overloads cloud infrastructure and completely unrelated functionality fails.

Sometimes the on-premise network stops working for whatever reason and data is lost.

Sometimes operators do not understand a perfectly valid error message like: "The batch you loaded into input position XY has expired on XZ and cannot be used for production": "But we have been told to use it..."

So when you get called out at night, because the production line stopped and "MES is displaying an error message", it is mostly about finding out what integration failed and who else to wake up. Getting this right is very much appreciated by your colleagues.

And this is where you need a mental model of how things are connected, what error message happens because of what external causes etc.

Observability can only work perfectly for known problems. In a complex system for unexpected problem you can either provide too much data, so analyzing it and finding the relevant part becomes really hard, or too little data which makes finding the issue impossible.

There are so many companies claiming to provide the perfect observability solution and there are certainly solutions that help. But it is all very far from perfect.

Not relying on people is managers wet dream. And for a lot of people it might be true that they can be easily replaced. But for complex systems there are always some key people that you cannot replace without causing issues.

	▲	b112 an hour ago \| parent [-]
		And here's the thing... juniors become seniors become experts, by doing this their entire career. By having an understanding built during their entire career. Right now we live in a fairly-land of mixed capacity. LLMs being used in parallel with skilled people. But as time progresses, there will be no more skilled people, because no one will learn and develop those skills. If you're in the world of LLMs now, you are basically completely stalled in your personal growth in this field. You will never improve, and some seem to say they lose capabilities as they rely upon LLMs. The world always changes. But the decisions being made today, are being made by skilled people. What will the world look like, when it's just all "bro, lol, just tell it to make your thing" and then done?

▲

hack1312 4 hours ago | parent | prev [-]

The observability people are correct. It’s not either-or though.