I wrote something that touches on a few of the same points not too long ago (namely vendor lock-in and inability to project cost with LLMs at the wheel). Tangential topics but I think there's a healthy enough amount of intersection to at least join in on the discussion.

I do agree that if we just rely on AI for all outputs and some reviews (at least to a threshold, because we simply can't keep up with the AI throughput as humans) we will eventually have skills atrophy. Here's where the tangents intersect: I've been working on a way to have the best of both worlds. We can still use AI to generate a large swathe of code, but use good old software engineering to do it. My project (https://salesforce-misc.github.io/switchplane/) inverts the control. Rather than having LLM-as-runtime and doing all the things, you define and write LangGraph control flows that only use the LLM when judgement is actually required. The basic principle is:

If it's deterministic, write it in code. If it requires judgement, use the LLM.

Switchplane itself is local-only but the principles can be applied to deployed agentic services as well. Because the approach is code-first, we can have that vendor independence: Use whatever model you want anywhere in the graph. One goes down? No problem. Swap the config without impacting the overarching control flow.

Cost becoming a factor? Limit LLM loops or constrain their access however you want. It's just code that needs to be updated. You control the runtime, not the LLM.

Concerned about non-deterministic behaviour when you need determinism? Don't be. It's in code.

Worried about skills atrophying because we're handing off everything to an LLM? That's mitigated somewhat here because you still need to think in systems in order to build execution graphs in the first place.

It might not demo as well as a number of markdown files being executed by an LLM. It's definitely a more reliable approach in the long run though.