I've seen that mentality and gone to bat to convince a boardroom that it's the wrong approach, when people were star-struck by the possibilities. Luckily I'm in a position as CTO of a (very non-tech, brick and mortar) company that entrusts me to manage their budget for new features, and prevent erosion of our software/logistics over the long term. And I've come down decidedly on the side of not having LLMs fuck with any schema or architecture changes or anything in the codebase that would touch upon business logic. When your code actually encapsulates business logic, which is often counterintuitive and full of weird exceptions, 90% of the code work is done by prior planning to map out all possible branches and the algorithms to assist employee decision making. The 10% that's actually writing code needs to be done by someone who understands the entire stack and business model perfectly. Some nice HTML/CSS fluff here and there is great to hand off to an LLM, and you don't need frontier models for that.

I shot down similar arguments in favor of outsourcing overseas for years. Outsourcing any critical logic to an LLM is even worse.

The sad part is that you work in FINANCE of all things and this happens there.

Like: What competence do decision makers in FINANCE have, when they are this oblivious to economics?

The thing with language models is they are tailor made to fool managerial types into thinking it’s the holy grail.

Just like many managers, the appearance of productivity is all that counts. And LLMs shine at giving the appearance of having solved all of the managers problems, and all they have to do to use it is spend on tokens.

This isn’t to say that LLMs aren’t truly useful, they absolutely are. But they’re very nature is one of simulating intelligence through next word prediction.

The chat modes and models are by their nature supremely attractive to management layers, because they give answers that sound so damn plausible even when they are complete fictions, and uttered with such confidence how could they be anything but the singularity.

Everyone is falling over themselves out of FOMO.

The market is priced at expecting AGI levels of breakthroughs. Just a very useful tool for programming is definitely not enough to keep the music playing.