Basically what I’ve been saying since OldJob forced LLMs down our throats and pegging performance to usage metrics: why the fuck are we handing deterministic processes to probabilistic systems when it should be the other way around (using probabilistic systems to design deterministic ones)?

LLMS should be abstracted out of a process as soon as practicable, replaced with deterministic processes or procedures. Otherwise you’ve built the world’s most fragile process at the mercy of token cost, vendor hostility, geopolitics, and model deprecation.

Actually... yes. I was bracing to be very annoyed with your comment starting with "why is everyone using AI so stupid?!" (I know those weren't your words, but it felt like that kind of post)

And then... yeah. You got it exactly right. Once a problem or process is deterministic, that's the wrong application of an LLM.

But I had never quite thought of it in these exact terms. The way I've been thinking about it up until now is that the very best way to use LLMs is to have them produce tools. The tools get to stay reliable and predictable. They boost your performance. But I think you found the more general abstraction of the same idea. Tool-making is not deterministic. But the tools themselves can be. That's why it fits. Trying to stuff LLMs into what's otherwise a deterministic process is an absurd waste and error-prone.

Smart. I like it.

Thats the best description I have heard of the problem so far. I ran into this recently where I automated a ton of stuff and got essentially threatened by leadership for not using AI. My system produces the same output 100% of the time, is free, and scales plus is reliable. Doing what they wanted with an LLM was fragile, didn't always produce the same output and was subject to costs. I don't think they could wrap their brains around it.

> … deterministic processes…

Just to be clear, software development itself is not deterministic, though? The software developer pushes a given business process from less-deterministic toward more deterministic? When we say we’ve “abstracted LLMs out of a process” we’d also say that we’ve abstracted software developers out that process as well?

I think a big part is the misperception that it’s “easier” and less effort to run stuff through LLM than to design an effective deterministic process.

Would love to know how you’ve managed to counter this as the drive to throw everything at LLMs is driving me insane.

This is exactly right. Abstracted out of the process, or to a point of most optimal application.

I love the way you put this. Are there any sites or forums or places where people discuss/hash this out?

I've genuinely never considered it from this angle before.