My management is pushing for us to come up with ideas on where we can use LLMs in our product. The whole team has been very resistant for this exact reason. Anything we can think of will only make things worse, and we’ve already been told anything above a 1-2% failure rate is unacceptable. If anything we need more structure and standards to hit that, not less.

I believe that llm’s can be used to re-imagine experiences but it’s definitely not the way people think. The constraint is imagination and thinking about complex trade offs more than anything else. Which is the essence of innovation.

The agent paradigm will eventually give way to experiences that are a hybrid of deterministic and non deterministic and you won’t even know the llm was involved or visible.

We just got dropped into hackatons for having ideas a few weeks ago, AI at all costs, similar feeling.

Luckily for programmatic or logic following, smaller models tend to do better, it can be surprising at first to see the more expensive models do worse at a task but it’s true.