Needing the right scaffolding is the problem.

Today I asked 3 versions of Gemini “what were sales in December” with access to a sql model of sales data.

All three ran `WHERE EXTRACT(MONTH FROM date) = 12` with no year (except 2.5 flash did sometimes gave me sales for Dec 2023).

No sane human would hear “sales from December” and sum up every December. But it got numbers that an uncritical eye would miss being wrong.

That’s the type of logical error that these models produce that are bothering the author. They can be very poor at analysis in real world situations because they do these things.