>It hasn’t thought about the problem at all. It’s pattern-matching against its training data and producing the most plausible-sounding response. But it sounds so good that nobody pushes back.

Well, can you prompt it to think about the problem?

> A good architect’s most important skill isn’t designing systems. It’s knowing which systems not to build. It’s pushing back on complexity. It’s asking “why?” five times until the actual requirement emerges from the aspirational nonsense. It’s telling the CTO that their conference-inspired idea is a terrible fit for the team they actually have.

Except for that last one, that all sounds very solvable. Of course, the last one is the most important one. But most humans will struggle there too.