Anything that requires deep “understanding” or novel invention is not a job for a statistical word regurgitator. I’ve yet to see a single example, in any field, of an LLM actually inventing something truly novel (as judged by the experts in that space). Where LLMs shine is in producing boilerplate -- though that is super useful. So far I have yet to see anything resembling an original “thought” from an LLM (and I use AI at work every day).

There are many LLMs that are producing original "thought".

ESM3: https://www.evolutionaryscale.ai/blog/esm3-release

AlphaProof/AlphaGeometry2: https://deepmind.google/discover/blog/ai-solves-imo-problems...

MatPilot discovering new materials: https://arxiv.org/abs/2411.08063

Then of course NVidia Omniverse with their digital-twin learning.

https://blog.google/technology/ai/google-ai-big-scientific-b...

Experiment: you think LLMs can innovate on chip design? Ask it to do something much simpler: invent a new better sorting algorithm. We use names such as Timsort or Djikstra for a specific reason: because it requires rare human ingenuity to invent such things. If an LLM can’t invent a new sorting algorithm that is meaningfully better in some way than existing known algorithms, then good luck on something much harder like chip design.

Define novel