But mining all the tracking data in order to show profitable targeted ads is extremely intensive. That’s what kicked off the era of “big data” 15-20 years ago.

Mining tracking data is a megaFLOP and gigaFLOP scale problem while just a simple LLM response is a teraFLOP scale problem. It also tends towards embarrassingly parallel because tracks of multiple users aren't usually interdependent. The tracking data processing also doesn't need to be calculated fresh for every single user with every interaction.

LLMs need to burn significant amounts of power for every inference. They're exponentially more power hungry than searches, database lookups, or even loads from disk.