If all frontier LLM labs agreed to a truce and stopped training to save on cost, LLMs would be immensely profitable now.

That isn't what I've seen: https://www.wheresyoured.at/oai_docs/