Interesting. New models are estimated at ~5T params, so 45,000x increase over BERT base (110m). But vocab size of 200k, so only an increase of 7x over BERT base (30k).

Interesting that Grok 4.5 nears or exceeds OpenAI and Anthropic models in some benchmarks at only 1.5 trillion parameters per their announcement post.

Where do these estimates come from?

BERT is not an LLM, it's an encoder-only model, i.e. it doesn't generate new text. the first somewhat useful publicly accessible LLM was GPT-3 with 175B params, and it was also the last frontier model whose parameter count was disclosed.