The fact that OpenAI documents theirs is already a big improvement over Anthropic. But, also, the OpenAI tokenizer got more efficient when they last updated it, rather than less. https://mdstudio.app/o200k-base-tokenizer

▲

alansaber 6 hours ago | parent | next [-]

Interesting. New models are estimated at ~5T params, so 45,000x increase over BERT base (110m). But vocab size of 200k, so only an increase of 7x over BERT base (30k).

▲

satvikpendem 3 hours ago | parent | next [-]

Interesting that Grok 4.5 nears or exceeds OpenAI and Anthropic models in some benchmarks at only 1.5 trillion parameters per their announcement post.

▲

kamranjon 6 hours ago | parent | prev | next [-]

Where do these estimates come from?

	▲	cyanydeez 5 hours ago \| parent [-]
		a dark anal cavity somewhere next to marketing.

▲

vlian2088 2 hours ago | parent | prev [-]

BERT is not an LLM, it's an encoder-only model, i.e. it doesn't generate new text. the first somewhat useful publicly accessible LLM was GPT-3 with 175B params, and it was also the last frontier model whose parameter count was disclosed.

▲

ianberdin 6 hours ago | parent | prev [-]

[dead]