Why is the insane speed of 13KTPS of this site is not more on the the top of the AI conversations?

mike_hearn 11 hours ago | parent | next [-]

Because there's been nothing to discuss since their announcement. Their API access immediately closed due to overwhelming demand and they didn't fab newer models than Llama3 yet.

Probably they will make bank selling to HFT for a while.

▲

Ey7NFZ3P0nzAe 19 hours ago | parent | prev | next [-]

It's pretty well known by now.

▲

chromadon 16 hours ago | parent | prev | next [-]

I asked it for a block of C++ code and it hit 14,189 tok/s. I assume it cached someone else's session?

	▲	fcsp 15 hours ago \| parent [-]
		No - it's custom silicon https://news.ycombinator.com/item?id=48693490

▲

mlrtime 12 hours ago | parent | prev [-]

Because I just tested it and it took 3-4 clarifications before it actually gave a correct response vs gemini/google search. It's not great, but good.

I'd rather wait 3x as long.