this is cool but like, are we just vibe coding NAND burners at this point? these decode times don't really tell the whole story, because prefill becomes the bottleneck.

half an hour to process 10k tokens on an M5 seems... not great

▲

fsuts 3 hours ago | parent | next [-]

This is how progress happens, someone gets to 3t/s, the next person gets to6/s and eventually we get to 100t/s.

People like this person are laying the foundations.

	▲	IsTom 2 hours ago \| parent [-]
		There is a limit how much you can squeeze out of given hardware. Betting on it being closer to 100t/s than 6t/s is only that, a bet.

▲

kennywinker 5 hours ago | parent | prev [-]

Not great for coding, or realtime agent interactions. But for background processing tasks overnight? Seems like it’d work pretty well

▲

selcuka 3 hours ago | parent [-]

I'm pretty sure one can rent a GPU for a few minutes with the electricity cost of leaving an M5 overnight.

	▲	hdgvhicv 2 hours ago \| parent \| next [-]
		Domestic electricity is free nowadays, certainly for most of the year, as solar plus battery covers your usage for a tiny percentage of the cost of your house.
	▲	kennywinker 2 hours ago \| parent \| prev [-]
		Sure. One could. But then one wouldn’t be in control of every step of the process.