It seems we could use a new kind of memory that streams the weight data in, like GDDR in reverse.

0x457 12 hours ago | parent | next [-]

rzzzt 7 hours ago | parent | next [-]

Optane? (Too soon?)

	▲	0x457 7 hours ago \| parent [-]
		Optane was targeting the latency gap, while HBF targets bandwidth. Given how LLMs work, HBF is perfect for offloading.

interesting!

yes! I guess future hardware designs will have something like that!