Missing a chance to note (or configure for?) installation on a Raspberry Pi --- that'd make an affordable option to leave powered down, but ready to go in an EMI-shield/Faraday Cage.

▲

pdpi 4 hours ago | parent [-]

They specifically state that they’re aiming for a “fatter” model that expects higher-end hardware, and other projects like Internet in a box already target rpi-style devices.

	▲	moffkalast 20 minutes ago \| parent [-]
		I think there are technically some 3 bit byteshape quants that are aimed specifically at running up to 30B MoEs on the 16GB Pi 5, so it would be possible to do something reasonably fat at very low speeds and extremely short contexts (like 4k maybe). One of those 32 or 64GB Rockchip based boards would do better, but there's rarely usable software to go along with them. An industrial grade Jetson Thor would probably be the ultimate platform for this if you ignore the money part.