It seems like a stronger story for robotics, since smaller models can always react to the environment faster than large models at a given hardware budget. Also because robots that keep their models local for latency or reliability aren't going to be carrying many kilowatts of inference capacity.

▲

lumost 2 hours ago | parent [-]

remote inference should be sufficient for most robotics applications with potentially a small model for safety critical actions running locally.

Unless you are in military robotics or automotive of course :)

	▲	AlotOfReading 2 hours ago \| parent [-]
		There are many, many factories that still don't have internet access on the floor, and commercial inference generally has response latencies measured in seconds. I struggle to imagine a factory spending hundreds of thousands for the local compute to run a large model either, given how cheap they are about expenses. I'm also skeptical that you can cleanly differentiate between "safety critical actions" and "actions", though this is less of a practical concern given how laissez-faire some manufacturers are. For context, I work on safety critical robotics (in automotive).