| ▲ | Tuna-Fish 2 hours ago | |
GPU-style platforms (lots of compute connected to a big shared pipe that connects to a large pool of fast memory) are inherently more efficient for centralized inference. Local inference gets truly interesting when the proper next-gen, natively AI and not just a repurposed GPU, stuff starts to land. Initially things like compute-in-memory and fancy flash, and later possibly the truly exotic stuff like various RMM (read-mostly-ram, for example stacked FeFET on logic) tech. IMO there is no chance for ubiquitous local AI to happen if that means people using GPUs. But the industry is chasing all possible solutions for more efficient inference platforms, and if something like pachinko machine architecture with FeFETs ends up winning, suddenly buying a local ai inference solution in a box is no longer as insane. | ||