If I had the capital I’d make an household inference appliance.

No peripherals except Ethernet, integrated compute (cpu+gpu+mem) and secondary storage (+mobo, psu). No accoutrements, just the minimum amount of hardware to run a model as a utility.

Even the appliance faceplate would be a display showing stats like an old HiFi stereo.

Edit: something like a series of modules consisting of a RISC-V CPU + Vortex GPGPU + memory

It could heat your home in the winter and your pool in the summer.

I'm keeping an eye on Tenstorrent for this. Pricing seems like its going to end up being in between a super memory dense unified memory platform, and a purpose built GPU.

Definitely on the edge of what would make sense at home, but its interesting.

You're describing the mac mini/studio with some facelift.

Isn't that what what George Hotz is doing over at tiny? https://tinycorp.myshopify.com/

I think the closest to that in existence is the LLM ASIC designed by Taalas:

https://taalas.com/products/

Unfortunately their chatbot, while amazingly fast, doesn't know anything about the company running it.

Anyway I wouldn't mind an ASIC running a diffusion language model locally. Even if eventually it would become dated. Beats outsourcing all that to a company that's running on VC money which in the future might either perish or worse - dominate the market and charge whatever they wish.

Here you go: https://www.truffle.net/

Is that the nvidia spark?

Sounds like reinventing the home server.

the pheriphels support, or the appliance faceplate is tens of dollars, that not where you make the saving

95% of the price is going to be in GPU+CPU+RAM

build a Xeon / epyc 4u server. 12 channel ram.

Yes, just a big cool Cerebras wafer for the closet please.

[flagged]