What are the variables that prefer local GPUs vs cloud inference? Is connectivity the dividing line or are there other variables that influence the choice?

Anduril submersibles probably need local processing, but does my laundry/dishes robot need local processing? Or machines in factories? Or delivery drones?

Any sort of continuous video processing, especially low-latency.

Imagine you were tracking items on video at a self-service checkout. Sure, you could compress the video down to 15 Mbps or so and send it to the cloud. But now, a store with 20 self-checkouts needs 300 Mbps of upload bandwidth. That's one more problem making it harder for Wal-Mart to buy and roll out your product.

Also, if you know you need an NVIDIA L4 dedicated to you 24/7 for a year, a g6.xlarge will cost $7,000/year on-demand or $4,300/year reserved [1] while you can buy the card for $2,500.

Of course for many other use cases the cloud is a fine choice. If you only need a fraction of a GPU, or you only need a monster GPU a tiny fraction of the time, or you need an enormous LLM that demands water cooling and tolerates latency easily, the cloud can be a fine choice.

[1] https://instances.vantage.sh/aws/ec2/g6.xlarge?currency=USD&...

Anything latency sensitive. Anything bandwidth constrained.

Simple example, Security cameras that only use bandwidth when they've detected something. The cost of live streaming 20 cameras over 5g is very high. The cost of sending text messages with still images when you see a person is reasonable.

Why the hell would a dishwasher need to be connected, or smart for that matter?

I just want clean dishes/clothes, not to be upsold into some stupid shit that fails when it can’t ping google.com or gets bricked when the company closes.

I would pay premium for certified mindless products.

Anecdotally, I don't have any direct physical evidence or written evidence to support this. But I talked to someone in the industry over a decade ago when "run it on a GPU" was just heating up. It's drones. Not DJI ones, military ones with surveillance gear and weapons.

Mining, remote construction, remote power station inspection, battlefields. there are many many places where a stable network connection can't be taken for granted.

I want local processing for my local data. That includes my photos, documents and surveillance camera feeds.

If I had to guess there is significant interest in this product from a certain Eastern European nation. I don’t think they are intending to use it for “robotics” though.

it depends if the plates were expensive.