Remix.run Logo
▲ alerighi 2 hours ago

Because maybe you don't want to send your private data to a company that could use it for whatever it wants?

▲anonym29 a minute ago | parent | next [-]

GP isn't proposing using closed proprietary models over an API, they're proposing having local hardware serving open weight models at your house/apartment being remotely accessible through an overlay network like Tailscale. The idea being that local inference is very hungry for power and thermal budget, so you run the inference somewhere that power and thermal headroom is abundant (your house/apartment, on a tower, or dedicated unified memory boxes like DGX Spark, Strix Halo, Mac Studio, etc) and then remotely utilize that inference from a regular laptop. You still get all of the benefits of local, private AI inference, without having to expend a laptop's limited power and thermal budget on doing the inference on the laptop itself.

▲barrkel 34 minutes ago | parent | prev [-]

Can you explain what data is going where?

▲entrope 8 minutes ago | parent [-]

Your prompts and whatever data they collect from your dev environment go to the inference provider, and typically the customer has neither control nor assurance over what the inference provider (re-)uses that data for.