I think its worth noting that if you are paying for electricity Local LLM is NOT free. In most cases you will find that Haiku is cheaper, faster, and better than anything that will run on your local machine.

▲

gyrovagueGeist 7 hours ago | parent | next [-]

Electricity (on continental US) is pretty cheap assuming you already have the hardware:

Running at a full load of 1000W for every second of the year, for a model that produces 100 tps at 16 cents per kWh, is $1200 USD.

The same amount of tokens would cost at least $3,150 USD on current Claude Haiku 3.5 pricing.

	▲	ac29 7 hours ago \| parent [-]
		This 35B-A3B model is 4-5x cheaper than Haiku though, suggesting it would still be cheaper to outsource inference to the cloud vs running locally in your example

▲

postalrat 7 hours ago | parent | prev [-]

If you need the heating then it is basically free.

	▲	mrob 6 hours ago \| parent [-]
		Only if you use resistive electric heating, which is usually the most expensive heating available.