| ▲ | Hamuko 8 hours ago |
| I’m not that interested in models that I can’t run on my desktop for ~0€, which is my AI budget. |
|
| ▲ | andai 7 hours ago | parent | next [-] |
| Electricity cost seems to be about $30/month for a 32B model on a GPU. It's probably better on Apple hardware. https://github.com/QuantiusBenignus/Zshelf/discussions/2 Not accounting for hardware, of course :) |
| |
| ▲ | Hamuko 7 hours ago | parent | next [-] | | My Mac Studio uses about 60–80 watts whenever I’m running a model (as measured by the system metrics), so it’s less than 2 kWh/day at full blast. Electricity is like 0.125 €/kWh, so that 24-hour period would be <0.25 €. Not accounting hardware in my costs, since I didn’t buy my hardware for running models. Running models is just something it can do in addition to what I got it for. | |
| ▲ | NorwegianDude 6 hours ago | parent | prev [-] | | The price, processed tokens, and output can be anything, it just depends on what GPU it is. Nvidia GPUs are much more efficient than Apple hardware for inference(and training). |
|
|
| ▲ | igravious 8 hours ago | parent | prev [-] |
| Cool beans. You're not the target audience then. |
| |
| ▲ | Hamuko 8 hours ago | parent [-] | | Did I claim I was? I just said why I and people like me are not talking about it. | | |
|
