| ▲ | TuringNYC 3 hours ago | |
I'd love a way to do this locally -- pool all the PCs in our own office for in-office pools of compute. Any suggestions from anyone? We currently run ollama but manually manage the pools | ||
| ▲ | zozbot234 a minute ago | parent | next [-] | |
If you set CPUSchedulingPolicy=idle Nice=19 IOSchedulingClass=idle in the ollama server configuration it should run in the background with lowest priority. | ||
| ▲ | utopiah 14 minutes ago | parent | prev | next [-] | |
Seems like so much more work than "just" paying for https://huggingface.co or whichever other neocloud who already did all the setup for you and just waits for your credit card per minute/seconds/token. | ||
| ▲ | damezumari 3 hours ago | parent | prev [-] | |