|
| ▲ | drusepth 4 days ago | parent | next [-] |
| Are 235B models classified as local LLMs? I guess they probably are, but others in this thread are probably looking more toward 20B-30B models and sizes that generally fit on the RAM you'd expect in average or slightly-higher-end hardware. My beefy 3D gamedev workstation with a 4090 and 128GB RAM can't even run a 235B model unless it's extremely quantized (and even then, only at like single-digit tokens/minute). |
|
| ▲ | codazoda 4 days ago | parent | prev | next [-] |
| How much machine do you have to be able to run Qwen-235B locally? |
|
| ▲ | oblio 4 days ago | parent | prev | next [-] |
| Can you run any of those models without $20 000 worth of hardware that uses as much power and makes as much noise as a small factory? |
| |
| ▲ | segmondy 2 days ago | parent [-] | | I run them with under $3,000 hardware and inference is about 500-600watts with no noise. |
|
|
| ▲ | nomel 4 days ago | parent | prev | next [-] |
| Without knowing what you were doing with that 1300 lines of code, there's not much insight that can be had from this. |
|
| ▲ | brookst 3 days ago | parent | prev [-] |
| I’m a mediocre C programmer on my best day and I assure you a highly competent programmer could probably use 200 lines of code to do what I achieve in 1300. Just counting lines is not a good proxy for how much effort it would take a good programmer. (And I am 100% pro LLM coding, just saying this isn’t a great argument) |
