▲ | NitpickLawyer 2 days ago | |
> If you're inferencing, you're not getting much more out of this than you would a ~$2K Framework desktop. I was with you up till here. Come on! CPU inferencing is not it, even macs struggle with bigger models, longer contexts (esp. visible when agentic stuff gets > 32k tokens). The PRO6000 is the first gpu that actually makes sense to own from their "workstation" series. | ||
▲ | refulgentis 2 days ago | parent [-] | |
Er, CPU inferencing? :) I didn't think I mentioned that! The Framework Desktop thing is that has unified memory with the GPU, so much like an M-series, you can inference disproportionately large models. |