| ▲ | mkesper 3 hours ago | |||||||||||||||||||||||||||||||
Can you share some details about your setup? Would probably be interesting for a lot of people. | ||||||||||||||||||||||||||||||||
| ▲ | jacquesm 3 hours ago | parent [-] | |||||||||||||||||||||||||||||||
Sure: https://jacquesmattheij.com/local-inference-with-large-langu... If you have more questions feel free to mail or ask here in the thread and I'll do my best to answer. The hardware related parts are near the end of the article. | ||||||||||||||||||||||||||||||||
| ||||||||||||||||||||||||||||||||