Remix.run Logo
▲ mkesper 3 hours ago

Can you share some details about your setup? Would probably be interesting for a lot of people.

▲jacquesm 3 hours ago | parent [-]

Sure:

https://jacquesmattheij.com/local-inference-with-large-langu...

If you have more questions feel free to mail or ask here in the thread and I'll do my best to answer.

The hardware related parts are near the end of the article.

▲notpublic 2 hours ago | parent | next [-]

Your sites SSL certificate just expired..

▲jacquesm 2 hours ago | parent [-]

Thank you, I'll fix it right away.

Edit: ok, should be good again. For some reason on that box running Python from cron jobs is unpredictable, usually it works but every now and then it does not and I've yet to find out why.

▲t_mahmood 2 hours ago | parent | prev [-]

That was a good read!

▲jacquesm an hour ago | parent [-]

Thank you. That was a lot of work to put it all together.

It's basically the document I wished I had when I started all this.