I asked a few of my friends that are ML engineers this question and all of them said to run the LLMs in the cloud with their infrastructure because it was going to be way faster. If you just want to tinker around I would look at @JSR_FDD's comment.

▲

al_borland 10 hours ago | parent | next [-]

My dad avoided buying a home PC for a long time, because he felt the systems he used at work in the 80s were so much more powerful than anything for the home at the time, that he didn’t see a point.

I can’t help but draw parallels here.

	▲	SenHeng 6 hours ago \| parent [-]
		And he wouldn't be wrong for that era. Prices also dropped pretty quickly while specs would almost double within a span of months. Whatever he could've bought in Jan would be easily outclassed by anything 12 months later. We don't have that anymore. Specs have more or less stabilised and what you're buying now could easily last you years.

▲

throwawaytea a day ago | parent | prev | next [-]

Next you're going to tell me that car salesman recommend just leasing new cars, doctors recommend just following the standard of care, construction workers just recommend subbing it out, and your tech friends say just use AWS.

▲

ramraj07 a day ago | parent [-]

If your car salesman friend gives you stupid advice it either means he is stupid or he is not your friend.

	▲	throwawaytea a day ago \| parent [-]
		I sold cars for a decade and wouldn't even take the insanely cheap priced company demo car, just driving my 10 year old car instead. On that note though, most car salesman are somewhat stupid, although the best ones are atleast normal. (I was a cal mechE grad and top 0.2% nationwide car salesman). They also slowly brainwash themselves into believing most of what they say.

▲

derwiki a day ago | parent | prev [-]

There are more factors than speed though, like: privacy