Isn’t that mostly economics? I definitely prefer using Claude to GPT-OSS120B for a code assistant.

I don’t know about you, but I don’t have $500,000 laying around to buy myself a DGX B200 with a TB of HBM and 2TB of system ram, nor the 14.3kW of power to run the thing.

▲

m4rtink 17 hours ago | parent [-]

But will you still be able to afford using it once you have to pay the real price ? Once the venture capital dries up and the dumping stops ?

▲

selectodude 17 hours ago | parent [-]

I'm pretty sure all of these LLMs operate in the black on inference costs.

If I were to set up a DGX200 in my garage, say the 5 year TCO is a million dollars. Split that among 500 people and we can get it done for maybe $30/mo per user in total operating cost. I would bet that these LLMs are far more oversubscribed than 500 subs per server.

▲

coliveira 2 hours ago | parent | next [-]

How much of the current usage is paying at least 1 cent per inference? AI providers are giving away AI for anyone to use. Only professionals and big companies, that are at most 1% of the market, are paying anything at this point.

▲

irishcoffee 16 hours ago | parent | prev [-]

> I would bet that these LLMs are far more oversubscribed than 500 subs per server.

Seems like on hn a lot of people pay for the subscriptions.

I don't personally know a single person who pays for any type of llm subscription. I am a staff sw engineer, been doing this a long time.

I acknowledge this is an anecdote. I just happen to know a lot of people at a lot of different companies from my network. Nobody pays for any of this. My company has banned llms, even if I wanted to use one, I can't.

I actually even gave one a shot tonight. I asked for a list of repos I needed to clone to build a yocto image for an nxp board. This was the result:

mkdir -p sources && cd sources

git clone -b $BRANCH git://git.yoctoproject.org/poky

git clone -b $BRANCH git://git.yoctoproject.org/meta-freescale

git clone -b $BRANCH github.com

git clone -b $BRANCH git://git.openembedded.org/meta-openembedded

git clone -b $BRANCH github.com

I then pointed out that three of those lines were useless and asked it to fix those lines. The result I got was even more hilarious, and just as useless.

Disclaimer: this was the "dive deeper" button on a google search. No idea what fucking model it tried to use.

	▲	inavida 10 hours ago \| parent [-]
		I think your skepticism is warranted. Top comments look a lot like ads to me.