I'm pretty sure all of these LLMs operate in the black on inference costs.

If I were to set up a DGX200 in my garage, say the 5 year TCO is a million dollars. Split that among 500 people and we can get it done for maybe $30/mo per user in total operating cost. I would bet that these LLMs are far more oversubscribed than 500 subs per server.

How much of the current usage is paying at least 1 cent per inference? AI providers are giving away AI for anyone to use. Only professionals and big companies, that are at most 1% of the market, are paying anything at this point.

> I would bet that these LLMs are far more oversubscribed than 500 subs per server.

Seems like on hn a lot of people pay for the subscriptions.

I don't personally know a single person who pays for any type of llm subscription. I am a staff sw engineer, been doing this a long time.

I acknowledge this is an anecdote. I just happen to know a lot of people at a lot of different companies from my network. Nobody pays for any of this. My company has banned llms, even if I wanted to use one, I can't.

I actually even gave one a shot tonight. I asked for a list of repos I needed to clone to build a yocto image for an nxp board. This was the result:

mkdir -p sources && cd sources

git clone -b $BRANCH git://git.yoctoproject.org/poky

git clone -b $BRANCH git://git.yoctoproject.org/meta-freescale

git clone -b $BRANCH github.com

git clone -b $BRANCH github.com

git clone -b $BRANCH git://git.openembedded.org/meta-openembedded

git clone -b $BRANCH github.com

I then pointed out that three of those lines were useless and asked it to fix those lines. The result I got was even more hilarious, and just as useless.

Disclaimer: this was the "dive deeper" button on a google search. No idea what fucking model it tried to use.