LLMs need significant optimization or we get significant improvement on computing power while keeping the energy cost the same. It's similar with smartphone, when at the start it's not feasible because of computing power, and now we have one that can rival 2000s notebooks.

LLMs is too trivial to be expensive

EDIT: I presented the statement wrongly. What I mean is the use case for LLM are trivial things, it shouldn't be expensive to operate

▲

killerstorm 17 hours ago | parent | next [-]

LLM can give you thousands of lines of perfectly working code for less than 1 dollar. How is that trivial or expensive?

▲

sgt101 16 hours ago | parent | next [-]

Looking up a project on github, downloading it and using it can give you 10000 lines of perfectly working code for free.

Also, when I use Cursor I have to watch it like a hawk or it deletes random bits of code that are needed or adds in extra code to repair imaginary issues. A good example was that I used it to write a function that inverted the axis on some data that I wanted to present differently, and then added that call into one of the functions generating the data I needed.

Of course, somewhere in the pipeline it added the call into every data generating function. Cue a very confused 20 minutes a week later when I was re-running some experiments.

▲

brulard 15 hours ago | parent [-]

Are you seriously comparing downloading static code from github with bespoke code generated for your specific problem? LLMs don't keep you from coding, they assist it. Sometimes the output works, sometimes it doesn't (on first or multiple tries). Dismissing the entire approach because it's not perfect yet is shortsighted.

▲

ozgrakkurt 13 hours ago | parent [-]

They didn’t dismiss it, they just said it is not really that useful which is correct?

	▲	brulard 3 hours ago \| parent \| next [-]
		Obviously YMMV, but it is extremely useful for me and for many people out there.
	▲	Matticus_Rex 9 hours ago \| parent \| prev [-]
		Many obviously disagree that it's correct

▲

fendy3002 17 hours ago | parent | prev | next [-]

well I presented the statement wrongly. What I mean is the use case for LLM are trivial things, it shouldn't be expensive to operate

and the 1 dollar cost for your case is heavily subsidized, that price won't hold up long assuming the computing power stays the same.

	▲	killerstorm 11 hours ago \| parent [-]
		Cheaper models might be around $0.01 per request, and it's not subsidized: we see a lot of different providers offering open source models, which offer quality similar to proprietary ones. On-device generation is also an option now. For $1 I'm talking about Claude Opus 4. I doubt it's subsidized - it's already much more expensive than the open models.

▲

zwnow 17 hours ago | parent | prev [-]

Thousands of lines of perfectly working code? Did you verify that yourself? Last time I tried it produced slop, and I've been extremely detailed in my prompt.

▲

killerstorm 3 hours ago | parent | next [-]

Yes. I verified it myself. Best results from Opus 4 so far, Gemini might be OK too.

▲

DSingularity 7 hours ago | parent | prev [-]

Try again.

	▲	mrbungie 3 hours ago \| parent [-]
		Any retries before nailing the prompt are still going to be billed, so this supports GP position about LLMs being expensive for trivial things.

▲

jsnell 11 hours ago | parent | prev | next [-]

But the thing is, LLMs are already incredibly cheap to operate compared to the alternatives. Both for trivial things and for complex things.

	▲	fendy3002 11 hours ago \| parent [-]
		Well recently cursor got a heat for rising price and having opaque usage, while anthropic's claude reported to be worse due to optimization. IMO the current LLMs are not sustainable, and prices are expected to increase sooner or later. Personally, until models comparable with sonnet 3.5 can be run locally on mid range setup, people need to wary that the price of LLM can skyrocket

▲

lblume 18 hours ago | parent | prev | next [-]

Imagine telling a person from five years ago that the programs that would basically solve NLP, perform better than experts at many tasks and are hard not to anthropomorphize accidentally are actually "trivial". Good luck with that.

▲

hyperbovine 4 hours ago | parent | next [-]

It still doesn't pass the Turing test, and is not close. Five years ago me would be impressed but still adamant that this is not AI, nor is it on the path to AI.

▲

jrflowers 17 hours ago | parent | prev | next [-]

>programs that would basically solve NLP

There is a load-bearing “basically” in this statement about the chat bots that just told me that the number of dogs granted forklift certification in 2023 is 8,472.

	▲	lblume 17 hours ago \| parent [-]
		Sure, maybe solving NLP is too great a claim to make. It is still not at all ordinary that beforehand we could not solve referential questions algorithmically, that we could not extract information from plain text into custom schemas of structured data, and context-aware mechanical translation was really unheard of. Nowadays LLMs can do most of these tasks better than most humans in most scenarios. Many NLP questions at least I find interesting reduce to questions of the explanability of LLMs.

▲

clarinificator 17 hours ago | parent | prev | next [-]

Yeah it solved NLP about 50% of the time, and also mangles data badly and in often hard-to-detect ways.

▲

Applejinx 13 hours ago | parent | prev [-]

"hard not to anthropomorphize accidentally' is a you problem.

I'm unhappy every time I look in my inbox, as it's a constant reminder there are people (increasingly, scripts and LLMs!) prepared to straight-up lie to me if it means they can take my money or get me to click on a link that's a trap.

Are you anthropomorphizing that, too? You're not gonna last a day.

	▲	lblume 12 hours ago \| parent [-]
		I didn't mean typical chatbot output, these are luckily still fairly recognizable due to stylistic preferences learned during fine-tuning. I mean actual base model output. Take a SOTA base model and give it the first two paragraphs of some longer text you wrote, and I would bet on many people being unable to distinguish your continuation from the model's autoregressive guesses.

▲

trashchomper 18 hours ago | parent | prev [-]

Calling LLMs trivial is a new one. Yea just consume all of the information on the internet and encode it into a statistical model, trivial, child could do it /s

	▲	fendy3002 17 hours ago \| parent \| next [-]
		well I presented the statement wrongly. What I mean is the use case for LLM are trivial things, it shouldn't be expensive to operate
	▲	hammyhavoc 17 hours ago \| parent \| prev [-]
		> all of the information on the internet Total exaggeration—especially given Cloudflare providing free tools to block AI and now tools to charge bots for access to information.