> Just use the LLM to make a thing that's actually good at what you need it to do.

This is the end state of most imagined use cases for “agentic” software IMO. Use the AI to build software, with a classifier or LLM as the expansion joint in awkward spots.

▲

Twirrim 3 hours ago | parent | next [-]

The frustrating part about an LLM is at a fundamental level it's designed to do the wrong thing, it's designed to provide an answer for things it can't do, e.g. one argument I've been in recently at work is someone using it to summarise numerical data. They've a raw data spreadsheet they're putting into ChatGPT and asking it to produce a report in a particular style. So the LLM is doing its job, predicting the tokens to emit, and not actually doing mathematical analysis of the underlying data.

I had a team proudly show me how they were feeding their dashboard data into an LLM to predict growth needs, instead of doing basic linear regression or any number of pre-existing basic statistical analysis. The part that depressed me the most on that was it was a bunch of engineers that did it, the people you'd most hope would understand what an LLM is/does.

▲

DougBTX 3 hours ago | parent | next [-]

For numerical work, there’s no reason to assume the computation is happening in the LLM. All frontier models could handily write a Python script to do linear regression then report the result, so the quality of the responses here depends entirely on the prompt + harness.

	▲	41 minutes ago \| parent \| next [-]
		[deleted]
	▲	girvo 2 hours ago \| parent \| prev \| next [-]
		They’re very much trained to do this, in fact. Even Step 3.5 Flash, a medium sized open model from way earlier this year is trained to use python to calculate answers on the fly :)
	▲	charrondev 2 hours ago \| parent \| prev \| next [-]
		Definitely. I’ve found Cursor + Grok to be the best budget models for this type of tool use, although I’ve yet to try out the latest Anthropic ones. You can go “here’s a spreadsheet of some data, do a thing to it” and it’ll go through, have it plan out the type of calculations you want to do, and it’ll write you a program to parse your sheet, calculate what you want, then output an result sheet. The key to good results with minimal effort seems to be the planning modes to me, combined with the /goal command. A little bit of validation before implementation goes a long way.
	▲	hdjrudni 2 hours ago \| parent \| prev [-]
		+1. I noticed ChatGPT now pulls out a calculator or writes a script instead of hallucinating numbers. Initially it was terrible, but recently it seems pretty good... that said, if you're doing serious work with this you should probably double check. And if you're doing repeated work, obviously get it to write a script or formula instead of doing 1-offs.

▲

Foobar8568 2 hours ago | parent | prev [-]

Either chatgpt or claude are building scripts to do those and has been for a few months at least now.

▲

trollbridge 4 hours ago | parent | prev [-]

This is exactly what we’ve been doing.

For example, a client want a classifier to decide to make a go/no go decision on roofing jobs based on weather. The end result doesn’t use LLMs at all, but does use a lot of other techniques including human feedback when it makes the wrong call.

(Now the work crew gets the decision at 5 AM instead of 5:30 and the guy making the decision doesn’t have to wake up at all.)

▲

harpiaharpyja 3 hours ago | parent [-]

(Presumably you mean the guy affected by the decision?)

▲

jorgen123 2 hours ago | parent [-]

I think it means the person making the decision does not have to get up early anymore in order to make a decision. The people receiving the decision still receive the decision, and a bit earlier.

	▲	trollbridge 2 hours ago \| parent [-]
		A complaint from the crews was not getting the go/no-go at the same time consistently. This fixes that problem.