> but they may not yield as much benefit.

This morning, I was talking to a friend of mine, who is a marketing (not tech) guy, and he was gushing about LLMs.

He loves being able to just dump in a bucket of seemingly unrelated data, and getting actionable BI from it.

I think we're in for a wild ride. Fasten your seat belt, and keep your hands inside the vehicle...

Yeah, I think it's at the level of self-driving Excel.

Now, Excel also makes mistakes while trying to get clever - remember the biologists who had to rename several genes to prevent Excel treating them as dates - but it's also a ubiquitous business tool. And you can combine them. But equally Excel hasn't replaced every white collar job.

Yes, but BI is not a good example of industry.

It's like Software, its value is to be a complement and/or an abstraction of existing businesses. All of that sits on the very basics of the essence of civilization, which is how much things we can get off the soil and what how quickly and effective we can transform it.

Not to be anti-tech, but every new advancement, no matter how next-level, has to sell more of the same goods and services people have been using since the dawn of civilization, and those goods and services need to solve the mundane problems people have been getting since millenia.

I worry about this. Dumping data and getting a report on which actions are based prolly isn’t giving real data.

I say that because I have Claude review code and then review the review and it will point out all the things it got wrong the first time. And I end up having to review it myself. Don’t get me wrong, I’m burning tokens all day and love the legit productivity gains but I don’t see how this replaces everybody in tech for a while.

>He loves being able to just dump in a bucket of seemingly unrelated data, and getting actionable BI from it.

Sorry, just don't trust accurate LLM data analysis even from the latest claude model. Have had some embarrassing bloopers over the last year. It is just better to code a deterministic tool to do the data analysis.

I don't doubt it. Earlier today I got an estimate for the revenue, profit margin, and headcount for a potential supplier of a hardware component to a product I'm working on. It would've taken a couple hours of really tedious work for me to do that with Google translate, a search engine, and various market research sites. The result is I know more what I can expect from that vendor in terms of customization and support.

But those are the kinds of estimates that I can eyeball and be more than 80% certain they are plausible. That won't cut it in a lot of domains.

marketing is one of those domains, if everyone gets better, no gets better in the end

I hope he validates the output, the number of times colleagues have done exactly that and gotten the wrong output is.. well, not an insignificant amount.

Not saying it can't be done, but my recommendation is still "ask your LLM to make a tool that process the data, do not process it via the LLM itself".

> He loves being able to just dump in a bucket of seemingly unrelated data, and getting actionable BI from it.

* what he perceives to be actionable BI.

> BI

Business intelligence

> and getting actionable BI from it.

I’ve always had doubts about the validity of this “actionable BI,” even before it was slop, serving, mostly to serve leadership’s biases and justify their preferred ideas.

If accuracy was never that valuable to begin with then this should not be a problem.