AI has been improving at a very rapid pace, which means that a lot of people have really outdated priors. I see this all the time online where people are dismissive about AI in a way that suggests it's been a while since they last checked-in on the capabilities of models. They wrote off the coding ability of ChatGPT on version 3.5, for instance, and have missed all the advancements that have happened since. Or they talk about hallucination and haven't tried Deep Research as an alternative to traditional web-search.

Then there's a tendency to be so 'anti' that there's an assumption that anyone reporting that the tools are accomplishing truly impressive and useful things must be an 'AI booster' or shill. Or they assume that person must not have been a very good engineer in the first place, etc.

Really is one of those examples of the quote, "In the beginner's mind there are many possibilities, but in the expert's mind there are few."

It's a rapidly evolving field, and unless you actually spend some time kicking the tires on the models every so often, you're just basing your opinions on outdated experiences or what everyone else is saying about it.

I feel like I see these two opposite behaviors. People who formed an opinion about AI from an older model and haven't updated it. And people who have an opinion about what AI will be able to do in the future and refuse to acknowledge that it doesn't do that in the present.

And often when the two are arguing it's tricky to tell which is which, because whether or not it does something isn't totally black and white, there's some things it can sometimes do, which you can argue either way about that being in its capabilities or not.

To play devil's advocate, how is your argument not a 'no true scottsman' argument? As in, "oh, they had a negative view of X, well that's of course because they weren't testing the new and improved X2 model which is different". Fast forward a year .. "Oh, they have a negative view on X2, well silly them, they need to be using the Y24 model, that's where it's at, the X2 model isn't good anymore". Fast forward a year .. ad infinitum.

Are the models that exist today a "true scottsman" for you?

There is another big and growing group: charlatans (influencers). People who don't know much but make bold statements, select 'proof' cases. Just to get attention. There are many of them on youtube. When you someone on thumbnail making faces this is most likely it.

There are also a bunch of us who do kick the tires very often and are consistently underwhelmed.

There are also those of us who have used them substantially, and seen the damage that causes to a codebase in the long run (in part due to the missing gains of having someone who understands the codebase).

There are also those of us who just don’t like the interface of chatting with a robot instead of just solving the problem ourselves.

There are also those of us who find each generation of model substantially worse than the previous generation, and find the utility trending downwards.

There are also those of us who are concerned about the research coming out about the effects of using LLMs on your brain and cognitive load.

There are also those of us who appreciate craft, and take pride in what we do, and don’t find that same enjoyment/pride in asking LLMs to do it.

There are also those of us who worry about offloading our critical thinking to big corporations, and becoming dependent on a pay-to-play system, that is current being propped up by artificially lowered prices, with “RUG PULL” written all over them.

There are also those of us who are really concerned about the privacy issues, and don’t trust companies hundreds of billions of dollars in debt to some of the least trust worth individuals with that data.

Most of these issues don’t require much experience with the latest generation.

I don’t think the intention of your comment was to stir up FUD, but I feel like it’s really easy for people to walk away with that from this sort of comment, so I just wanted to add my two cents and tell people they really don’t need to be wasting their time every 6 weeks. They’re really not missing anything.

Can you do more than a few weeks ago? Sure? Maybe? But I can also do a lot more than I was able to a few weeks ago as well not using an LLM. I’ve learned and improved myself.

Chances are if you’re not already using an LLM it’s because you don’t like it, or don’t want to, and that’s really ok. If AGSI comes out in a few months, all the time you would have invested now would be out of date anyways.

There’s really no rush or need to be tapped in.

Is there anything you can tell me that will help me drop the nagging feeling that gradient descent trained models will just never be good?

I understand all of what you said, but I can't get over that fact that the term AI is being used for these architectures. It seems like the industry is just trying to do a cool parlor trick in convincing the masses this is somehow AI from science fiction.

Maybe I'm being overly cynical, but a lot of this stinks.

There's a middle ground which is to watch and see what happens around us. Is it unholy to not have an opinion?

> They wrote off the coding ability of ChatGPT on version 3.5, for instance, and have missed all the advancements that have happened since.

I feel like I see now more dismissive comments than previously. As if people, initially confused, formed a firm belief since. And now new facts don't really change it, just entrench them in chosen belief.

> They wrote off the coding ability of ChatGPT on version 3.5, for instance

I found I had better luck with ChatGPT 3.5's coding abilities. What the newer models are really good at, though, is doing the high level "thinking" work and explaining it in plain English, leaving me to simply do the coding.

I agree with you. I am a perpetual cynic about new technology (and a GenXer so multiply that by two) and I have deeply embraced AI in all parts of my business and basically am engaging with it all day for various tasks from helping me compare restaurant options to re-tagging a million contact records in salesforce.

It’s incredibly powerful and will just clearly be useful. I don’t believe it’s going to replace intelligence or people but it’s just obviously a remarkable tool.

But I think at least part of the dynamic is that the SV tech hype booster train has been so profoundly full of shit for so long that you really can’t blame people for skepticism. Crypto was and is just a giant and elaborate grift, to name one example. Also guys like Altman are clearly overstating the current trajectory.

The dismissive response does come with some context attached.

And that's why I keep checking back in.

They're still pretty dumb if you want the to do anything (ie with MCPs) but they're not bad at writing and code.

But the reports are from shills. The impact of ai is almost non existent. The greatest impact it had was on role-playing. It's hardly even useful for coding.

And that all wouldn't be a problem if it wasn't for the wave of bots that makes the crypto wave seem like child's play.

I think one of the issue is also the sheer amount of shilling going on like crypto level

I got a modest tech following and you wouldn’t believe the amount I’m offered to promote the most garbage AI company

I do see this a lot. It's hard to have a reasonable conversation about AI amidst, on the one hand, hype-mongers and boosters talking about how we'll have AGI in 2027 and all jobs are just about to be automated away, and on the other hand, a chorus of people who hate AI so much they have invested their identify in it failing and haven't really updated their priors since ChatGPT came out. Both groups repeat the same set of tired points that haven't really changed much in three years.

But there are plenty of us who try and walk a middle course. A lot of us have changed our opinions over time. ("When the facts change, I change my mind.") I didn't think AI models were much use for coding a year ago. The facts changed. (Claude Code came out.) Now I do. Frankly, I'd be suspicious of anyone who hasn't changed their opinions about AI in the last year.

You can believe all these things at once, and many of us do:

* LLMs are extremely impressive in what they can do. (I didn't believe I'd see something like this in my lifetime.)

* Used judiciously, they are a big productivity boost for software engineers and many other professions.

* They are imperfect and make mistakes, often in weird ways. They hallucinate. There are some trivial problems that they mess up.

* But they're not just "stochastic parrots." They can model the world and reason about it, albeit imperfectly and not like humans do.

* AI will change the world in the next 20 years

* But AI companies are overvalued at the present time and we're mostly likely in a bubble which will burst.

* Being in a bubble doesn't mean the technology is useless. (c.f. the dotcom bubble or the railroad bubble in the 19th century.)

* AGI isn't just around the corner. (There's still no way models can learn from experience.)

* A lot of people making optimistic claims about AI are doing it for self-serving boosterish reasons, because they want to pump up their stock price or sell you something

* AI has many potential negative consequences for society and mental health, and may be at least as nasty as social media in that respect

* AI has the potential to accelerate human progress in ways that really matter, such as medical research

* But anyone who claims to know the future is just guessing

There's three important beliefs at play in the A(G)I story:

1. When(if) AGI will arrive. It's likely going to be smeared out over a couple months to years, but relative to everything else, it's a historical blip. This really is the most contention belief with the most variability. It is currently predicted to be 8 years[1].

2. What percentage of jobs will be replaceable with AGI? Current estimates between 80-95% of professions. The remaining professions "culturally require" humans. Think live performance, artisanal goods, in-person care.

3. How quickly will AGI supplant human labor? What is the duration of replacement from inception to saturation? Replacement won't happen evenly, some professions are much easier to replace with AGI, some much more difficult. Let's estimate a 20-30 years horizon for the most stubborn to replace professions.

What we have is a ticking time bomb of labor change at least an order of magnitude greater than the transition from an agricultural economy to an industrial economy or from an industrial economy to a service economy.

Those happened over the course of several generations. Society: culture, education, the legal system, the economy, where able to absorb the changes over 100-200 years. Yet we're talking about a change on the same scale happening 10 times faster - within the timeline of one's professional career. And still, with previous revolutions we had incredible unrest, and social change. Taken as a whole, we'll have possibly the majority of the economy operating outside the territory of society, the legal system, and the existing economy. A kid born on the the "day" AGI arrives will become an adult in a profoundly different world as if born on a farm in 1850 and reaching adulthood in a city in 2000.

1. https://www.metaculus.com/questions/5121/date-of-artificial-...