Did they stop teaching "actions speak louder than words" in schools, or something?

I don't think there is any line drawn here. I think if they executed well (and by they I mean any one of the three SOTA LLM vendors), they could already mortally wound the entire software industry today.

Whether or not they want, or will want, to do it at some point, is unknown; the reasons to not do it now are obvious:

1) it's more profitable to keep renting intelligence per token to everyone, preserving the status quo and milking it indefinitely (i.e. while the models aren't yet good enough to reliably single-shot complex software products from half-baked prompts, because once they get there, disruption will happen organically)

2) trying to compete with ~every other software product today is not likely to succeed in the end; a serious attempt would still burn down the software industry, but the major players don't have the capacity to handle it all at once, and doing it gradually will give enough time for regulatory agencies to try and stop it; either way, no one wins

What I'm saying is not (directly) related to whether or not LLMs are "true AI" or not. It's sufficient that they are fully general problem solvers.

A software product (whether bought or rented as a service) is defined by its boundaries - there's a narrow set of specific problems, and specific ways it can be used to solve those problems, and beyond those, it's not capable (or not allowed) to be used for anything else. The specific choices of what, how, and on what terms, are what companies stick a name to to create a "software product", and those same choices also determine how (and how much) money it will make for them.

Those boundaries are what LLMs, as general-purpose problem solvers, break naturally, and trying to force-fit them within those limits means removing most of the value they offer.

Consider a word processor (like MS Word). It's solving the problem of creating richly-formatted, nice-looking documents. By default it's not going to pick the formatting for you, nor is it going to write your text for you. Now, consider two scenarios of adding LLMs to it:

- On the inside: the LLM will be able to write you a poem or rewrite a piece of document. It could be made to also edit formatting, chat with you about the contents, etc.

- From the outside: all the above, but also the LLM will be able to write you an itinerary based on information collected from maps/planning tool, airline site, hotel site, a list of personal preferences of your partner, etc. It will be able to edit formatting to match your website and presentation made in the competitor's office tools and projected weather for tomorrow.

Most importantly, it will be able to do both of those automatically, just because you set up a recurring daily task of "hey, look at my next week's worth of calendar events and figure out which ones you can do some useful pre-work for me, and then do that".

That's the distinction I'm talking about, that's the threat to software industry, and it doesn't take "true AI" - the LLMs as we have today are enough already. It's about generality that allows them to erase the boundaries that define what products are - which (this is the "mortal wound to software industry" part) devalues software products themselves, reducing them to mere tool calls for "software agents", and destroying all the main ways software companies make money today - i.e. setting up and exploiting tactics like captive audience, taking data hostage, bundled offers, UI as the best marketing/upsale platform, etc.

(To be clear - personally, I'm in favor of this happening, though I worry about consequences of it happening all at once.)