> even in the 70s people recognized that supersonic travel had real concrete issues with no solution in sight. I don't think LLMs share that characteristic today

I think they pretty strongly do

The solution seems to be "just lower your standards for acceptable margin of error to whatever the LLM is capable of producing" which should be concerning and absolutely unacceptable to anyone calling themselves an Engineer

> The reason why I partially note this as wrong is that even in the 70s people recognized that supersonic travel had real concrete issues with no solution in sight. I don't think LLMs share that characteristic today.

The fundamental problem has already been mentioned: Nobody can figure out how to SELL it. Because few people are buying it.

It's useful for aggregation and summarization of large amounts of text, but it's not trustworthy. A good summary decreases noise and amplifies signal. LLMs don't do that. Without the capability to validate the output, it's not really generating output of lasting value. It's just a slightly better search engine.

It feels like, fundamentally, the primary invention here is teaching computers that it's okay to be wrong as long as you're convincing. That's very useful for propaganda or less savory aspects of business, but it's less useful for actual communication.

There are sound math reasons for skyscrapers topping out, mostly due to elevator capacity and the inability to effectively get people in and out of the floorspace as you go past a few hundred ft. There's no construction engineering reason you can't go taller - the Burj Khalifa, for example, is three times taller than a typical Western major city skyscraper - it just doesn't make economic sense unless you're a newly rich nation looking to prove a point.

>I think this is both right and wrong. There was a good book that came out probably 15 years ago about how technology never stops in aggregate, but individual technologies tend to grow quickly and then stall. Airplane jets were one example in the book. The reason why I partially note this as wrong is that even in the 70s people recognized that supersonic travel had real concrete issues with no solution in sight. I don't think LLMs share that characteristic today.

I don't see any solution to hallucinations, nor do I see any solution in sight. I think that could count as a concrete issue that would stop them.

Yeah, and with LLMs the thing I can't shake, however, is that this time it's pretty strongly (maybe parasitically) latched onto the aggregate progress of Moore's law. Few other technologies have enjoyed such relatively unfettered exponential improvement. It's like if skyscraper materials double in strength every n years, and their elevators approach teleportation speed, the water pumps get twice as powerful, etc., which would change the economics vs the reality that most of the physical world doesn't improve that fast.

Was the problem that supersonic flight was expensive and the amount of customers willing to pay the price was even lower than the number of customers that could even if they wanted to?

that's not a technology problem, many airports are super efficient, e.g. Singapore. Public transport in the US is held back by other forces.

They don't optimize for our convenience, they optimize for their profit.

SST came and went in an era when none of that security theater existed to begin with

Google does put AI output at the top of every search now, and sometimes it's helpful and sometimes it's crap. They have been trying since long before LLMs to not just provide the links for a search but also the content.

Google used to be interested in making sure you clicked either the paid link or the top link in the results, but for a few years now they'd prefer that a user doesn't even click a link after a search (at least to a non-Google site)

I think the thing people hate about that is the lack of effort and attention to detail. It’s an incredible enabler for laziness if misused.

If somebody writes a design or a report, you expect that they’ve put in the time and effort to make sure it is correct and well thought out.

If you then find the person actually just had ChatGPT generate it and didn’t put any effort into editing it and checking for correctness, then that is very infuriating.

They are essentially farming out the process of creating the document to AI and farming out the process of reviewing it to their colleagues. So what is their job then, exactly?

These are tools, not a replacement for human thought and work. Maybe someday we can just have ChatGPT serve as an engineer or a lawyer, but certainly not today.

Low effort Youtube shorts with AI voice annoy the crap out of me.

After all this hype, they still can't do text to speech properly. Pause at the wrong part of the sentence all the time.

> While people seem to love the output of their own queries they seem to hate the output of other people's queries

Listening or trying to read other peoples chats with these things is like listening to somebody describe a dream. It’s just not that interesting most of the time. It’s remarkable for the person experiencing it but it is deeply personal.

If I cared about the output from other people's queries then wouldn't they be my queries? I don't care about ChatGPTs response to your queries is because I don't care about your queries. I don't care if they came from ChatGPT or the world's foremost expert in whatever your query was about.

Had someone put up a project plan for something that was not disclosed as LLM assisted output.

While technically correct it came to the wrong conclusions about the best path forward and inevitably hamstrung the project.

I only discovered this later when attempting to fix the mess and having my own chat with an LLM and getting mysteriously similar responses.

The problem was that the assumptions made when asking the LLM were incorrect.

LLMs do not think independently and do not have the ability to challenge your assumptions or think laterally. (yet, possibly ever, one that does may be a different thing).

Unfortunately, this still makes them as good as or better than a very large portion of the population.

I get pissed off not because of the new technology or the use of the LLM, but the lack of understanding of the technology and the laziness with which many choose to deliver the results of these services.

I am more often mad at the person for not doing their job than I am at the use of a model, the model merely makes it easier to hide the lack of competence.

Especially if it's not disclosed up front, and especially when it supplants higher-value content. I've been shocked how little time it's taken for AI slop SEO optimized blogs to overtake the articles written by genuine human experts, especially in niche product reviews and technical discussions.

However, whether or not people like it is almost irrelevant. The thing that matters is not whether economics likes it.

At least so far, it looks like economics absolutely loves LLMs: Why hire expensive human customer support when you can just offload 90% of the work to a computer? Why pay expensive journalists when you can just have the AI summarize it? Why hire expensive technical writers to document your code when you can just give it to the AI and check the regulatory box with docs that are good enough?

No… dude… it’s a new household name. We haven’t had those in software for a long time, maybe since TikTok and Fortnite.

This isn't some niche outcry: https://www.forbes.com/sites/bernardmarr/2024/03/19/is-the-p...

And that was 18 months ago.

Yes, believe it or not, people eventually wake up and realize slop is slop. But like everything else with LLM development, tech is trying to brute force it on people anyway.

"Here's what chatGPT said about..." Is the new lmgtfy

Asking your Instagram Users to hop on to your ready made TikTok Clone is hardly in the same sphere as spinning up that much users from nothing.

And while Threads growth and usage stalled, ChatGPT is very much still growing and has *far* more monthly visits than threads.

There's really nothing meaningless about ChatGPT being the 5th most visited site on the planet, not even 2 years after release. Threads doesn't make the top 50.

Neal Stephenson has a recent post that covers some of this. Also links to teachers talking about many students just putting all their work into chatgpt and turning it in.

https://nealstephenson.substack.com/p/emerson-ai-and-the-for...

This echoes a lot of the rhetoric around "but how will facebook/twitter/etc make money?" back in the mid 2000s. LLMs might shake out differently from the social web, but I don't think that speculating about the flexibility of demand curves is a particularly useful exercise in an industry where the marginal cost of inference capacity is measured in microcents per token. Plus, the question at hand is "will LLMs be relevant?" and not "will LLMs be massively profitable to model providers?"

That's fixable, a gradual adjusting of the free tier will happen soon enough once they stop pumping money into it. Part of this is also a war of attrition though, who has the most money to keep a free tier the longest and attract the most people. Very familiar strategy for companies trying to gain market share.

Then cut off the free riders. Problem solved overnight.

They'd be profitable if they showed ads to their free tier users. They wouldn't even need to be particularly competent at targeting or aggressive with the amount of ads they show, they'd be profitable with 1/10th the ARPU of Meta or Google.

And they would not be incompetent at targeting. If they were to use the chat history for targeting, they might have the most valuable ad targeting data sets ever built.

That's calculating value against not having LLMs and current competitors. If they stopped improving but their competitors didn't, then the question would be the incremental cost of Claude (financial, adjusted for switching costs, etc) against the incremental advantage against the next best competitor that did continue improving. Lock in is going to be hard to accomplish around a product that has success defined by its generalizability and adaptability.

Basically, they can stop investing in research either when 1) the tech matures and everyone is out of ideas or 2) they have monopoly power from either market power or oracle style enterprise lock in or something. Otherwise they'll fall behind and you won't have any reason to pay for it anymore. Fun thing about "perfect" competition is that everyone competes their profits to zero

But if Claude stopped pouring their money into research and others didn't, Claude wouldn't be useful a year from now, as you could get a better model for the same price.

This is why AI companies must lose money short term. The moment improvements plateau or the economic environment changes, everyone will cut back on research.

For me, if Anthropic stopped now, and given access to all alternative models, they still would be worth exactly $240 which is the amount I'm paying now. I guess Anthropic and OpenAI can see the real demand by clearly seeing what are their free:basic:expensive plan ratios.

> Well worth 4 figures a year IMO

only because software engineering pay hasn't adjusted down for the new reality . You don't know what its worth yet.

And ARR is not revenue. It's "annualized recurring revenue": take one month's worth of revenue, multiply it by 12--and you get to pick which month makes the figures look most impressive.

It's a good point. Any business can get revenue by selling Twenty dollar bills for $19. But in the history of tech, many winners have been dismissed for lack of an apparent business model. Amazon went years losing money, and when the business stabilized, went years re-investing and never showed a profit. Analysts complained as Amazon expanded into non-retail activities. And then there's Uber.

The money is there. Investors believe this is the next big thing, and is a once in a lifetime opportunity. Bigger than the social media boom which made a bunch of billionaires, bigger than the dot com boom, bigger maybe than the invention of the microchip itself.

It's going to be years before any of these companies care about profit. Ad revenue is unlikely to fund the engineering and research they need. So the only question is, does the investor money dry up? I don't think so. Investor money will be chasing AGI until we get it or there's another AI winter.

Last year, ChatGPT was 75% of OpenAI's revenue[1], not the API.

[1]: https://www.businessofapps.com/data/chatgpt-statistics/

> In order to break even they would have to minimize the operating costs (by throttling, maiming models etc.) and/or increase prices. This would be the reality check.

Is that true? Are they operating inference at a loss or are they incurring losses entirely on R&D? I guess we'll probably never know, but I wouldn't take as a given that inference is operating at a loss.

I found this: https://semianalysis.com/2023/02/09/the-inference-cost-of-se...

which estimates that it costs $250M/year to operate ChatGPT. If even remotely true $10B in revenue on $250M of COGS would be a great business.

> I also imagine it’s hard to go to investors with “while all of our competitors are improving their models and either closing the gap or surpassing us, we’re just going to stabilize and see if people will pay for our current product.”

Yeah, no one wants to be the first to stop improving models. As long as investor money keeps flowing in there's no reason to - just keep burning it and try to outlast your competitors, figure out the business model later. We'll only start to see heavy monetization once the money dries up, if it ever does.

It is absolutely possible for the unit economics of a product to be profitable and for the parent company to be losing money. In fact, it's extremely common when the company is bullish on their own future and thus they invest heavily in marketing and R&D to continue their growth. This is what I understood GP to mean.

Whether it's true for any of the mainstream LLM companies or not is anyone's guess, since their financials are either private or don't separate out LLM inference as a line item.

> most people understand “AI” and “chatbot” to be synonyms

This might be true (or not), but for sure not on this site.

I mean, thats the point, they aren't the same. Concorde was one dimensional, AI is not.

> There's never been an innovation like this ever

There have been plenty of innovations like this. In fact, much of the hype around LLMs is a rehash of the hype around "expert systems" back in the '80s. LLMs are marginally more effective than those systems, but only marginally.

Most of them are a bunch of prompts and don't even have actual developers. For the good reason that there is no training system yet and the wording of how you call the people that build these system isn't even there or clearly defined. Local companies haven't even setup a proper internal LLM or at least a contract with a provider. I am in France so probably lagging behind USA a bit especially NY/SF but the word "LLM developer" is just arriving now and mostly under the pressure of isolated developers and companies like me. This feel really really early stage.

>Or why else would it not already have revolutionized everything a year or two ago when everyone started doing this?

The internet needed 20 years to take over the world. All of the companies of the first dot com bust are in the past. The tech is solid.

[flagged]

It's 3 years away.

This means I believe we currently underuse LLM capabilities and their empirical nature makes it difficult to assess their limitations without trying. I've been studying LLMs from various angles during a few months before coming to this conclusion, as an experienced software engineer and consultant. I must admit it is however biased towards my experience as an SME and in my local ecosystem.

This is a bit developer centric, I am much more impressed by the opportunities I see in consulting rather than applying LLMs to dev tasks. And I am still impressed by the code it can output eventhough we are still in the funny intern stage in this area.

I don't have a subscription not even an Open AI account (mostly cause they messed up their google account system). You can't extract value of an LLM by just using the official UI, you just scratch the surface of how they work. And yet there aren't much developers able to actually build an actual agent architecture that does deliver some value. I don't include the "thousands" of startups that are clearly suffer from a signaling bias: they don't exist in the economy and I don't care about them like at all in my reasonning. I am talking about actual LLM developers that you can recruit locally the same way you recruit a web developer today, and that can make sense out of "frontier" LLM garbage talk by using proper architectures. These devs are not there yet.

I cannot emphasize how much I agree with this comment. Thank you for writing it, I would never have had written it as well.

I pay $300 to fly from SF to LA when I could've just walked for free. Its true. How classist!

I think that's more reflective of the deteriorating relationship between OpenAI and Microsoft than an true lack of demand for datacenters. If a major model provider (OpenAI, Anthropic, Google, xAI) were to see a dip in available funding or stop focusing on training more powerful models, that would convince me we may be in a bubble about to pop, but there are no signs of that as far as I can see.

correct and companies will be exposing their data via mcp instead of standard rest apis.

Yup. Lots of products in the education space. Even doctors are using LLMs, while talking with patients. All sorts of teams are using the adjacent products for image and (increasingly) video generation. Translation freelancers have been hit somewhat hard because LLMs do "good enough" quite a bit better than old google translate.

Coding is relevant to the HN bubble, and as tech is the biggest driver of the economy it's no surprise that tech-related AI usages will also be the biggest causes of investment, but it really is used in quite a lot of places out there already that aren't coding related at all.

LLMs are amazing at anything requiring text analysis (go figure). Everyone I know doing equity or economic research in finance is using it extensively for that, and from what I hear from doctors the LLMs are as good as that in their space if not better

It feels like there's a lot of shifting goalposts. A year ago, the hype was that knowledge work would cease to exist by 2027.

Now we are trying to hype up enhanced email autocomplete and data analysis as revolutionary?

I agree that those things are useful. But it's not really addressing the criticism. I would have zero criticisms of AI marketing if it was "hey, look at this new technology that can assist your employees and make them 20% more productive".

I think there's also a healthy dose of skepticism after the internet and social media age. Those were also society altering technologies that purported to democratize the political and economic system. I don't think those goals were accomplished, although without a doubt many workers and industries were made more productive. That effect is definitely real and I'm not denying that.

But in other areas, the last 3 decades of technological advancement have been a resounding failure. We haven't made a dent in educational outcomes or intergenerational poverty, for instance.

That's fine, that's what I want ;) I just grow tired of people hating on me for thinking that we really need to localize the models for them to take off.

Looking up a project on github, downloading it and using it can give you 10000 lines of perfectly working code for free.

Also, when I use Cursor I have to watch it like a hawk or it deletes random bits of code that are needed or adds in extra code to repair imaginary issues. A good example was that I used it to write a function that inverted the axis on some data that I wanted to present differently, and then added that call into one of the functions generating the data I needed.

Of course, somewhere in the pipeline it added the call into every data generating function. Cue a very confused 20 minutes a week later when I was re-running some experiments.

well I presented the statement wrongly. What I mean is the use case for LLM are trivial things, it shouldn't be expensive to operate

and the 1 dollar cost for your case is heavily subsidized, that price won't hold up long assuming the computing power stays the same.

Thousands of lines of perfectly working code? Did you verify that yourself? Last time I tried it produced slop, and I've been extremely detailed in my prompt.

Well recently cursor got a heat for rising price and having opaque usage, while anthropic's claude reported to be worse due to optimization. IMO the current LLMs are not sustainable, and prices are expected to increase sooner or later.

Personally, until models comparable with sonnet 3.5 can be run locally on mid range setup, people need to wary that the price of LLM can skyrocket

It still doesn't pass the Turing test, and is not close. Five years ago me would be impressed but still adamant that this is not AI, nor is it on the path to AI.

>programs that would basically solve NLP

There is a load-bearing “basically” in this statement about the chat bots that just told me that the number of dogs granted forklift certification in 2023 is 8,472.

Yeah it solved NLP about 50% of the time, and also mangles data badly and in often hard-to-detect ways.

"hard not to anthropomorphize accidentally' is a you problem.

I'm unhappy every time I look in my inbox, as it's a constant reminder there are people (increasingly, scripts and LLMs!) prepared to straight-up lie to me if it means they can take my money or get me to click on a link that's a trap.

Are you anthropomorphizing that, too? You're not gonna last a day.

well I presented the statement wrongly. What I mean is the use case for LLM are trivial things, it shouldn't be expensive to operate

> all of the information on the internet

Total exaggeration—especially given Cloudflare providing free tools to block AI and now tools to charge bots for access to information.

Eh, I doubt it, tech only got massively better in each world war so far, through unlimited reckless strategic spending. We'd probably get a TSMC-like fab on every continent by the end of it. Maybe even optical computers. Quadrotor UAV are the future of warfare after all, and they require lots of compute.

Adjusted for inflation it took over 120 billion to build the fleet of liberty ships during WW2, that's like at least 10 TSMC fabs.