| ▲ | simonw 8 hours ago |
| I am confident that Anthropic make revenue from that $20 than the electricity and server costs needed to serve that customer. Claude Code has rate limits for a reason: I expect they are carefully designed to ensure that the average user doesn't end up losing Anthropic money, and that even extreme heavy users don't cause big enough losses for it to be a problem. Everything I've heard makes me believe the margins on inference are quite high. The AI labs lose money because of the R&D and training costs, not because they're giving electricity and server operational costs away for free. |
|
| ▲ | tverbeure 7 hours ago | parent | next [-] |
| Nobody questions that Anthropic makes revenue from a $20 subscription. The opposite would be very strange. |
| |
| ▲ | simonw 6 hours ago | parent | next [-] | | A lot of people believe that Anthropic lose money selling tokens to customers because they are subsidizing it for growth. | | |
| ▲ | Drakim 3 hours ago | parent [-] | | But that has zero effect on revenue, it only affects profit. |
| |
| ▲ | brandensilva 7 hours ago | parent | prev [-] | | Yeah it's the caching that's doing the work for them though honestly. So many cached queries saving the GPUs from hard hits. | | |
| ▲ | xienze 3 hours ago | parent [-] | | How is caching implemented in this scenario? I find it unlikely that two developers are going to ask the same exact question, so at a minimum some work has to be done to figure out “someone’s asked this before, fetch the response out of the cache.” But then the problem is that most questions are peppered with specific context that has to be represented in the response, so there’s really no way to cache that. | | |
| ▲ | marcyb5st 3 hours ago | parent [-] | | From my understanding (which is poor at best), the cache is about the separate parts of the input context. Once the LLM read a file the content of that file is cached (i.e. some representation that the LLM creates for that specific file, but I really have no idea how that works). So the next time you bring either directly or indirectly that file into the context the LLM doesn't have to do a full pass, but pull its understanding/representation from the cache and uses that to answer your question/perform the task. |
|
|
|
|
| ▲ | Esophagus4 8 hours ago | parent | prev | next [-] |
| I always assumed that with inference being so cheap, my subscription fees were paying for training costs, not inference. |
| |
| ▲ | beAbU 2 hours ago | parent | next [-] | | Is inference really that cheap? Why can't I do it at home with a reasonable amount of money? | |
| ▲ | simonw 6 hours ago | parent | prev | next [-] | | Anthropic and OpenAI are both well documented as losing billions of dollars a year because their revenue doesn't cover their R&D and training costs, but that doesn't mean their revenue doesn't cover their inference costs. | | |
| ▲ | overgard 6 hours ago | parent | next [-] | | Does it matter if they can't ever stop training though? Like, this argument usually seems to imply that training is a one-off, not an ongoing process. I could save a lot of money if I stopped eating, but it'd be a short lived experiment. I'll be convinced they're actually making money when they stop asking for $30 billion funding rounds. None of that money is free! Whoever is giving them that money wants a return on their investment, somehow. | | |
| ▲ | vidarh 3 hours ago | parent | next [-] | | At some point the players will need to reach profitability. Even if they're subsidising it with other revenue - they'll only be willing to do that as long as it drives rising inference revenue. Once that happens, whomever is left standing can dial back the training investment to whatever their share of inference can bear. | | |
| ▲ | ben_w 3 hours ago | parent [-] | | > Once that happens, whomever is left standing can dial back the training investment to whatever their share of inference can bear. Or, if there's two people left standing, they may compete with each other on price rather than performance and each end up with cloud compute's margins. | | |
| ▲ | vidarh an hour ago | parent [-] | | Sure, but they will still need to dial it back to a point where they can fund it out of inference at some point. The point is that the fact they can't do that now is irrelevant - it's a game of chicken at the moment, and that might kill some of them, but the game won't last forever. |
|
| |
| ▲ | simonw 5 hours ago | parent | prev | next [-] | | It matters because as long as they are selling inference for less than it costs to serve they have a potential path to profitability. Training costs are fixed at whatever billions of dollars per year. If inference is profitable they might conceivably make a profit if they can build a model that's good enough to sign up vast numbers of paying customers. If they lose even more money on each new customer they don't have any path to profitability at all. | | |
| ▲ | citrin_ru an hour ago | parent [-] | | > If they lose even more money on each new customer they don't have any path to profitability at all. In theory they can increase prices once the customers will be hocked up. That's how many startups works. |
| |
| ▲ | krainboltgreene 5 hours ago | parent | prev [-] | | There's an argument to be made that a "return on investment by way of eliminating all workers" is a reasonable result for the capitalists. | | |
| ▲ | generic92034 3 hours ago | parent [-] | | At least until they are running out of customers. And/or societies with mass-unemployment destabilize to a degree that is not conducive for capitalists' operations. | | |
|
| |
| ▲ | vrighter 3 hours ago | parent | prev [-] | | Models are fixed. They do not learn post training. Which means that training needs to be ongoing. So the revenue covers the inference? So what? All that means is that it doesn't cover your costs and you're operating at a loss. Because it doesn't cover the training that you can't stop doing either. |
| |
| ▲ | smashed 7 hours ago | parent | prev [-] | | Doubtful |
|
|
| ▲ | what 5 hours ago | parent | prev [-] |
| >make revenue from that $20 than the electricity and server costs needed to serve that customer Seems like a pretty dumb take. It’s like saying it only takes $X in electricity and raw materials to produce a widget that I sell for $Y. Since $Y is bigger than $X, I’m making money! Just ignore that I have to pay people to work the lines. Ignore that I had to pay huge amounts to build the factory. Ignore every other cost. They can’t just fire everyone and stop training new models. |
| |
