| |
| ▲ | razodactyl 5 hours ago | parent | next [-] | | Not sure I know where I fall regarding your point: Yes to trade secrets, but also science and AI should be for the good of all. OpenAI seems to be trading roles back with Anthropic becoming misanthropic. I hope they both start heading in the direction of how the AI field was prior to LLMs. Collaboration and benefit for all should always be the primary motivator. | | |
| ▲ | georgemcbay 4 hours ago | parent [-] | | > Collaboration and benefit for all should always be the primary motivator. Of all the things to never happen, this is never going to happen the most. That train left the station for good once hundreds of billions to trillions of dollars were involved. On the bright side, in the long run I suspect the vast majority of the value of AI will not be captured by the model making labs and the vast investments in them are going to implode, so... | | |
| ▲ | fragmede 3 hours ago | parent [-] | | implode, how? | | |
| ▲ | reinitctxoffset 3 hours ago | parent [-] | | They're in a price war with the People's Republic of China running flat out with the full backing of a government that literally does not care if they ever see a financial return on the investment, they just want to drive the value of LLM training and inference to zero because we banked the market on it being arbitrarily high margin forever. China was like hold my beer. They have a staggering surplus of grid capacity and can bring more online without any difficulty. We couldn't get a serious nuclear project done if Jeffrey Epstein was offering private flights to the ribbon cutting. In the United States at any given time more than half of the FLOPs are badly misallocated, Meta has like, a double digit percentage of the total capacity going down the drain every day and has for years. That's a conspicuous example but on OpenRouter rankings it's rare to see more than one or two American vendors in the top 10, sometimes the top 20. But 3rd, 4th, and 5th place are all merrily burning half the compute duplicating effort and missing key innovations because we stopped publishing real results. In China if DeepSeek makes a breakthrough it's at Zhupai and Moonshot and MiniMax and MiMo and Qwen that week. Our only lever, export restrictions, seems to do nothing but breed multiply antibiotic resistant super hackers who just get more efficient and immediately propagate all of those efficiencies to the rest of the Chinese AI industry. At the beginning of 2026 there was one Chinese lab with a model that had any real relevance fielding modern tool users. Today in July there are like, eight lagging the absolute frontier by maybe 3-6 months. Barring some massive bend in some curve 3-4 of the top 5 and 6-8 of the top 10 will be Chinese and open weight by January. The great irony in all of this is that our current playbook is straight out of the 1960s USSR, and the PRC's current playbook is straight out of 1960s USA. We're the ones with the opaque decision making and gross resource misallocation driven by the personal agendas of a shadowy cabal of frenemies wired back channel into government in the form of the individuals rather than the offices. They're the ones with a thriving marketplace of ideas powered by robust public/private partnership and a paved path running bidirectionally to the university system. It's going to implode because the Kruschev system does. Theirs is going to thrive because the Kennedy system puts a man on the moon before the decade is out. | | |
| ▲ | maxglute 2 hours ago | parent | next [-] | | >full backing of a government that literally does not care if they ever see a financial return on the investment There's no evidence of this, the parsimonious explanation is PRC AI, by virtue of being sanctioned, simply is not able to run magnitude more expensive compute model, and even if they could, they don't have the $$$ or market cap to do so. So they optimize and involute margins like they do in everything, and US misallocated expensive flops because the entire industry has been financially engineered for phat margins along the entire producer supply chain is just cherry on cake. Like wipe out the 50%+ margins from toolmakers, fabs, gpu/memory/data center components to some reasonable level and US is overpaying for tokens by a stupid multiplier on top of actual compute misallocation due to incompetent infra. Maybe PRC AI has unsound economics, but it's structurally simply not able to misallocate as much as US who will find a way to financialize compute to point of absurdity. | |
| ▲ | satvikpendem an hour ago | parent | prev [-] | | The US can just ban Chinese weights being used in US companies. | | |
| ▲ | reinitctxoffset 12 minutes ago | parent [-] | | If the Trump administration decides to annoint Altman and Amodei in defiance of market forces it will rapidly discover that it no longer has the sovereign bond auction pricing power to prop them up. This isn't 1998: the Treasury has taken five major body blows in the last 25 years, the world's energy markets, maritime insurance regimes, electronic payments rails, and moral authority in places like the UN Security Council are effectively bifurcated. Oil and money and data and people just flow around the United States now. Canada's sovereign debt yields tank harder when the market is spooked, that's the North American flight to quality. The administration could probably put some serious friction on open weight model use in the Fortune 500 for a little while, but the opposition never got such a gift right before a squeaker midterm. And outside of major enterprises with puckered ass compliance departments? Not a chance. It's popular around here to forget Uber and AirBnB and yes, OpenAI and Anthropic all got their start flagrantly breaking the law and grew lawyers and lobbyists faster than anyone could enforce it. And this time everyone from the DNC to the EFF would be holding hands wearing "Save The Models" t-shirts. Not even NVIDIA is remotely pretending they're anything but all in on GLM 5.2, they had an NVFP4 quant up by the time most people read the blog post. And the Trump Administration isn't exactly enamored of Comrade Amodei at the moment, being as they're appealing the lawsuit Anthropic brought against the Pentagon during a shooting war. Forcing the American proprietary AI megalab financing event was our fiscal Ukraine Special Military Operation, the market is calling the bluff and neither the capital markets nor the Federal Reserve has the dry powder to absorb this one. The Treasury auctions will flat not clear in an orderly way. We can't raise 2-4 trillion dollars on a dime in 2026 and if CoreWeave turns out, as many suspect, to be Patient Zero? It would be that big a hole. We play by the same rules as everyone else now. I hope we regard it as being worth it, but I fear we will not. |
|
|
|
|
| |
| ▲ | KronisLV 2 hours ago | parent | prev | next [-] | | > the company limits “the number of people who are aware of a given compute multiplier” because it could give other AI labs a leg up if they were to be able to replicate them. I wonder if that makes sense if the orgs within the industry are starting to shift their mindset towards "Tokens are expensive, we should use AI less." which feels like an existential threat to the status quo, if those AI providers can't find ways to keep costs affordable for their clients. Otherwise those orgs would just be using GLM 5.2 or DeepSeek V4 Pro but it seems like what they're doing instead is trying to use AI just less, period. | |
| ▲ | bloppe an hour ago | parent | prev | next [-] | | I agree that Dario is pretty annoying, but I think the "tech villain" archetype is essentially survivorship bias. The tech leaders who don't act that way are not nearly as visible because they're not nearly as successful. | | |
| ▲ | solenoid0937 an hour ago | parent [-] | | HN is just a massive Anthropic hate fest now, probably funded/manipulated by OAI's $8B PR budget. OP phrases it as a bad thing that Dario is keeping compute multipliers to Anthropic. How naive can one be? Compute multipliers are the whole business. Those are the trade secrets every lab is built on. It is the alpha of the business. How does protecting this make Dario evil? This website is getting out of hand with the uninformed hot takes. I wish when HN was still people that knew what they were talking about. | | |
| ▲ | bloppe an hour ago | parent | next [-] | | I'm saying I would do the same thing if I were Dario. I don't think he's evil. I just think his hero complex is annoying. | |
| ▲ | satvikpendem an hour ago | parent | prev [-] | | [dead] |
|
| |
| ▲ | alightsoul 4 hours ago | parent | prev | next [-] | | i really hope it's just what Deepseek V4 does. Deepseek V4 is very cheap and highly performant OpenAI tried to pull off the same trade secret thing with RL when they announced o1 and o3, aka "Compute time scaling". Then Deepseek revealed it with Deepseek R1. Could also be something like Deepseek DSpark. Or using diffusion like DiffusionGemma as a draft model. The timing between the release of those, and this article, makes me think its maybe one or both of those things | |
| ▲ | llelouch 4 hours ago | parent | prev | next [-] | | Dario tells the truth. If you look at everything through their safe AGI mission it all makes sense. They are not bs'ing about that. Also I think most people just read headlines or 10 second clips and make false extrapolations from there. (BTW Anthropic only exists because Sam Altman is a liar, Dario admitted this.) | | |
| ▲ | nmfisher 4 hours ago | parent [-] | | > If you look at everything through their safe AGI mission it all makes sense. Except for, you know, all the outside investors and the forthcoming IPO. | | |
| ▲ | adahn 3 hours ago | parent | next [-] | | A no-investment policy would take them off the scene entirely. Essentially handing over the reins to OpenAI, Google, and others. Their position is something close to "if I don't do it, someone worse will". Related: https://80000hours.org/2012/03/the-replaceability-effect-wor... There's a more nuanced discussion that could be had about how to balance relevance with outside influence. But at a foundational level it should be acknowledged that the tradeoff exists, and that receiving outside investment can't alone be seen as evidence of corruption. Besides that, there's more that can be said about other things like their corporate structure or the degree to which they accelerated the AI race. | | |
| ▲ | bloppe an hour ago | parent [-] | | "if I don't do it, someone worse will" Of course that's what Dario thinks because that's what every tech CEO thinks. Dario, Sam, Sundar, probably many Chinese CEOs as well. It's what everyone thinks. That's why they're competing so fiercely with one another. That's why they basically make all the same decisions. That's why we need properly open source AI. | | |
| ▲ | adahn 40 minutes ago | parent | next [-] | | It's hard to be certain what each individual thinks. We can do our best to judge based on what they each say and do. And there are significant differences in what each of these individuals have chosen to say and do over the years. The info available to the public makes it seem a lot like Dario's motivations & priorities differ from those of Sam and others. This doesn't seem like the right place to spend my time litigating that point to its fullest extent (no-one here is doing that). But there's plenty of relevant info surrounding eg.: * The New Yorker article on Altman [1] * The story behind Anthropic's founding * Various efforts to influence government policy (a16z policies and contributors [2], Trump's inauguration donors [3], giving Trump credit for AI infrastructure [4], Dario's op-eds [5]) 1: https://www.newyorker.com/magazine/2026/04/13/sam-altman-may... 2: https://a16z.com/portfolio/ 3: https://www.opensecrets.org/trump/2025-inauguration-donors 4: https://www.youtube.com/watch?v=pe11mJ8mCHU 5: https://darioamodei.com/ | |
| ▲ | solenoid0937 an hour ago | parent | prev [-] | | Open source AI fails first contact with sufficiently-intelligent-as-to-be-dangerous AI. The day Mythos class models are open sourced will not be a good day. I don't think you understand the impact that will have on the world and on cyber defenders everywhere. It will be pure chaos. Even if you don't think Mythos-class is the bar, open source has to stop at some point, you don't hand everyone a superweapon. | | |
| ▲ | bloppe 28 minutes ago | parent [-] | | Every single one of those sentences is highly dubious. Cyber-defenders would be pretty jazzed about having easier access to Mythos-class models. Cyber-defense is easier with better tools. |
|
|
| |
| ▲ | solenoid0937 an hour ago | parent | prev [-] | | You expect AGI to be built without additional investor money? |
|
| |
| ▲ | 5 hours ago | parent | prev | next [-] | | [deleted] | |
| ▲ | bigyabai 5 hours ago | parent | prev [-] | | > He sure did seem to speed run the 'tech leader with scruples' to 'tech villain' path! What kind of rosy-eyed chump believes in the "tech leader with scruples" bullshit? It always lies. Did some people just ignore Mark Zuckerberg and Tim Cook's sociopathy, somehow? Did anyone buy into their "privacy is a human right" nonsense? | | |
| ▲ | lern_too_spel 3 hours ago | parent | next [-] | | Zuckerberg never had scruples, and everyone knew that from the start. | |
| ▲ | senordevnyc 4 hours ago | parent | prev | next [-] | | I find this kind of cynicism fascinating tbh. On the one hand, it seems so relatable in some ways, because there is something uncomfortable about being seen as naive, in a way that being seen as cynical or negative doesn't seem to carry. I guess it's just self-protective, almost like some kind of perverse Pascal's wager: it's better to think everyone is horrible and be wrong than to think the opposite and be taken advantage of? The thing I can't quite square is that it doesn't really fit my lived experience. I have known sincere, genuine people in the types of positions that I'm sure someone like you would declare to be sociopathic. But beyond that, I just don't know why it would actually be true that everyone at the top is a villain. Why couldn't someone like Dario (or even Altman, gasp) be sincere? Because if he is, it does seem like a lot of the moves he's made would make sense given his worldview. But if you assume he's just a villain, then you can twist any of those moves to just be further evidence of that which you already believe. I don't know, I just find cynicism interesting, and a little sad. | | |
| ▲ | kouteiheika 3 hours ago | parent | next [-] | | > But if you assume he's just a villain You don't have to assume anything. A true "good guy" doesn't openly say that he's fine with autonomous, AI-powered weapons being used against me, and mass surveillance applied to me and my family just because I don't live in the US. A true "good guy" doesn't say "privacy is a human right", and then immediately (and completely) bend the knee to an authoritarian government on this issue. | | |
| ▲ | user43928 an hour ago | parent [-] | | OpenAI's agreement with the Pentagon was "No use of OpenAI technology to direct autonomous weapons systems". And about the mass surveillance, I don't see why the military should not use AI to do surveillance abroad. | | |
| ▲ | chillfox 5 minutes ago | parent [-] | | Maybe because it will make people abroad like you less and that has flow-on effects, mostly economic. |
|
| |
| ▲ | solenoid0937 an hour ago | parent | prev | next [-] | | It's a lot easier to sound smart on the internet if you're a bitter cynic. Lots of nerds for some reason have made cynicism a personality trait. They think optimism/honesty is hopelessly naive, therefor cynicism is the correct default. | |
| ▲ | DenisM 3 hours ago | parent | prev | next [-] | | It stops being interesting, or even sad, after a while. People get stuck in all kinds of places, mentally. Some get unstuck eventually. It’s only sad if you have come to a counter factual belief that it could have gone better. I went in the opposite direction - how far can I push myself to see multiple facets of a story? That is a wild ride, and it gets progressively more wild. | | |
| ▲ | bigyabai 3 hours ago | parent [-] | | > how far can I push myself to see multiple facets of a story? Please, I'm dying to hear the optimist's take on Mark Zuckerberg's career. It wouldn't happen to be embarassingly foolish, would it? |
| |
| ▲ | grey-area 2 hours ago | parent | prev | next [-] | | For these particular characters, the evidence is heavily against your panglossian take. All have collaborated with the current US regime. All have shown signs of being quite willing to compromise their principles in order to make money. | | |
| ▲ | solenoid0937 an hour ago | parent [-] | | IDK, only one company has held their two red lines in open conflict with the government, while all the others capitulated to "all lawful use." |
| |
| ▲ | bigyabai 3 hours ago | parent | prev [-] | | It's the Tragedy of the Commons. There will always be a vacuum of predatory bullshit that can be filled, and the victor is always the biggest sociopath. Rockefeller, Cecil Rhodes, Elon Musk, it's the same traceable pattern all the way back through history. It's not that everyone is like this, but that a few crafty marketeers are able to ruin it for everyone. Why should I treat Sam and Dario with special white gloves? Are they different, this time? They have peers in China that do the same research and actually release it to the public. They let you run the production weights on your own machine. Am I a cynic, for comparing these CEOs to their populist superiors? Am I stupid for assuming their hostility when they refuse to give us the benefit of the doubt? I'll believe their actual altruism when I see it. Both are seeped in "boy genius" puffery and lie out their ass. If this is the future of intelligent innovation, then America is truly declining. |
| |
| ▲ | cindyllm an hour ago | parent | prev | next [-] | | [dead] | |
| ▲ | cindyllm 4 hours ago | parent | prev [-] | | [dead] |
|
|
| |
| ▲ | turtleyacht 6 hours ago | parent [-] | | Maybe cache similar answers from others. Surprised if this is not already being done. | | |
| ▲ | wahnfrieden 6 hours ago | parent [-] | | Like google search, this does not work because of how common long tail use is. What you think could be a big chunk, is more likely to be a fraction of a percent of queries. And what use is similar query caching - so you (very often! if actually cost effective, maybe half the time) get a response to a query that was different from yours. Including for when you have a lot of context input already. You’re going to get trash. If it were constrained to only very common initial prompts, and somehow the long tail did not actually dominate as it does with Google search (can't find the reference at the moment but it was a famous article some years ago), it also wouldn't account for serious enough cost savings. Long context is what is expensive. This might only work in constrained domains like customer service where there’s tolerance for generic answers and escalation paths. For technical work? For general purpose use, with secretly canned responses charged at full price? | | |
| ▲ | joegibbs 5 hours ago | parent | next [-] | | But there must be a ton of generic questions that people ask. Stuff like "What's the capital of country X?" - it's probably at least 10% of queries. Memories, custom instructions etc would invalidate them, but if you can return the answers basically free it's probably worth it. | | |
| ▲ | nearbuy 4 hours ago | parent [-] | | Questions like that cost a tiny fraction of a cent. "What's the capital of Sri Lanka?" cost a fifth of a cent at GPT 5.5 API price, and would cost a fraction of that if the question were routed to a more suitable, cheaper model. The output was 78 tokens. By contrast, when coding, devs typically have hundreds of thousands of tokens in the context window, and may use many millions of input tokens per day. Caching requires the full prefix to match exactly. If a single word differs near the beginning of the prompt, nothing after that can share the cache. So this type of caching would save a few queries that cost virtually nothing, but wouldn't help with the stuff where cost matters. |
| |
| ▲ | dools 6 hours ago | parent | prev | next [-] | | I would be very surprised if they hadn’t sorted out some form of shared KV caching | | |
| ▲ | wahnfrieden 5 hours ago | parent [-] | | I wouldn't | | |
| ▲ | ewild 5 hours ago | parent [-] | | people really dont understand how the transformer works to think this is something trivial if possible at all | | |
| ▲ | dools 3 hours ago | parent [-] | | Orly? “ Automatic Prefix Caching (APC in short) caches the KV cache of existing queries, so that a new query can directly reuse the KV cache if it shares the same prefix with one of the existing queries, allowing the new query to skip the computation of the shared part.” https://docs.vllm.ai/en/latest/features/automatic_prefix_cac... | | |
| ▲ | wahnfrieden an hour ago | parent [-] | | This thread is about cutting costs in half for GPT across the board. The technique you linked only makes a substantial difference for particular use cases where you are going to have many LONG CONTEXT queries with the same prefix. For instance, when having a set of documents that commonly get loaded in as context. It's a way for application developers to keep prefixes they manage (or prefixes managed by some set of their users) cached. It has no relevance for long tail general purpose use. |
|
|
|
| |
| ▲ | turtleyacht 6 hours ago | parent | prev [-] | | Please pardon the pure speculation incoming. Yes, caching the answer doesn't seem useful. Caching the progression, the graph, may be. This is similar to making code changes with ed(1) instead of editing in vi. The transform script(s) are cached and can be played back or adjusted. Surely for some breadth of question inputs, they map more often to similar answers--but not static answers; instead, evented edits. It's nearly untenable for a human to keep private edit scripts to generate code changes. The extra steps for custom regex, essentially one-offs for a shared codebase, is inefficient. But maybe not to an LLM. | | |
|
|
|
