| |
| ▲ | A_D_E_P_T 4 hours ago | parent | next [-] | | > You should. Maybe they have it open now just to destroy competition and then they close it. Now you have a closed system, with no alternatives. What makes you think there would be no alternatives, though? Put it like this: Kimi K3.1 is downloadable and can be run locally. (If you have a datacenter.) This is a very capable, ~2.8T parameter model. Now let's say that every Chinese firm goes closed in six months or a year and jacks up pricing... What's stopping a bunch of enterprising upstarts from updating all 2.8 trillion parameters, extending them, distilling the model into very competent flash models, or applying targeted RL? Once the principles are understood and there are models that can be pulled off the open internet, it doesn't seem that hard to spin up your own LLM. You just need some financial resources and a reason for doing it. And that would come into play if the Chinese kill every closed American provider and then go closed themselves. | | |
| ▲ | vlovich123 3 hours ago | parent | next [-] | | Training is really really expensive and open models are not getting funded by VCs. Distillation also doesn’t improve performance. Until training gets exponentially cheaper so that you can do it for a few hundred grand to train a competitive model, open models are labs trying to flood the market to get market share / undercut more successful labs, not a sustainable strategy that will extend indefinitely necessarily. It’s like claiming Uber and DoorDash are fantastic because their prices are so low - once they eat the market the prices become higher than ever. | | |
| ▲ | A_D_E_P_T 10 minutes ago | parent [-] | | Sure, but once they release an open model, it's open forever. You can download it, share it with friends, make a torrent, whatever. So in this sense every open model is a gift to the commons. It's not quite like Uber. It's more like... well... imagine there are two computer operating systems, Gray and Teal. Gray is always slightly better, but it costs $50/user/month for a license. Teal is free. Eventually, when both happen to be in version 9.0, Teal puts Gray out of business. Starting from version 10.0, Teal starts charging $100/user/month from a license. But nothing's stopping you from sticking with 9.0, which remains free forever. Perhaps what's "sustainable" is not clear, given that Gray was eventually dominated by Teal? In any case, versions 1.0-9.0 were, effectively, gifted to the public. We can appreciate (and even disapprove of) that commercial strategy while benefitting from its fruits. |
| |
| ▲ | ericmay 4 hours ago | parent | prev [-] | | > What makes you think there would be no alternatives, though? There could be, they can actually come from open American companies too. Ask yourself why we aren't seeing a thousand or more American AI startups spring into life overnight to build "open" models. Follow the economics. > What's stopping a bunch of enterprising upstarts from updating all 2.8 trillion parameters, extending them, distilling the model into very competent flash models, or applying targeted RL? > And that would come into play if the Chinese kill every closed American provider and then go closed themselves. What's stopping American or European tech companies from doing this today? It's economic and technical capability. I'm very suspicious of any suggestion that it is in any way not incredibly difficult to build and scale up LLMs. Maybe in the future it can be much easier, but in that case you run into a sort of tautology where it becomes easier but the leading companies continue to lead and do so by a significant margin because they're the ones that have the most talent and access to capital. The premise seemed to be "an open China" versus a "closed USA". I'm not sure that's a great premise. If China can create open-weight models so can the US. If the US can close their models (as clearly evidenced by the article) so can the Chinese. | | |
| ▲ | A_D_E_P_T 19 minutes ago | parent [-] | | > What's stopping American or European tech companies from doing this today? It's economic and technical capability. I'm very suspicious of any suggestion that it is in any way not incredibly difficult to build and scale up LLMs. Seems more economic than technical. If the economics are covered, the technical aspects seem straightforward enough, and may perhaps be fully solved. I think there's proof of this in that there are a lot of different LLMs on the market. Even Xiaomi -- in-between making cars, e-scooters, backpacks, and home appliances -- spun-up an exceptionally capable LLM! (And one which is rather idiosyncratic. It's weirdly good at music composition and generation, which both Opus and Astra cannot do.) The economic case doesn't exist for VC-funded US corporations, nor does it exist for cash-strapped nonprofits. In China, all such ventures are likely funded by a government development bureau. If Chinese ventures kill expensive closed providers and universally go closed themselves, there's nothing stopping the US or Europe from copying that same playbook -- unless, as may well be the case, they're ideologically opposed to subsidizing strategic industries. |
|
| |
| ▲ | jacquesm 4 hours ago | parent | prev | next [-] | | Explain to me how they're going to take back the weights they shipped? | | |
| ▲ | ericmay 4 hours ago | parent [-] | | They won't be useful in the future so they won't matter | | |
| ▲ | jacquesm 4 hours ago | parent | next [-] | | They'll be just as useful as they are today. Which is plenty useful. Just like my 10 year old laptop is still useful. I could replace it, but why would I, it serves me well. | | |
| ▲ | glenstein 3 hours ago | parent | next [-] | | >They'll be just as useful as they are today I mean how useful are GPT-3 weights today? That's the kind of useful they'll be relative to the frontier. I feel like the so-called "Deepseek moment" broke too many people's brains and they've lost the ability to reason about why frontier advantages matter. I think we have to learn some new core concepts and rate of improvement over time is going to be a big one, and not understanding that causes people to exaggerate the significance of downstream distillation that's done in response to it. It's cheaper and open and that matters, but it's not the kind of mattering that replaces the relevance or advantage of the frontier, and I feel like I'm not seeing people demonstrate that they know how to really compare the relative merit of these two things so much as zoom in on one to talk past the other. | | |
| ▲ | jacquesm 2 hours ago | parent [-] | | Even the original DeepSeek is old hat. The capabilities already released are game changers, whether you want to acknowledge that or not is not my affair. I was using GPT-3 and the original Llama when they came out and realized that it is interesting, but not good enough - yet. But they are good enough now. And just like there are still people driving Renaults and Volkswagens the latest batch of open models will be useful for a long time to come. |
| |
| ▲ | ericmay 4 hours ago | parent | prev [-] | | I just disagree, I don't think your 10-year-old laptop is very useful either. It's like sure yea you can access the Internet and tinker around or whatever, and with some old LLM model you can have it do fun things, but on a scale of absolute usefulness and if we include what people actually pay money for and what is highly adopted, your laptop and model are severely outdated and by and large irrelevant. That's why the iPhone sells like hotcakes all day every day and twice on Sunday and Apple is a $5 trillion company. Society disagrees with your measure of usefulness. Sorry I'm rate limited since Dang thinks I'm arguing to much on political topics (he's probably right), but to answer a couple of questions regarding "my old technology does this one thing" you are conflating your personal and nostalgic uses for old hardware/software with what is useful for society or business. If the old hardware was still just as useful as the new hardware it would be used instead of the new hardware. | | |
| ▲ | jacquesm 3 hours ago | parent | next [-] | | > I don't think your 10-year-old laptop is very useful either. Technology never was about the tech an sich but about what you do with the tech. For one person that 10 year old laptop is useless, for another it is a source of $500K / year in income. You, the user are the biggest factor in how much value you can extract from the technology at your command. I'm not sure how much longer that will be true but it definitely is still true today. | | |
| ▲ | stickfigure 3 hours ago | parent [-] | | The laptops generating income are the relevant metric here; most software developers would struggle with a 4GB Pentium these days. It matters to your company and your economy if your engineers can work. Compared to laptop quality, I suspect model quality will be a much bigger competitive advantage in the tournament of the marketplace. High-quality laptops are still limited by the quality of the software developer. | | |
| ▲ | jacquesm 2 hours ago | parent [-] | | > High-quality laptops are still limited by the quality of the software developer. For now it is exactly the same with the models. | | |
| ▲ | stickfigure an hour ago | parent [-] | | You're not wrong but the amount of leverage you get out of a better model dwarfs the leverage you get out of a better laptop. Almost to the point that these are different categories of thing. Also... employment in some parts of the stack is genuinely going away. I don't even look at CSS anymore. | | |
| ▲ | jacquesm an hour ago | parent [-] | | > the amount of leverage you get out of a better model dwarfs the leverage you get out of a better laptop To some degree, yes. But 'dwarfs' isn't really the term I would use. It is a bit slower, but the upsides are >> than the downsides for me. Not having Sam, Dario or Xi's minions looking over my shoulder matters to me more than any technological edge. > Also... employment in some parts of the stack is genuinely going away. I don't even look at CSS anymore. Yes, that's a fact, but that's orthogonal. I haven't touched either HTML or CSS in a long time now and yet I've cobbled together (I wouldn't claim ownership) a lot of stuff that 'just works'. I also would not expose that stuff to the world at large because I suspect it isn't nearly as secure as it looks. |
|
|
|
| |
| ▲ | KellyCriterion 2 hours ago | parent | prev | next [-] | | Im developing at home a fairly large data app on a 4x3GHz Athlon with 16GB RAM and a SSD disk with VS.NET + full Microsoft backend tooling. The machine was bought in 2011 and it runs Windows Server. I can do whatever I need with the device, apart from Gaming. Even YT works in 4k. So, tell me now please why my 15 year old computer is not useful here? | |
| ▲ | WD-42 3 hours ago | parent | prev | next [-] | | My daily driver is a 10 year old laptop. Any computer built since Moores law plateaued is plenty capable. | |
| ▲ | hypfer 3 hours ago | parent | prev [-] | | [dead] |
|
| |
| ▲ | mbesto 3 hours ago | parent | prev [-] | | They're already useful. People are shipping real products with them. Also, it's entirely plausible that an previously open model (that then became closed) will be much easier to distill than having a bunch of closed models. |
|
| |
| ▲ | freejazz 3 hours ago | parent | prev | next [-] | | So at worst the Chinese models would do what the US models already do?? | |
| ▲ | FabCH 4 hours ago | parent | prev [-] | | Note that only one country has so far banned foreigners from using their models, and it wasn’t China. | | |
| ▲ | glenstein 3 hours ago | parent | next [-] | | When you're not in the lead, your best strategic move is to neutralize trade secrets advantages of the leaders. I think a strategic approach is being misinterpreted as a demonstration of virtuous ethos, which is crazy given (as the other commenter mentioned) the norm of secrecy when it strategically suitable in basically every other context. | |
| ▲ | ericmay 3 hours ago | parent | prev [-] | | China bans all sorts of things, so this doesn't really matter. I actually view it as a good thing. Americans get the best access to the best technology and we can go on to use it to maintain or gain a leading edge in the progress of our society and businesses. Why would we want to give that away for free so people from countries like Russia or China can build competing companies or use these tools to undermine our society's aspirations? |
|
|
