| |
| ▲ | jcgrillo 2 hours ago | parent | next [-] | | But the things they say they believe are insane and totally unmoored from physical, societal, and economic reality. If they actually believe those things they're untrustworthy because they're delusional. If they don't, they're untrustworthy because they're fraudulent. Either way it's not good.. | | |
| ▲ | reducesuffering 2 hours ago | parent [-] | | They're not. They're in the eye of the storm and see what's going on the clearest. They were ahead of the curve to be where they're at now, and they're still ahead of the curve for where we're going. All the other heads of labs like Sam Altman and Demis have been saying the same thing since 2015-2016 way before any of this "marketing" would ever have been at play. | | |
| ▲ | jcgrillo an hour ago | parent [-] | | There's a simpler explanation that fits the data better: they're lying. Generally, in the past when tech companies have made outlandish claims that were not backed by evidence, they're later found out to have lied. This is an ancient pattern going back to the dotcom era and before, but for recent examples you need only look back a few years to the web3 era. If they're not lying, they can show it by producing the results they claim. Until then, they're probably just lying. | | |
| ▲ | estearum an hour ago | parent [-] | | What data does "they're lying" fit better than "they're earnest?" > If they're not lying, they can show it by producing the results they claim. Until then, they're probably just lying Brilliant framework: Anyone making claims about the future is not just speculating, not just wrong, but they are lying. |
|
|
| |
| ▲ | shimman 2 hours ago | parent | prev [-] | | What are you referring to? The cult belief that they are ushering in a machine god or that they strictly care about making as much money as humanely possibly while ignoring the absolutely destructive impacts these companies have had on society? IMO they are using the cult messaging to distract the public so they take out all the oxygen in the room regarding people that care about the immediate impacts (climate exacerbation, ease of scamming, degrading job prospects, increasing income inequality). Whenever real concerns are brought up against these companies they are always ignored while claiming the real concern is the fantasy of a machine god turning into skynet. | | |
| ▲ | estearum 2 hours ago | parent [-] | | "Why don't they just not participate in the arms race?!" - guy who's never heard of arms races If they believe they're creating "a machine god" and that it's better it's their machine god than someone else's (which, given the other contenders, I tend to agree with), then all the corollaries you mention are mostly irrelevant. Whether you believe they're creating a machine god is irrelevant. They believe that they are. It would be helpful if you could create an actually good argument for why they cannot or are not creating a machine god, but it turns out there are no good arguments for why it's impossible to do so. And so... they shall try. | | |
| ▲ | shimman 2 hours ago | parent [-] | | Oh okay, they're all just legit crazy and are allowed to poison the environment, murder teenagers, and ruin the material lives of millions for fantasy level delusions. Good to know. |
|
|
|
| |
| ▲ | estearum 2 hours ago | parent | next [-] | | Let's assume that Anthropic believes they're in an arms race to create a potentially dangerous technology, and they believe they're the best ones to win this race. Unlike nuclear weapons, advancing in this arms race requires actually deploying the product over and over again. Deploying the product makes your advancements visible to your competitors. It makes complete sense to try to limit the degree to which that's true. | | |
| ▲ | sobellian 2 hours ago | parent [-] | | It's an interesting assumption. The idea behind this with nukes was that we'd like to nuke Germany before they could nuke us. Even after we defeated Germany, we nuked Japan even though they had no possibility of getting their own nukes. The nuclear 'race' was based on the premise that the winner could use it to destroy all other racers (a faulty assumption, see the USSR among others). I will charitably assume Anthropic does not intend to literally destroy anyone and merely wants to become an AGI monopoly. But if AGI is so powerful, any monopoly would not be stable since the incentives for entry into the market are massive. Why would China stop developing AGI just because Anthropic has it? | | |
| ▲ | estearum 2 hours ago | parent [-] | | Do you believe the current situation is more akin to the race to the first nukes, where no one could know for sure the other competitors were even racing... or is it more similar to the Cold War, where there were obviously competitors engaged in the race? And yes, agreed the equilibrium dynamics for AGI are very different (and far harder to predict) than nukes. That sounds like a good reason to be sure we get there first since presumably any potential advantage wouldn't go to the second or third runner-ups | | |
| ▲ | sobellian 2 hours ago | parent [-] | | I can't really say I see a similarity to either the Manhattan Project or the Cold War. I don't see how one could apply either massive retaliation or MAD. These are private companies, they are not vested with the necessary authority to destroy anything. Even if they had it, they couldn't. You can't destroy China, they have 1.4B people, nukes, and a large part of the world's manufacturing. So multiple organizations want to do something first, that could be anything from nukes to railroads to lining up for communion wafers. | | |
| ▲ | estearum an hour ago | parent [-] | | You think "arms race" is a dynamic that only applies to literal arms? "Ability to literally destroy the other entity" is not a necessary or even typical feature of arms races. | | |
| ▲ | sobellian an hour ago | parent [-] | | Well it's difficult to argue against something that was never specifically stated. If someone is able to state specifically how this is an arms race in any other way than that it's a race at all then I'm happy to have that conversation. | | |
| ▲ | estearum an hour ago | parent [-] | | "Arms race" is the term used colloquially to describe the dynamic that emerges in "winner-take-all" markets. It seems that the frontier labs believe they're participants in a winner-take-all market. Therefore they're in "an arms race." Winner-take-all markets do not require that the winner literally destroys the losers, but only that the winner enjoys disproportionate returns compared to their actual superiority. Whether or not this is actually true is TBD, but I think you're naive to think the frontier labs do not believe this to be true. | | |
| ▲ | sobellian 44 minutes ago | parent | next [-] | | I don't know why you think I'm taking anything literally, cf. my first comment. I understand what a metaphorical arms race is. I don't think that Anthropic can forestall others' AI development by getting there first. It can't be literal destruction. It can't be economic destruction (some actors interested in it aren't motivated by money). What's left? I'm all ears. As far as naivete, wouldn't it be more naive to take their EA claims at face value, rather than the more realistic assumption that they like money? | | |
| ▲ | estearum 32 minutes ago | parent [-] | | > These are private companies, they are not vested with the necessary authority to destroy anything You're pretty explicitly saying that dominating the competition is not the type of "destruction" necessary to qualify as an arms race. > As far as naivete, wouldn't it be more naive to take their EA claims at face value, rather than the more realistic assumption that they like money? Huh? Greed is – quite obviously – the major driving force behind the arms race. That is not a mitigation whatsoever. | | |
| ▲ | sobellian 29 minutes ago | parent [-] | | > I will charitably assume Anthropic does not intend to literally destroy anyone and merely wants to become an AGI monopoly. |
|
| |
| ▲ | zozbot234 15 minutes ago | parent | prev [-] | | Creative destruction is absolutely a thing in the market, but the way things are going it seems more likely that open source models will just destroy everything else as far as most users are concerned. The big proprietary labs will be effectively left with Fable, GPT-Pro and Gemini Deep Research - stuff that by all indications needs very large scale compute to even feasibly run. We'll probably find out that each has its own strengths, weaknesses and viable niches, so there's no reason to expect any of those models to utterly destroy the others. They can all survive as specialty services. | | |
| ▲ | estearum 5 minutes ago | parent [-] | | Sure, but: > Whether or not this is actually true is TBD, but I think you're naive to think the frontier labs do not believe this to be true. |
|
|
|
|
|
|
|
| |
| ▲ | Terr_ 2 hours ago | parent | prev | next [-] | | Or if Google Chrome were blocking/degrading access to sites and services that might be useful to someone trying to make a competing web-browser. P.S.: On reflection, it's even worse than that, because it'd trigger based on anything the user types or reads on any site. Someone mentions a "critical rendering path" and now you can't participate on that thread in the Blender forums. | |
| ▲ | jstummbillig 2 hours ago | parent | prev | next [-] | | > Then what is it they are trying to guard against, if its not simply protecting their moat ahead of their IPO? Let's just assume it was "only" that? It's unreasonable to assume they are aiming to upset people who are just giving them money in the way they want. It makes no business sense, for any company. So that has to be a byproduct. Model training is one of the more expensive undertakings in the world right now and distilling models from competitors against the TOS is apparently something that is going on for very little money. Why would they not "just" try to take measures against that? | | |
| ▲ | thewebguyd 2 hours ago | parent | next [-] | | It's about how they took measures against it. Sabotaging the requests is super shady and breaks all other areas of trust in the company their models. All they had to do was have a simple, transparent output "Sorry, that request is against our terms of service. This session has been terminated" | |
| ▲ | zozbot234 2 hours ago | parent | prev [-] | | The hidden safeguard was not against distilling, it was against "frontier" ML research with no indication whatsoever of what "frontier" might mean, but possibly even including research into model safety or alignment. That amounts to deliberately boobytrapping research across an entire legit academic field, which is ridiculously unaligned behavior. | | |
| ▲ | solenoid0937 2 hours ago | parent [-] | | This is the same as saying "well some unaligned countries will use refined nuclear material for energy, too!" lmao. The vast majority of frontier research is about how to build better models, not about alignment. | | |
| ▲ | zozbot234 2 hours ago | parent [-] | | And as a matter of fact, there's a lot of meaningful research into how to have different sorts of nuclear material that might be usable for power production but not hidden malicious development. That's the closest analog to "safety" and "alignment" in your scenario. |
|
|
| |
| ▲ | whimsicalism 2 hours ago | parent | prev [-] | | They are trying to guard against other people building ASI before they do because they think they are uniquely safety oriented relative to their competitors. Frankly, based on my knowledge of Anthropic and the people who work there, they are very possibly right. They care a ton about this in a way that is difficult for people outside this bubble to understand. | | |
| ▲ | zozbot234 2 hours ago | parent | next [-] | | ASI? We are nowhere near even human-like AGI. We have no idea if ASI is even physically possible, but going by the usual scaling laws and the capabilities of existing models, it would require raw compute and storage on an extreme scale, at the very minimum rivaling the existing AI datacenter deployments. (When Dario talks about hosting "a country of geniuses in a datacenter" at some point - which is not even ASI yet as generally projected - the operating word there is datacenter. That's the scale of buildouts you should be thinking about.) This is nowhere near a serious concern at present. | |
| ▲ | thewebguyd 2 hours ago | parent | prev | next [-] | | > guard against other people building ASI before they do because they think they are uniquely safety oriented relative to their competitors All this longtermism though is harmful. There are real problems of data theft, bias, labor displacement, and environmental costs that are happening right now but every push for regulation and regulatory capture, and all the safety talk, is always focused on some speculative future machine god to distract from the current problems. I'd have a higher opinion of these labs if the issues they openly talked about and worked toward where the real issues we face currently, not speculative defenses against some future AGI that may never happen in my lifetime. I'm less worried about "our new model might kill all humans in the future" and more worried about how we are going to address anti-competitive behavior, copyright protections, labor rights, and the energy impact. | | |
| ▲ | whimsicalism 2 hours ago | parent [-] | | I cannot overstate how much I think this take is wrong. Please please reconsider, look at the rate of progress being made, and consider that even if you only think ASI 'may' never happen in your lifetime it should still be one of your #1 concerns. Honestly, that respect for 'copyright protections' has somehow become a leftist shibboleth is bizarre to me and indicative that something has become deeply warped in our discussions around this topic. | | |
| ▲ | nozzlegear an hour ago | parent | next [-] | | > I cannot overstate how much I think this take is wrong. Please please reconsider, look at the rate of progress being made, and consider that even if you only think ASI 'may' never happen in your lifetime it should still be one of your #1 concerns. Frankly, this appeal comes across as the same kind of impassioned plea that a missionary might make when begging the faithless to repent and come to Christ before it's too late. This weird religiosity some people around here use to talk about AI, ASI and AGI is bizarre. Take what I've quoted and replace the words "progress" and "ASI" with "sinning" and "the Book of Revelations", and the zeal becomes apparent. | | |
| ▲ | whimsicalism 14 minutes ago | parent [-] | | Maybe if you really squint. I'm asking them to reconsider their views because the cumulative result of many opinions is policy. And yes, I'm making moral claims. So perhaps that makes it religious? I don't really think so, but I recognize that comparing things to religion is an effective dismissal tactic on here. |
| |
| ▲ | thewebguyd 2 hours ago | parent | prev [-] | | There's nothing warped about it at all. Like it or not, it is a real issue. It's also an issue of license washing GPL code to privatize it. It's full scale theft of collective human knowledge, being sold back to us in a for profit private product. Outside of that though, there are other issues right now that need addressed before we speculate about what might be possible with ASI in the future. If the potential for a harmful ASI is truly that near, and that great, then why push forward at all? Where's the push for a global stop order on development of this technology until regulation can catch up? The talk of a potential future serves as a distraction from the very real problems people are facing in their lives today. While Dario and team are worrying about ASI, real people are worrying about how they are going to continue to feed their family after wide spread layoffs set a very large portion of the population back into a lower quality lifestyle. Real people are concerned about water usage is draught stricken areas, the massive energy demand driving grid instability in their communities, or that the environmental and economic externalities of model training is being socialized while the profits continue to be strictly private. What about the mass proliferation of misinformation at scale having a real effect on our democratic process? Forgive me if I'd like to see those addressed first, and fast, before we start worrying about an unpromised future technology. | | |
| ▲ | oncensher 9 minutes ago | parent [-] | | The "global stop order" is just generally perceived as an impossible coordination problem. So instead we see a mix of labs voluntarily putting in guardrails and regulatory efforts (which are not only aimed at hypothetical super-AIs of the future). Of course labs are also in a competitive race. And I actually think that it does make sense that the richest companies in the most dominant positions would in a better position to worry about safety than a startup that is just trying to survive at all. And just in general, it seems reasonable that the fewer companies have access to dangerous tech the better. This isn't really about some highly speculative future tech either -- current models already pose lots of risks, and the pace of model improvement is something wildly unprecedented. Whether or not you call it ASI, the capabilities we will have two years from now are hard to even imagine properly. Also, I don't think the issues that you are highlighting are all ones that Anthropic would dismiss as second-tier. In particular, mass unemployment from AI is how we will deal with a massive devaluation of human labor is one of the most serious concerns. And about other issues, reasonable people may differ. I'm more worried about biorisk than environmental damage, for example, but clearly we should be keeping an eye on both. Serious risks and problems, just because they aren't already harming people today, are not just a distraction. |
|
|
| |
| ▲ | pishpash 2 hours ago | parent | prev [-] | | Define safety oriented. |
|
|
