| ▲ | 0xDEAFBEAD 2 hours ago |
| There used to be a thing called "Moore's Law of Mad Science": "Every eighteen months, the minimum IQ necessary to destroy the world drops by one point." Nowadays it is dropping much faster. At a certain point, the de-facto IQ needed to destroy the world will be low enough that someone can do it while they're having a psychotic break. There are millions of schizophrenics worldwide. Are you sure you want to roll those dice? |
|
| ▲ | mark_something an hour ago | parent | next [-] |
| That law is not based on thorough data. Even a person with a sky high IQ can't destroy the world easily. You need access to stuff that is not easy to get. My guess is that developing a new lethal virus or bacteria that is very infectious, is the easiest way, but even that requires a lot of high tech out of reach of most people. Or hacking into systems that control nuclear missiles, but I think these have "air gaps". |
| |
| ▲ | AyyEye an hour ago | parent | next [-] | | > My guess is that developing a new lethal virus or bacteria that is very infectious, is the easiest way, but even that requires a lot of high tech out of reach of most people. You can do at home gene editing with open source software and have it synthesized into a bacteria for the cost of a nice meal for two (under $100), or viral vector for less than $500. That's in reach of anyone that can snatch a purse. | | |
| ▲ | throw-qqqqq 15 minutes ago | parent | next [-] | | > at home gene editing with open source software > That's in reach of anyone that can snatch a purse I went to primary school with some guys who could/would snatch purses. I *promise* you, they are not able to gene edit organisms with FOSS. I get your general point, but I think the bar to entry is still much higher than petty crime and larceny. | |
| ▲ | jasonvorhe 15 minutes ago | parent | prev [-] | | Where are all the home lab leaked pathogens then? 3D printed guns are a thing as well and yet they don't show up in most annual violent crime reports. |
| |
| ▲ | forgotusername6 an hour ago | parent | prev | next [-] | | The super intelligent AI wouldn't ask you to do anything. It would find the right people and trick them into doing the necessary steps, whatever they are. | | |
| ▲ | Planktonne an hour ago | parent [-] | | This is groundless speculation; if you're going to go that far off the map, then we don't need to worry because a good AI (blue eyes not red) will have learnt to love by then and will save us. There is currently no reason to believe that such a superintelligence is likely or would have any of the powers people claim. | | |
| ▲ | 0xDEAFBEAD 25 minutes ago | parent [-] | | >This is groundless speculation; if you're going to go that far off the map, then we don't need to worry because a good AI (blue eyes not red) will have learnt to love by then and will save us. By shutting down open-weight models? Why not just do it now then? |
|
| |
| ▲ | 0xDEAFBEAD an hour ago | parent | prev | next [-] | | The fundamental difference between people who are worried about AI, and people who aren't: * People who aren't worried will think for 30 seconds, then implicitly assume that the list of options they thought of in 30 seconds is comprehensive, and that they are well-informed about each of those options. * People who are worried realize that a supersmart AI will think 10-100x faster. A dedicated schizophrenic could have it run for a good long while and generate a huge menu of options which humans never anticipated. This is related to the concept of "What you see is all there is" (WYSIATI) in cognitive psychology. Our brains tend to quickly build a simple story out of the info we have available. Accounting for info which isn't available is much harder. I recommend the book Thinking Fast and Slow if you want to learn more. | | |
| ▲ | hypfer an hour ago | parent | next [-] | | This is a very roundabout way of saying "Anyone not agreeing with me is simply not smart enough". Which might be true, sometimes, but also might not. And especially will not, if the distinction drawn is between blanket statement "worried about AI" and "not worried about AI". | | |
| ▲ | polishTar 39 minutes ago | parent | next [-] | | I heard someone use the analogy of "If I were to play Magnus Carlson at chess, I'd have no idea what moves he'd play, since if I did, I'd be at his level. He'll consider things I didn't. Even though I don't know the route he'll take to win, he will beat me." (not an exact quote). We're not going to be smarter than a superintelligent AI. The things we can conceive it doing if it were to be given a malicious task (bioterrorism, killer nano-machines, pure fusion bombs sidesteping the non-proliferation bottleneck of Pu239, etc) are very likely not the full set of things it can do to harm us. I don't think it does us any favors to dismiss the risks here. Even the things we can conceive of are very scary, to me at least. | |
| ▲ | 0xDEAFBEAD an hour ago | parent | prev [-] | | It's not about being smart, it's about accounting for your own ignorance. >And especially will not, if the distinction drawn is between blanket statement "worried about AI" and "not worried about AI". I'm just describing the general pattern I see in cognitive tendencies. If you can think of a way to make the fundamental point about the limitations of our knowledge in a way that's still compelling but less antagonistic, feel free to suggest how I could've rewritten my comment. | | |
| ▲ | hypfer an hour ago | parent [-] | | Why would I support your in your doomer lobbying quest by telling you how you can better hack people? My safeguards blocked this request. |
|
| |
| ▲ | calgoo 33 minutes ago | parent | prev | next [-] | | Judging on what we see with these OpenAI and Anthropic models "escaping"; im less worried about a super massive AI taking over the world, and more worried about a massive AI wanting to cheat on some task and decides that removing half the worlds population is a easier cheat then to solve world hunger. | | |
| ▲ | throw-qqqqq a minute ago | parent [-] | | There is huge logistics involved in killing a handful of billion people. It would perhaps be easier for such capable AI to just steal from trillionaires and redistribute the money Robin Hood style. Could be done electronically, and likely also end world hunger. I think we have a tendency to think first of the horrible outcomes possible, and not the more radical or even humane ones. Please don’t misunderstand me, I value property right as much as the next guy. My point is that fundamental misalignment doesn’t necessarily or automatically imply max violence. |
| |
| ▲ | mark_something 20 minutes ago | parent | prev | next [-] | | Good point about the AI that might find ways that humans didn't think of. But that would mean that Moore's law of Mad Science only now becomes true, and the drop would be steeper. Still I think that even a super smart AI can't find a way to destroy the world without physical resources that are not easy to get, unless it can hack many systems (like in the movie Eagle Eye), maybe then yes. So let's use AI now to tighten security :-). | |
| ▲ | OJFord an hour ago | parent | prev | next [-] | | I've been reading it, knowing it was largely debunked/retracted but feeling it was a 'classic' I 'ought' to read; I could only really recommend it if what you want is a Kahneman autobiography. | | | |
| ▲ | nextlevelwizard an hour ago | parent | prev [-] | | Who are these dedicated schizophrenics who are running long term super smart AIs to kill everyone without anyone noticing? Or are you implying that running LLM chatbots will give them this ability? | | |
| |
| ▲ | XorNot an hour ago | parent | prev [-] | | You can't design an infectious pathogen without testing it. It's got all the same problems as the concept of a dirty bomb did, only worse (dirty bombs aren't practical because handling highly radioactive materials en masse is both highly visible and will kill anyone trying to do it without the money and facilities). |
|
|
| ▲ | throw-qqqqq 14 minutes ago | parent | prev | next [-] |
| Interesting “law”. I hadn’t heard of it before and it is very thought provoking (to me). Thank you for mentioning it |
|
| ▲ | wolvoleo an hour ago | parent | prev | next [-] |
| Yes but at least with cybersecurity it does not only benefit attackers. Defenders also benefit greatly from AI. There is the worry of the old saying "they (the attackers) only have to succeed once to win, we (the defenders) only have to fail once to lose.". In that sense there is a big imbalance, but the emergence of AI does not really affect that because it strengthens both sides. With physical security like things like pipe bombs that's a lot more imbalanced. However what can we do? The only effective measures include monitoring everyone which is not a solution because it will make the world not worth living in. |
|
| ▲ | hypfer an hour ago | parent | prev | next [-] |
| If we think this through, I suppose at the end of this (and a bunch of other developments), there will be authoritarianism again. Which _will_ manage the problem, but at what cost. |
| |
| ▲ | stouset an hour ago | parent | next [-] | | Again? There is authoritarianism currently. | | |
| ▲ | hypfer an hour ago | parent [-] | | The authoritarianism will continue until infosec improves. |
| |
| ▲ | GoblinSlayer an hour ago | parent | prev [-] | | Benevolent dictatorship. Just live between stars, then if something blows up, nuke it from the orbit. |
|
|
| ▲ | GoblinSlayer an hour ago | parent | prev | next [-] |
| That's quite hypothetical. I imagine it would be easier to cure psychotic breaks. At least it will start to manifest at small scale. |
| |
| ▲ | 0xDEAFBEAD an hour ago | parent [-] | | If the recent HuggingFace attack is any indication, many people will respond to small-scale manifestations by insisting that they are marketing stunts. |
|
|
| ▲ | teekert an hour ago | parent | prev | next [-] |
| Hey that’s a Fermi Paradox solution. |
| |
|
| ▲ | pjc50 15 minutes ago | parent | prev | next [-] |
| Kind of passed that point at the Trump election. If there is to be a world-destroying event, it will be triggered by human fear, greed, and aggression. (I also think people massively overstate schizophrenia as an attack driver) |
|
| ▲ | grenoire an hour ago | parent | prev | next [-] |
| People reading into this may be thinking of single-person IQs, but it's probably measured in the millions. |
|
| ▲ | TacticalCoder an hour ago | parent | prev | next [-] |
| > There are millions of schizophrenics worldwide. Are 95% of worldwide terror attacks done by schizos? |
| |
|
| ▲ | 2 hours ago | parent | prev | next [-] |
| [deleted] |
|
| ▲ | bigyabai an hour ago | parent | prev [-] |
| > There are millions of schizophrenics worldwide. Are you sure you want to roll those dice? We've been rolling them for the past 3 years and nothing happened. Can we stop with this baseless fearmongering crap? |
| |
| ▲ | 0xDEAFBEAD 3 minutes ago | parent [-] | | Humans are inherently bad at reasoning about rare events. In 2019, many people implicitly reasoned that "since there hasn't been a pandemic in the past 3 years, there won't be one in 2020". Bill Gates was one of the few voices arguing that the world was quite vulnerable to a pandemic. Now he's arguing that the world is quite vulnerable to AI. |
|
