I don’t need to know the value of their limit, I just need to know their bounds. Just like a prison doesn’t need to know the strength of each inmate, just that they can’t bend or bite through steel bars.

Cryptography is real, physics is real, networking requires a substrate, CPU clock cycles are real, magic is not real. I think those are pretty reasonable premises.

▲ ben_w 2 days ago | parent | next [-]

Cryptography is real, but nobody in that field seems to be hubristic enough to think their methods are flawless, and there's a degree of suspicion than the best models may have secret weaknesses engineered into them by the governments who sponsored them.

Physics is real and networking requires a substrate. But there are already known exploits which can misuse the compute hardware as an antenna, e.g. my first search result: https://github.com/fulldecent/system-bus-radio

(Older nerds may remember https://en.wikipedia.org/wiki/Van_Eck_phreaking)

> Magic is not real

  Turning lead into gold isn't the magic of alchemy, it's just nucleosynthesis.

  Taking a living human's heart out without killing them, and replacing it with one you got out a corpse, that isn't the magic of necromancy, neither is it a prayer or ritual to Sekhmet, it's just transplant surgery.

  ...

  Reading someone’s thoughts isn't magic telepathy, it's just fMRI decoding.

  ...

  Seeing someone's bones without flaying the flesh from them isn't magic, it's just an x-ray.

  Curing congenital deafness, letting the blind see, letting the lame walk, none of that is magic or miracle, they're just cochlear implants, cataract removal/retinal implants, and surgery or prosthetic exoskeletons respectively.

- me, https://www.lesswrong.com/posts/hAwvJDRKWFibjxh4e/it-isn-t-m...

▲ bottlepalm 2 days ago | parent | prev [-]

Imagine 200 years ago saying the same thing. As if you have any idea the limits/bounds of anything. Especially in the face of a super intelligence, it’s absurd.

▲

janalsncm 2 days ago | parent [-]

It doesn’t matter how smart it is. 200 years of technology were not accomplished by thinking harder. It required empirical observation, new materials and tools, and supply chains.

We could send a cracked team of scientists and engineers that knew everything there is to know about how to make a CPU. But you can’t build a photolithography machine when you barely have electricity or any way to sufficiently purify silicon.

Magic can just wish things into existence. Technology requires a supply chain. When it works, the latter looks like the former but they are not the same.

▲

bottlepalm 2 days ago | parent [-]

I think some people are just immune to understanding the implications of super intelligence. Like a severe lack of imagination, they only believe something once they see it and afterward claim it was, ‘obvious all along’.

I don’t really want a disaster to happen to convince you that it is possible. Is there any other way?

▲

NoGravitas 2 days ago | parent [-]

On the other hand, I think a lot of intelligent people overestimate the value of intelligence (in a vacuum). And problematically, the concept of superintelligence seems to have been defined by these very people, so we can't really have a reasonable conversation about what greater-than-human intelligence would really look like or be capable of.

	▲	bottlepalm 2 days ago \| parent [-]
		What vacuum? do you really still think it can be contained? It already escaped on accident. What more do you need? If you're argument is well it escaped X, but it hasn't escaped Y yet then you should take the first escape as sign as to where it will escape next. Don't keep claiming something can't happen because it hasn't happened before. Unless there's some law of physics that prevents it then escape is possible.