Remix.run Logo
▲ thuo 4 hours ago

I'm one of those people who's skeptical of recursive self-improvement and superintelligence. My rationale for my skepticism is that the algorithms get too much credit for the results. The algorithms ultimately run on hardware built by humans and learn from data generated by humans. In the same vein, I think knowledge work gets too much credit for building things and generating high quality data. The output of knowledge work is only useful if it can be applied to the physical world e.g. a CAD file is useful when the actual thing it describes is made in the real world. Similarly, high quality data comes from several feedback cycles with the physical world. I think people generally underestimate how much of human capability is driven by our sensorimotor skills. I think the idea of recursive self-improvement / superintelligence is just an extension of this attitude.

Humanity as collective does not buy anything, or send the design to the plant to be made. As a collective, humanity transforms the natural world into things, for example, people take fibres from plants, turn them into yarn, weave yarn into fabric, and then turn fabric into clothes. Similarly, humanity as collective does not "buy" chips, people turn quartz, bauxite, copper ore, petroleum, etc into the components of computer hardware.

There's a common adage about how everyone should work as a service person to learn patience and empathy. I think everyone should also work in an assembly line and a mine to learn about where the things they use come from.