Remix.run Logo
bwestergard 2 hours ago

A counter argument: A strong distinction between "intelligence" (understanding what is) and "values/principles" (understanding what ought to be) was characteristic of much early modern European philosophy from Descartes to Kant, which received its influential strong formulation from David Hume.

But trying to maintain this distinction leads to insuperable difficulties. Our conceptual framework for understanding the world are always value-laden. There is no "view from nowhere", no historically unconditioned set of values or concepts. Your framing, in which "values" are external to "intelligence" and must be imposed on it (on pain of intelligence being "value-neutral"), leads inevitably to the dead end of "AI Alignment", "superintelligence", etc. Which is a kind of pseudo-theology.

"We humans better [be] refocusing our energy on our core values/principles, given most of our skills are becoming irrelevant."

In light of the untenability of a strong fact/value or intelligence/ethics distinction, I would suggest this alternative advice: humans should focus on critical appropriation and extension of the received wisdom, whether that comes to us directly from human beings or indirectly through an LLM. Perhaps this is compatible with the spirit of your original suggestion.