| ▲ | godelski a day ago | |
A well functioning market has competition that not only drives down prices but increases quality of products. Obviously customers want to simultaneously maximize quality and minimize price. But when customers cannot differentiate quality, they can only minimize price. Leading to the feedback loop, where producers are in a race to the bottom, making sacrifices to quality in favor of driving down prices (and thus driving up profits). Not because this is actually the thing that customers want! But because the market is inefficient. I think critical to these alignment issues is that they're not primarily driven by people trying to be malicious nor deceptive. They are more often driven by being short sighted and overlooking subtle nuances. They don't happen all at once, but instead slowly creep, making them more difficult to detect. It's like good horror: you might know something is wrong, but by the time you put it all together you're dead. It isn't because anyone is dumb or doing anything evil, but because maintaining alignment is difficult and mistakes are easy. | ||