Lovely thought Ben. Good to hear from you!

I spent a lot of my life and money thinking about building better algorithms (over five years).

We have a bit of a chicken / egg problem. Is it the algorithm or is it the preference of the Users which is the problem.

I'd argue the latter.

What I learned which was counter-intuitive was that the vast majority of people aren't interested in thinking hard. This community, in large part, is an exception where many members pride themselves on intellectually challenging material.

That's not the norm. We're not the norm.

My belief that every human was by their nature "curious" and wanting to be engaged deeply was proven false.

This isn't to claim that this is our nature, but when testing with huge populations in the US (specifically), that's not how adults are.

The problem, to me, is deeper and is rooted in our education system and work systems that demand compliance over creativity. Algorithms serve what Users engage with, if the Users were to no longer be interested in ragebait, clickbait, focused on thoughtful content -- the algorithms would adapt.

> Is it the algorithm or is it the preference of the Users which is the problem. I'd argue the latter.

> Algorithms serve what Users engage with

User engagement isn't actually the same thing as user preference, even though I think many people and companies take the shortcut of equating the two.

People often engage more with things they actually don't like, and which create negative feelings.

These users might score higher on engagement metrics when fed this content, but actually end up leaving the platform or spending less time there, or would at least answer in a survey question that they don't like some or most of the content they are seeing.

This is a major reason I stopped using Threads many months ago. Their algorithm is great at surfacing posts that make me want to chime in with a correction, or click to see the rest of the truncated story. But that doesn't mean I actually liked that experience.

> Algorithms serve what Users engage with, if the Users were to no longer be interested in ragebait, clickbait, focused on thoughtful content -- the algorithms would adapt.

Algorithms have been adapted; they are successful at their goals. We’ve put some of the smartest people on the planet on this problem for the last 20 years.

Humans are notoriously over-sensitive to threats; we see them where they barely exist, and easily overreact. Modern clickbait excels at presenting mundane information as threatening. Of course this attracts more attention.

Also, loud noises attract more attention than soft noise. This doesn’t mean that humans prefer an environment full of loud noises.

It is not the norm here either.

>This community, in large part, is an exception where many members pride themselves on intellectually challenging material.

That's not the norm. We're not the norm.

I recommend against putting HN on a pedestal. It just leads to disappointment.

All you need to do is read the other comments on this very page and you will see that there are very strict cultural and political norms here too, but for some reason they are invisible as such to those who hold them. They consider their views to be "common knowledge" and "what any reasonable person believes" because they, too, live in curated bubbles.

Any comment that challenges mainstream science, materialism/physicalism, and leftist politics gets downvoted into oblivion here because HN is definitely not a haven for people who "pride themselves on intellectually challenging material."

TL;DR: It's an echo chamber here, too, but most people who hold the worldview that is enforced here often cannot see their own presuppositions, nor do they see that their views are political in nature.