I think beyond the specific elements of any domain AI eclipses humans in, overall, the notion of the thing that grants you status, prestige, and a living wage being commodified would spark in anyone a profound sense of dread.

Beyond just making you redundant in your job, the whole architecture of your existence, the thousands of hours of struggle, the sacrifice of putting off social games or personal enjoyment are now rendered moot in hindsight. You are made one step closer to complete worthlessness, which will surely continue as AI becomes a more satisfying personal companion, a more worthwhile friend, lover, etc.

How is one to feel safe in the progress of the future when the only hints of the wonderful post-scarcity utopia we can quantify so far is the removal of all human leverage via their labor? All it tells us is we will be bargaining with superintelligence with nothing to stand on, only a trust that the 5000 lab executives and researchers solved the most important research problem of all time on the first try.

And remember, these AI models were trained on your work, your culmination of effort and sacrifice, without any consent, credit, or compensation, then sold back to you for a subscription fee.

> You are made one step closer to complete worthlessness, which will surely continue as AI becomes a more satisfying personal companion, a more worthwhile friend, lover, etc.

I recently thought about this and fear that AI may further drive us humans apart.

Instead of working with humans, I increasingly work with a fleet of agents.

> the notion of the thing that grants you status, prestige, and a living wage being commodified would spark in anyone a profound sense of dread.

Not for those without status (which is most people on this planet). But yes, AI is set to wipe out most established hierarchies and their gatekeepers, and to equalize most fields. A profound sense of dread is something most ordinary people live with throughout life, and if AI can reduce systemic unfairness, I bet most people will actually find some degree of relief from that dread. The complacency of the formerly-statused being shocked along the way isn't much of an argument against that equalization.

> Beyond just making you redundant in your job, the whole architecture of your existence, the thousands of hours of struggle, the sacrifice of putting off social games or personal enjoyment are now rendered moot in hindsight.

This passage answers itself, doesn't it? You can sense those things you postponed fun for (work and the apparatus around its struggle) were never worth it. So why fight for us all to have to keep working just to live? Isn't a post-scarcity world the thing we should all be fighting for?

It's more or less reverse to the mean. The period in which a people had both access to education and leverage through their labor and intellect is super short. For most of recorded history, the bulk of humanity lived as peasants, serfs, or slaves. They were mostly illiterate, bound to the land or to a master, and had little power to bargain over the terms of their (or their children) lives.

I think the fix is to get started on post scarcity economics now. Sooner or later we're going to have to come up with practices that boost or limit endeavors based on how people feel about them, rather than based on how much money somebody has.

It doesn't have to be ((post scarcity) economics), it can be (post (scarcity economics)). Both systems can coexist for a while, one providing a bit of a safety net while the other falls apart.

We'll be free of cancer, and dead on the inside.

> the removal of all human leverage via their labor

I’m not convinced at all that SI will ever be able to produce the correct result with little input. The advancements in capability I’ve seen in models is that they understand my intention better, and follow industry best practices better, so their output is often acceptable. But I absolutely have to make my intentions known, as detailed as possible, or I will get the mediocre, standard solution that falls out of the weights.

I’ve been having trouble with models always steering towards certain solutions - they insert a queue here, a mutex there, a backoff over there. If I remove those, and let the model freely rebase the PR stack, then it will re-insert those mechanisms because it has been specifically trained to do exactly that. It’s essentially boilerplate, smoothly aligning my codebase to match their weights. Which is often good enough! They are training the models well. But individual problems require individual solutions.

My personal take on the human part in all this is: We get to decide, and the models obey. Our decisions are now super leveraged. So let’s make good decisions, and communicate them well. The layer of where decisions yield the most leverage, has been pushed upwards (I see it on the type and function signature level, others will see it even higher), but IMO it won’t be possible to push it up much further without losing truth to the decisions we need to make for a good product.

Counter point: AI is trained on output from humans. We all made AI, together. It belongs to all of us. We use it to benefit us, and we celebrate our wins together.

The only thing constant in life is change.

You are made one step closer to complete worthlessness

Nobody's worth is tied to source code output.

Yes, and I think over time there will be a blatant inequality in who is commoditized and who isn’t.

If AI engineers didn’t create the data to produce the models, and they increasingly don’t write the code to produce the models either, then why are they treated as the agents of change rather than the subjects of change, like mathematicians are?

You know we (tech as an industry) have done this to a bunch of people already, right?

Thats what "disrupt" meant

It's mainly about status. No one cares that much about AI automating drug discovery or coding, but it's bad when it assists in math. Math results are attached to individuals , whereas for cancer drugs or other medical discoveries, it's more of a team effort by a lab. AI now has the effect of turning math into something that is more lab-like and less about the individual.

When I was a kid I built my entire identity around being #1. I was the most talented kid in school, anything endangering that resulted in an instant identity crisis.

Then I went to college and I understood that there are people who, for one reason or another, are thinking in ways so advanced I'll never catch up. I'll never be the #1. I'm actually much closer to being average than I thought.

But this doesn't mean I should just give up, lie down in my bed and cry. Since then I've learned a lot of skills - most of them being crap at best, but that's not the point. What matters is my own subjective experience. The fact that despite objectively not being good, I see myself as a skilled person with rich life.

Once I was watching a YouTube video about elderly care and there was a phrase along "These people won't achieve anything more in their lives, my goal is to keep them occupied until they die" which is a bit morbid at first glance, but it actually makes so much sense as a life philosophy.

I would answer to this that the mathematicians (humans) are the ones who give any meaning to mathematical discoveries. If we don't have mathematicians anymore, the result is that AI can find whatever they want, but 99% of that will be useless waste. Outside of practical applications, the only reason most mathematical results have any interest is that lots of mathematicians spend time trying to find their solution. Once humans stop caring, mathematics done in an automated fashion will also stop having any value, other than its possible use in finding solutions to existing engineering problems.

> bargaining with superintelligence

This part is nonsense, still. There's no known AI -> AGI path, and superintelligence doesn't mean anything, and AGI is assumed to be (going to be) both malicious and a slave, both for no good reason.

Nah. There is nothing beyond being made redundant in your job and labor leverage. That is all. It has got nothing to do with resentment over fucking machines. It’s all about people (the executives that you mention) fucking us over. The end. Completely human problem.

I know I’m being sold some bullshit when this is made into something inherently existential and psycho-identitarian. An egalitarian future of labor-worthlessness means that there is nothing above us to dictate anything at all—maybe not even nature itself, save for meteors and other cosmic events. I’m not afraid of being “worthless” in that world of complete freedom. Fuck no.

“Worthlessness”

Consider this… in the grand scheme of things, we are on a tiny planet in the universe. What cosmic contribution could you make?

Most people won’t make anything even close to “a ding in the universe”, as Steve Jobs has been reported to say.

So they may as well enjoy their life. And your worth can be to your family, to your children, your parents who took care of you.

Learning a hobby. Practicing religion. Mindfulness and meditation. Doing sports and improving your own body.

Look around if you are a heterosexual man, for instance: many women are far more attracted to men that perform in the moment, than men who labor quietly to create a product. The former enables untold hours of honing social skills and calibration, and humor and performance skills. The latter is of more durable value to society, yet relatively few women are genuinely sexually attracted to a man doing the deep work. (By the way, it is also true the other way: most men are not attracted to women for doing deep work or being a rocket scientist.)

Animals don’t do “deep work” and most animals are attracted for far “shallower” reasons. If everyone has a UBI and AI does the “deep work” then that frees up humans to be like animals in a zoo or in the wild… doing all the displays to attract mates, having babies etc. That’s not nothing — in fact it is 99.999% of everything that happened in history and 100% of what happened in prehistory.

You’re just ceding that 0.0001% which represents technological progress. The actual dread comes from the fact that you and all of humanity will be as completely powerless to change things as cats or any random animal species is powerless now. The difference is you will perhaps understand your powerlessness more than they do. But in the good scenario, you’ll be like animals in a zoo. We are constructing this zoo for ourselves now.

We will remain on this blue planet while the AIs will colonize the universe. In fact I wonder why aliens would ever come to Earth, when they’d just send their AIs or robots. Look at this: https://news.ycombinator.com/item?id=46200502

[dead]

[dead]

For me the most obvious conclusion is, use your maths skills and get into open source AI.

Make AI as cheap and accessible to as many people as possible and we will likely be much better off.

The more people empowered by the tech the less like you will be “redundant” in the way you describe. You will own the important tools only the techno oligarchy own today…

Would you feel the same if you were a part owner of OpenAI? I bet you’d feel excited about the money you’re about to make ?

> How is one to feel safe in the progress of the future when the only hints of the wonderful post-scarcity utopia we can quantify so far is the removal of all human leverage via their labor?

I'm sure the Telegraph operators, human calculators, mine workers and all sorts of people have asked themselves this question. It's obvious that it feels strange to suddenly be part of the cohort asking but I assure you it's not new. And the answer hasn't changed much since.