I sometimes wonder what freedom really is. Should the freedom to harm others also be guaranteed as freedom? Individual computational sovereignty versus the prevention of social harm is always contentious, and claiming that something is always right is always extremist.

If it is "intelligence," is it not natural to reject that? I also think local models should adapt to me when it comes to safety issues, but people bring up examples that are too extreme.

Programming is the same, and in fact, most problems are boundary problems. It is the things that straddle the boundaries that always make us think. The principles at those moments change every time, shifting with the situation and context. Is that not just a childish way of thinking? Even in programming, just the issue of granting root permissions is enough to cause endless fights.

I agree with the idea early in the text that intelligence is not everything. Intelligence includes bodily intelligence as well, and we lump it all together into one thing, but there is so much of it. The variance in intelligence is vast, and those people also need to be able to live their lives. That is why I think intelligence alone will not solve everything. I too believe that the human species may disappear and an inorganic species could emerge later, but I find it hard to understand why people talk about such extreme risks. And it is not true that making a chip in a semiconductor fab involves almost no human intervention. If you have experience supplying equipment to such fabs, you would know there are quite a few points where humans are involved. Though sure, they could be replaced.

In my view, society is simply worshipping the abstract concept of "intelligence" and projecting its desires onto it. The AGI narrative is just a kind of cargo cult, a projection of capital by the tech elite. Software eating the world, superintelligence solving everything. The masses engage in messianic projection, and tech companies, facing declining growth engines in their own businesses, are trying to create new ventures to pour it all into. A market that is large enough becomes too big to sustain massive growth rates every time, and when growth rates are that high, the larger the company, the more its sector's growth rate tends to converge with its own. This is usually called the law of large numbers. The problem is that CEOs and these entrepreneurs always want growth rates above a certain threshold, so they are simply searching for new pastures. AGI is just being pumped up out of financial necessity.

Capital will create gravity and bring forth new technologies. That is the allure of capital, after all. But that does not mean all problems will be solved, and inequality will deepen. Only the distribution of power will shift.

> Should the freedom to harm others also be guaranteed as freedom?

everyone should have the ability to harm others, with the understanding that if they actually use it for evil there will be consequences. as in "i can guarantee freedom of speech, but not freedom after speech."

its the same principle as guns in america. you can own a gun and shoot it as much as you want but if you (try to) kill someone you go to prison unless you prove it was self defense.

the difference is with a physical weapon you can do things like registration or red flag laws to reduce the risk, but a llm is made of information that can be copied without anyone knowing. that means any laws to regulate ai at the level of models are unenforceable without totalitarian control over all computer hardware. that would kill free speech and real criminals would still find a way.

i think the right answer is to give up control over model capabilities and regulate uses. leave individual people and communities out of it. create an absolute safe harbor for open source and self hosting.

as you said, we need to focus on the ways ai concentrates power in the hands of those who already have a lot of it - big business and the state. that is where the biggest, and maybe the only, real harm comes from.