From the point of view of the old people, exponentially large injections of improbability.

This is also where it is important to understand the argument is parameterized. If your acceptance function is "a full human body living a full human life in the way I'm used to", then you're asking for a huge whackload of improbability to keep that body alive. If, on the other hand, you merely ask for "the minimum system that stays conscious", while that is still a huge whackload it is exponentially smaller, by a very substantial factor.

Anyone who takes any sort of comfort in the argument doesn't understand it. It preserves minimal consciousness, for any and all definitions of "minimal consciousness". It does also preserve bodies, but at an exponentially smaller probability. You can even use as your acceptance function "civilization exactly as we know it today, with everyone currently living still alive in 1000 years" and that comes at a yet smaller exponential probability. All of these "exponential probabilities" as separated by numbers of zeros that themselves require exponential notation to write. But the universe itself is making so many splits and branches all the time that it has the "budget" for these probabilities to be true, somewhere in the quantum multiverse.

Or, put another way, any event that you can draw a non-zero probability pathway for occurs. It is not impossible for me to spontaneously have my particular genetic defect reversed in every cell in my body, so there are branches where that happens... they're just exponentially unlikely compared to what we actually experience. It can literally be in %10^(-10^1,000,000,000) of the universes... but if it's not zero, it will happen, if the quantum multiverse is the correct view.

This is a grevious misuse of Russell's Teapot. You realise your assertion, if followed to its logical conclusion, is that because civilization has existed for this long, it can never collapse no matter what.

> In that quote, the author is telling you that all of your past predictions of teapots have been wrong

Past predictions have zero influence on future ones. If you flip a coin 10 times and get heads every time, the probability of your next coin flip coming up heads is 50%, not 100%, because it is independent of any prior trials.

Moreover, if Alice and Bob each make 100 predictions and are 100% wrong, but Charles makes 10 predictions and is 100% correct, Alice and Bob's predictions have zero bearing on whether you should trust Charles' predictions. They are not even a factor to be considered.

I don't think I even disagree with you in the broad picture. I am not concerned about language models. But you have committed three blatant logical fallacies in a row. You literally could not make a worse case for your position if you tried, I suspect.

“History is a guide to apocalyptic predictions in only one way: all previous apocalyptic predictions have been wrong”

“History is a guide to state abuses of power in many ways, and most of them point in the same direction, that it is a very bad idea to invest powers like these with the state”

These are two separate arguments. I am sympathetic to the latter and endorse it essentially as written, which is why I didn’t criticise it. My criticism of the former is not a metonymic dismissal of every other argument in the piece.

Middle?

We're just starting. The air conditioning technology didn't hit its wall of fire yet. What we're going to do when that tech will cease to cool us?

Every AC system (from refrigerators to air conditioners) has an upper limit, which is around 41 degrees Celsius for T1 class.

Mongols did it on horseback. It's mostly a matter of will/capability and no scruples.