I'm not sure how well this take will age.

Its certainly true at the moment, but give it 10 years and we might have systems that are much cheaper and much better at context management than they are now.

(Apologies to anyone who is under the impression that we were very likely going to be at the singularity in 10 years time. Possible != very likely)

Sure, but it’s equally likely that we hit a point where scaling becomes economically unviable because we can’t come up with enough algorithmic improvements to break free of the tyranny of log linear scaling. (I’m not sure how many 2x in token cost people would be willing to pay)