My personal belief is that the most effective leaders do in fact deeply understand the technical side of their business, yes. To be fair, I am biased, because I was previously a nuclear reactor operator on a USN fast attack submarine, and the Commanding Officer is absolutely expected to fully understand technical issues. I think that a lot of organizational and technical problems would be solved if companies took several notes from the Nuclear Navy’s playbook. FWIW, Google’s SRE program was heavily influenced by it.

I of course don’t have a crystal ball, and can’t say with certainty that AI will never replace human intelligence and judgment. However, I don’t see a viable path forward where that occurs, and we all still have jobs. Why would I be kept around if a superhuman AI can do my job 1000x better and faster, and at a drastically cheaper price? Or, if I was kept around as a human QA agent, why would I be paid nearly what I am now?

I don’t see humans allowing AI to replace them. UBI is a pipe dream that would be destroyed by rent-seeking behavior anyway, and there’s no way billionaires would go for an AI tax, so we’ll be at an impasse.

▲

kypro 13 hours ago | parent [-]

> However, I don’t see a viable path forward where that occurs, and we all still have jobs. Why would I be kept around if a superhuman AI can do my job 1000x better and faster, and at a drastically cheaper price? Or, if I was kept around as a human QA agent, why would I be paid nearly what I am now?

Well as an AI doomer I agree with this. In my view you will lose your job, along with everyone else, and we'll just need to hope that those in power will give us some cash to feed our families. Ultimately us regular folks won't have the collective bargaining power of our labour and could not revolt with nation-wide strikes, nor are we any physical threat for those with power anymore.

But that's the good outcome. This assumes AI remains obedient to it's human owners and that bad actors don't have access and can't use it to create biological weapons and other civilisation-level threats.

I know we're not there yet, but the history of AI is one of continuous step changes in capabilities with each algorithmic breakthrough. AIs today are far more knowledgable and much quicker than humans, but still missing a few bits that keep them from being broadly super human. One or two algorithmic breakthroughs is all it takes though. It was just two breakthroughs over the last 8 years that took us from not even having code-autocomplete to having reasoning models advanced enough to solve math problems which have stumped humans for decades.

▲

sgarland 12 hours ago | parent [-]

> AIs today are far more knowledgable and much quicker than humans, but still missing a few bits that keep them from being broadly super human.

Generally, yes, but it’s the non-determinism that gets me. Even easy stuff: I’m currently on a road trip, and asked Claude (Fable) to plan various routes, optimizing for scenery while also ensuring a maximum drive time each day, and ending every night in a town that has a decent hotel. Along the way, I’ve made various tweaks, like “we’d like to drive an extra hour or two today, so adjust the rest of the route so we don’t end up with a short intermediate day.” While in general it’s done quite well, it occasionally made errors on distance between stops, something that’s trivially queryable. It wasn’t off by a few miles, either, it was hundreds.

Until they can get the confidently incorrect problem solved, I don’t see it being a viable replacement. Maybe if you have agents that are aggressively checking the main ones’ output, but that seems unnecessarily wasteful of resources to me.

	▲	kypro 8 hours ago \| parent [-]
		> Until they can get the confidently incorrect problem solved, I don’t see it being a viable replacement. Maybe if you have agents that are aggressively checking the main ones’ output, but that seems unnecessarily wasteful of resources to me. 2 years ago AI could even count the number of letters in a word or do basic maths. Now we're at the point where we can't trust it because makes occasional errors. Don't get me wrong, I agree with you. These are things that need to be solved first, I just don't have reason to believe these are particularly difficult problems. Anyway, I enjoyed the convo. Was interesting hearing your thoughts. I would give anything to be wrong.