I did programming back in the days of MS-DOS. If you've got specific questions, I'm more than happy to follow up in replies.

I thought AI safety was something we had decades to worry about. I wasn't sure it could be solved, but never thought I'd really have to worry about it.

The thing that is frightening is how readily people accept the outputs, and just use them verbatim. Some, if not most, people would trust the output of the latest LLMs over that of actual experts in a field.

We really don't have to worry about a super-clever AI, as the hallucinations of the existing crop of LLMs are far more than enough to random-walk us into disaster.

It would be a far better situation if we had our OS house in order, as it is now, we don't have any widely used capability based secure OSs. I'm slightly hopeful that Genode will kick things off soon. It's been a long time coming.

The biggest danger we face right now is the cost of LLM inference exploding through the roof as the hidden subsidies end after the bubble pops. It's quite likely that a $20 or $200 per month subscription actually costs 5x that or more. If you've used LLMs to generate and maintain a code base that's exploded in size... you're going to have a much harder time after the price hikes hit.