"No other human activity poses this level of danger." I know this is a bit over the top but let us look at some of the most pressing issues in the world today: global warming, nuclear weapons, wealth inequality, war, technology dependence. Wouldn't a much more capable LLM in the wrong hands make these accelerate faster in the wrong direction ? Many of us , me included, have this mental model of a rogue Terminator-like AI but what I am most worried about is these LLMs in the hands of people. Look at what we have done to this Planet with the tools that we have so far, we have repeatedly tried to subjugate , enslave and kill each other because of things as peety as skin colour , tribe , religion and who owns which patch of land .

I don't have faith in the human race as is to do the right thing when handed these tools , you can already see the nonsense like the fake nudes , fake news and all that other filth being pushed on social media by people with access to relatively daft models. What happens when we can package a Mythos 5 level model in a box , when any war lord , supremacist or religous zealot can access these ? , You now you have your personal bio-chemist and nuclear physicist in a box.

The tool in itself is not what I am worried about , it is the human in the loop. I don't have an answer to this but i believe it is something we should all take a moment to think about , just think what a different world we would be living if any random person could buy a nuclear weapon off the shelf ? There are extremes to both ends , you can either worry way too much or don't care at all, I believe the best place is the middle ground were we actively think this through instead of using our usually "Move fast and break things" mode..