| ▲ | kypro 5 hours ago | |
Not sure what you're defining as an "older programer", but I've been coding for 20 years at this point and have been an AI doomer since around 2010, and hardcore AI doomer since 2015-2016ish... I often wonder why I am so confident in my prediction of AI doom despite other intelligent people not being as convinced, and honestly I think it's just that I've been working with AI and building neural networks at a low level for well over decade. I'm far from an expert when it comes to the frontier, but I feel like I have had to developed a strong intuition for these systems because it used to be fucking hard to get AI to do anything useful, but what you find is that the bottlenecks are literally always the same – compute, data and time Algorithms matter, but mostly just because of efficiency. For example, a convolutional neural network can very efficiently be trained to solve computer vision problems, but there's nothing stopping you using a much worse network architecture then throwing a ton more compute, data and time at the problem and getting the same result. Algorithmic improvements are really just what we've needed to get around current compute, data and time bottlenecks. AIs like Deep Blue required a lot of algorithmic thinking back in the day to beat Kasparov in chess, but now we're so much less compute or data constrained that you could beat Kasparov without needing to put even a fraction of the algorithmic effort in if you just throw enough compute and data at it. The reality is, AI doesn't fail. They may have appeared stupid to users for a long time but that wasn't because they didn't work, but because we were compute, data or time bottlenecked. As soon as you assume compute and data will continue to grow, then it's really just a matter of time before all problems that are solvable with enough intelligence are solvable. Add significant algorithmic improvements every ~5 years to that equation and suddenly god-like AI seems like it's probably reasonably close. The only question to ask then is what happens after you've created all these super-human AIs. And here I'm much less certain, but I see so many low probability risks in an ASI world that I'd bet everything on one of those bad things happening relatively soon after such AIs exist. | ||
| ▲ | kooi 5 hours ago | parent [-] | |
I suppose the hinge point is if algorithmic improvement + compute capacity is or exponential improvement. I flip/flop between the options as I listen to the Moonshot Podcast bros (lols because of obvious over-hype) and Lex Friedman's state of 2026 AI with Nathan Lambert and Sebastian Raschka. If exponential, then yes I see more variance humanity's short/medium term outcome. More chance of doom, more chance of boom. If logarithmic, well then improvements continue, but they will be incremental to what we have now and we will continue into some form of widespread agent adoption. Similar to wireless communication technology is prolific, but now taken 100% for granted. Personally, even with Sol, the results I get from "stress test" one-shot prompts are abysmal. Hallucination of solutions to non-existent bugs as popped up multiple times. So AGI take over, meh, low chance IMO. I think one real risk is solo or sleeper cell type nefarious actors having a semi-competent engineering consultant which can enable small-scale weapons deployment. Overall, I think the most probable short term risk is AI companions, hyper individualization and deterioration of social institutions. Much less flashy, but much more likely. I mean look at the effects of social media on mental health. And that was only created 1 generation ago. | ||