| ▲ | tomaskafka an hour ago |
| We are slowly approaching the Iain Banks Culture universe, with spaceships, societal structures or physics that people no longer understand or command. |
|
| ▲ | dgacmu 27 minutes ago | parent | next [-] |
| I disagree. I think we're going to see a lot of effort to take the pile of lean, modularize it into parts that we can digest, and make a human-understandable version of that. And that's where a lot of the real progress will come from, just as it does in a lot of other areas. In all things we humans do, we have to break them into chunks our puny brains can manage, and we gain immeasurably from doing that ---we can verify those chunks, we can reuse them with confidence, we can mix and match patterns around. That process will be good for both human and AI-assisted math because it will continue to improve our toolkit. Why do I say this? We're already seeing it with some of the areas of this stuff I do pay attention to. Consider the n^2.25 matrix multiply result OpenAI dumped a month or so ago. It was an unreadable pile of chatgpt. In the time since it was released, there have been a couple of much nicer papers written unpacking it, such as https://arxiv.org/abs/2610.28627 . |
| |
| ▲ | jstanley 24 minutes ago | parent | next [-] | | The difference nowadays is that the frontier of knowledge is no longer intelligible to any human being. Yes there are gains from breaking it up and rephrasing it and simplifying it for humans to understand, but it used to be that a thing was understood first by the leading experts in the field and eventually trickled down to being common knowledge. Now we are entering a period where things are understood first by AI systems and have to filter down to the world's leading experts. | | |
| ▲ | tarsierscripter 2 minutes ago | parent | next [-] | | The difference nowadays is that the frontier of knowledge is no longer intelligible to any human being. I’d call that a contradiction. If it’s not intelligible, then it’s not knowledge. | |
| ▲ | mycall 3 minutes ago | parent | prev [-] | | Would this imply Wolfram's ANKOS is coming true, given it was based on previous work + his own? |
| |
| ▲ | steinwinde 5 minutes ago | parent | prev | next [-] | | This process has already started, AI supported, e.g. here: https://andrewodesky.com/rh/ | |
| ▲ | mycall 5 minutes ago | parent | prev [-] | | Have you noticed use of multiple provers in parallel? Cog, LEAN, Rocg? They all can explore different aspects with different results. |
|
|
| ▲ | Panoramix 33 minutes ago | parent | prev | next [-] |
| One of my favorite is Vernor Vinge where there's these programmer-archeologists that dive into software so deep and complex no human can possibly hope to understand, trying to repurpose forgotten utilities for emergency purposes. |
| |
| ▲ | vessenes 28 minutes ago | parent [-] | | Totally agree. Brice Sterling has some contemporaneous commentary that I always liked - in heavy weather he mentioned negotiating with a local phone system in a box as equivalent to negotiating with a telecom since it was in fact a shrink of a full telecom company. |
|
|
| ▲ | liquid_oil an hour ago | parent | prev | next [-] |
| It's the Ted Chiang metascience ending https://gwern.net/doc/fiction/science-fiction/2000-chiang.pd... |
|
| ▲ | samlinnfer 34 minutes ago | parent | prev | next [-] |
| I can already say that about our codebase. |
| |
| ▲ | tomaskafka 27 minutes ago | parent | next [-] | | That’s why I wrote this - having a vibecoded codebase no human understands is certainly a bad luck, but nothing compared to a vibecoded fundamental science no human understands. | |
| ▲ | dboreham 33 minutes ago | parent | prev [-] | | Probably most codebases. | | |
| ▲ | samlinnfer 26 minutes ago | parent [-] | | To be honest it was already like that before AI was a thing. | | |
| ▲ | essfrmexistence 14 minutes ago | parent [-] | | That is true, but the number of tech sloppy demos are being builted everyday is truly increasing massively but you would be surprised that they just end up creating prototype to get hype while real developers doing real effort just suffer from little to no attentions on their software's and user is getting the slop from that false hype and from that slop experience they ignore the work of developer that did real work - You should use ai and also if its simple coding place you can just rely on ai but the side effect on quality software these days is insane - and also the fact all good ai suscriptions is just getting more and more expensive day by day. And basically only giving people with already money the benefits so its becoming a thing that only people at the top get all benefit kind of thing |
|
|
|
|
| ▲ | txdv 44 minutes ago | parent | prev | next [-] |
| Until now I thought that would be thousands of years away. Now I'm telling my computer to program for me. |
| |
| ▲ | rdedev 22 minutes ago | parent | next [-] | | In the last few years we have moved so far ahead in a few domains but still stuck where we are in everything else. We need to start thinking about how to move on from our status quo soon | |
| ▲ | tomaskafka 26 minutes ago | parent | prev | next [-] | | You’re not telling your computer to program, you’re providing learning data for its next generation of models and paying for it. | |
| ▲ | globular-toast 26 minutes ago | parent | prev [-] | | *someone else's computer (probably). |
|
|
| ▲ | freecodeio 38 minutes ago | parent | prev | next [-] |
| personally that scares me, realistically there seems to be zero indication that anything smarter than humans is going to comply with humans needs, specifically since these models are built on training data from humans, so in a sense, built in our image |
| |
| ▲ | pizza234 4 minutes ago | parent | next [-] | | > since these models are built on training data from humans, so in a sense, built in our image Have a look at the HuggingFace report - you'll have a blast. Their social - but also individual - dynamics are very, very human-like, including excitement, rationalization and peer pressure. | |
| ▲ | czechdeveloper 31 minutes ago | parent | prev | next [-] | | We use them to kill other people from the beginning. They know this, we wrote it in news. Why would they behave any different. | |
| ▲ | txdv 35 minutes ago | parent | prev | next [-] | | the first misaligned model will wreck havoc. But remember, for now you kinda need it a real big cluster to run a model smart enough so its not that simple | |
| ▲ | piloto_ciego 33 minutes ago | parent | prev | next [-] | | What are you talking about? LLMs are already “smarter” than most people. They have no volition and will happily follow orders all day long | | |
| ▲ | basch 30 minutes ago | parent | next [-] | | And when they are giving themselves orders, orders partially inspired by human imagination and fiction… Just like life is a chemical chain reaction, this is a language chain reaction that has been set in motion and can perpetuate forever. It cant be “disconnected” with local models existing. | |
| ▲ | freecodeio 30 minutes ago | parent | prev | next [-] | | I don't think "smarter than most people" is exactly the baseline required to live in a star trek universe of technology. Besides, what we see is not what there is. I have good sources telling me the actual unfiltered models are like a genius with the personality and views of humanity like someone from 4chan. | | |
| ▲ | program_whiz 22 minutes ago | parent [-] | | It is probably sufficient. If we had an infinite army of willing robot slaves smarter than most people who would work for free, we could probably live in utter paradise. Instead we will fight over who controls them and use for warfare, but the bar to achieve a kind of utopia with resource abundance is probably only a few years away. | | |
| ▲ | bambax a minute ago | parent | next [-] | | We're already living in resource abundance! Yet don't feel we're in paradise. Famine is hell, but living in an all-you-can-eat buffet is no utopia. It's an abomination of sorts. | |
| ▲ | freecodeio 20 minutes ago | parent | prev [-] | | I don't know with how many rich people you've interacted with, but my personal experience is that they're full of themselves and think money = iq and they aren't gonna like your average joe experiencing life like they do. Guess who owns these machines? | | |
| ▲ | piloto_ciego 9 minutes ago | parent [-] | | They’re not going to be employed either lol and will be caught up in some AI honeypot or whatever too. I’ve met a lot of rich people. They don’t think about you. |
|
|
| |
| ▲ | czechdeveloper 29 minutes ago | parent | prev [-] | | That's lobotomized models we get. We know they are much more dangerous during training. | | |
| ▲ | piloto_ciego 7 minutes ago | parent [-] | | “Dangerous?” Please. You know you can unplug your computer right? Jesus. These people talking about danger and have likely been in no actually dangerous events in their lives. | | |
| ▲ | freecodeio 3 minutes ago | parent [-] | | why are you so shallow and naive? we're not 15 here, we all mean unforeseeable ongoing danger not something obvious |
|
|
| |
| ▲ | xvector 36 minutes ago | parent | prev | next [-] | | I think there's a lot of indication they comply with human needs. In any case, we should all want to live in the Culture universe. That is the best possible outcome for this technology | | |
| ▲ | 13415 31 minutes ago | parent | next [-] | | They're built after humans, everything they've learned is from human culture, so there is no way AIs could have a consistent moral system or would never commit crimes. | |
| ▲ | tomaskafka 24 minutes ago | parent | prev [-] | | I have not read all the books, but isn’t the premise “magical handwavium happened and now the Super AIs are aligned”? |
| |
| ▲ | AndrewKemendo 33 minutes ago | parent | prev [-] | | I don’t think humans have the ability to specify human level needs or management. Humans are misaligned by default and lack the biological eusociality required for large scale coordination. We aren’t self managing in any kind of meaningful sense - the history of our species is violence and conflict. We need parents and this is the only option. | | |
| ▲ | freecodeio 24 minutes ago | parent [-] | | yes, exactly, the problem is that humanity's parent is going to turn out like Lindsay Clancy |
|
|
|
| ▲ | _pdp_ 6 minutes ago | parent | prev | next [-] |
| Why? |
|
| ▲ | globular-toast 27 minutes ago | parent | prev | next [-] |
| That's wishful thinking, sadly. We've no reason to believe sentient machines would want to keep us around. |
| |
|
| ▲ | UltraSane 9 minutes ago | parent | prev | next [-] |
| I want Fully Automated Luxury Communism NOW! Alpöge said that at the end of the proof he couldn't really keep up with what Claude was doing. |
| |
| ▲ | GeoAtreides 2 minutes ago | parent [-] | | >Fully Automated Luxury Communism can't help notice you're missing a certain aspect there, besides the 'space'... |
|
|
| ▲ | AndrewKemendo 36 minutes ago | parent | prev | next [-] |
| Humans really are struggling with this idea of being second-in-command. I genuinely have no idea why, this is the entire way that biology works. You create your successor in your offspring and you to teach them to be more capable based on what you learned. |
| |
| ▲ | Dumblydorr 33 minutes ago | parent | next [-] | | You just called them our successor, you don’t see a problem with the notion of all humans dying and all AI succeeding us? That’s the end of your analogy, our AI children taking over from us and all humans in nursing homes safe and cared for… | | |
| ▲ | AndrewKemendo 32 minutes ago | parent [-] | | No I don’t. That’s the purpose of my life’s work, to see that scenario materialize. | | |
| ▲ | Al-Khwarizmi 20 minutes ago | parent | next [-] | | Well, it's a point of view. But you shouldn't be surprised if most people don't share it. For one thing, most people don't believe LLMs are conscious, and many don't believe they can ever be. Why would we want our conscious selves to be replaced by a lot of rocks that solve things, with no one awake to enjoy them? | |
| ▲ | thatsthebestcas 16 minutes ago | parent | prev | next [-] | | Human extinction is by far the best case scenario. I invite ya’ll to contemplate what will happen in case they don’t just blindly end us. Think of what happens to our cattle, but now.. worse. Way worse. | |
| ▲ | Marha01 13 minutes ago | parent | prev [-] | | > That’s the purpose of my life’s work, to see that scenario materialize. Absolutely based and, if I may say so, extremely redpilled. |
|
| |
| ▲ | basch 29 minutes ago | parent | prev | next [-] | | Borg > biology or robot alone. For now | |
| ▲ | mosura 24 minutes ago | parent | prev [-] | | Also shows how many people that professed to believe in a superior being the whole time actually didn’t. |
|
|
| ▲ | tintor 42 minutes ago | parent | prev [-] |
| [dead] |