Remix.run Logo
▲ elorant a day ago

Wonder how many mathematicians could reach similar breakthroughs if they had near infinite compute.

▲henry2023 a day ago | parent | next [-]

The first working transistor was invented 79 years ago, I bet we’ll consider today’s total compute trivial in another 80 years.

▲smithmayowa 14 hours ago | parent | next [-]

Funny enough this put things into perspective for me, it's like when breakthroughs happened after the first transistor based computers, yet for decades after those, we had nothing and this leads me to believe, that we might settle into problems that are still quite hard for even llms, and not everyone can even produce stuffs like this even if they had the same compute that alpoge had.

▲lossolo2 a day ago | parent | prev | next [-]

Aren't we approaching the limits of physics, with increasingly diminishing returns each generation? I remember reading an article about quantum effects at these scales, which make everything much harder. I mean, for example, even if we double the speed at which we can travel every year, you will never get to Alpha Centauri in your lifetime

▲the8472 a day ago | parent | next [-]

Chips are only 2D, and we're far from the landauer limit, and the landauer limit only applies to non-reversible computations anyway. If we could assemble matter atom by atom at scale we could get to some dirty diamond that fits in your hand, outclasses the world's top supercomputers and runs off 10W of sunlight.

▲thombat a day ago | parent | prev | next [-]

The overall point is sound, but in fact if we could double our speed every year then in just 23 years we'd get from 100km/h to the speed of light

▲ignoramous a day ago | parent [-]

Keeping with the Interstellar framing, the key is not only in faster ships (lightspeed) but in shrinking distance (warping space-time). For LLMs, that'd mean efficient model architectures on efficient chips.

▲GoblinSlayer 8 hours ago | parent | prev | next [-]

That's the Electron factor. Unix worked on PDP-7 with 18kb of RAM.

▲dist-epoch a day ago | parent | prev [-]

We are approaching the limits in a single chip. But we can add more chips, millions of them, billions of them. Gigawatts of compute.

▲twoodfin a day ago | parent [-]

Right. I’m sure at some point in the past more capital was spent on designing better airplanes than on building them & the infrastructure to support them.

We still spend money on designing better airplanes, but the two lines on that graph crossed long ago.

▲RAWGOD a day ago | parent | prev [-]

>I bet we’ll consider today’s total compute trivial in another 80 years.

Past performance is not indicative of future results. Airliners are still subsonic after a 100 years of aviation.

▲betterunix2 a day ago | parent | next [-]

> Airliners are still subsonic after a 100 years of aviation

That is a deliberate decision motivated primarily by economics and secondarily by environmental concerns. In fact airliners today are somewhat slower than they were a few decades ago because the lower speeds are more fuel efficient. We certainly could build fleets of supersonic airliners if it made sense to do so, it is not as if we have no idea how to do it and in fact we built supersonic passenger jets once upon a time.

▲recursive a day ago | parent [-]

Yes and? we can make deliberate decisions motivated by economics regarding computation as well.

▲Maxatar a day ago | parent | prev | next [-]

Your example contradicts your point. Airliners are still subsonic after an extended period of time, and hence based on that past performance it's a good indicator (although not definitive) that they will remain subsonic for an extended period into the future.

In general, past performance is actually indicative of future results. It's not definitive, and some phenomenon are random, but the trajectory of technology over time is not some random fluke that just has coincidentally trended by random chance.

▲RAWGOD a day ago | parent [-]

>Your example contradicts your point. Airliners are still subsonic after an extended period of time, and hence based on that past performance it's a good indicator (although not definitive) that they will remain subsonic for an extended period into the future.

There was a period in aviation development when it was widely assumed that speed will continue to rise. Just like in the development of CPUs there was such a period, quite recently.

▲ a day ago | parent [-]
[deleted]
▲szvsw a day ago | parent | prev | next [-]

But along other axes they have also improved tremendously, no? Ie rather than just time to destination, they have greatly improved efficiency (energy spent per unit of work) and throughput/parallelism (ie simultaneous flights in the air) right?

▲RAWGOD a day ago | parent | next [-]

Yes, but these axes are also not showing an unlimited growth. There is no expectation that either speed, fuel efficiency, ticket price, or total number of flights will improve by an order of magnitude in the foreseeable future, unless there's a technological breakthrough (Nuclear turbines? Artificial photosynthesis making kerosene free? 3D printing the entire plane? These are currently all sci-fi ideas).

▲cyanydeez a day ago | parent | prev [-]

but it's still just people in a tube.

Sure you're going to get a lot of math problems solved, but food distribution is still going to be stuck by warlords and oligarchies.

Think of it like highway designers and the "one more lane" gridlock delusion. In this case, it's more will smith eating speghetti than cheap, low cost cancer cures.

▲jcelerier 19 hours ago | parent | prev | next [-]

a single company was able to send more satellites to space in the last 4-5 years than all the things sent combined, across all countries since we started putting stuff into space in 1957. Saying "Airliners are still subsonic" is like saying that cars still are still not going much past 120/130 km/h thus we're technologically stuck, in a world where jet engines exist.

▲krferriter a day ago | parent | prev | next [-]

Passenger airliner speed is not the only scalable metric relevant to aviation.

▲vibebingo 16 hours ago | parent | prev | next [-]

Chips and airframes are different engineering challenges.

▲bartread a day ago | parent | prev | next [-]

> Airliners are still subsonic after a 100 years of aviation.

Yeah, but that's a choice made for very good reasons:

- Noise reduction

- Fuel efficiency

- Reduced pollution and lower environment impact

- Reduced cost of materials, equipment, and maintenance

- Passenger affordability

It's not that we can't make supersonic airliners or that the fact we don’t illustrates a lack of progress or performance: it's that it doesn't make economic or environmental sense to.

Gulfstream and Sukhoi even had a partnership to develop a supersonic business jet (which, if you look at the mockups, might be one of the coolest looking planes ever conceived) but they gave up after spending a fairly enormous amount of money already because it was going to cost an absolute fortune to bring to market... and there was no real market for it.

EDIT: the responses I’m getting are a bit weird. People are putting words in my mouth but all I was actually doing was pointing out the the example used wasn’t a great illustration of the point the person I was responding to was making. This isn’t Reddit so maybe we could reign it in a bit, please?

▲rafaelmn a day ago | parent | next [-]

That's kinda the point I think ? It's not like we can't turn half of global energy production or GDP to compute - it's more about does it make sense to. As much as I'm getting value out of this technology daily - the level of investment made to get this far is astronomical compared to the value I'm seeing created, even less captured by these companies. And because the progress is so good in some domains it's easy to believe that these things are magically reaching some intelligence generalization threshold. But then you leave the training distribution and you get disturbed by how poor the performance is and how obviously equivalent things don't automatically transfer at all. Given the recent market trends I would not be surprised we see a correction in level of compute investment going forward.

▲FloorEgg a day ago | parent | next [-]

The universe seems to be set up such that it wants entropy to increase. Any living thing that doesn't profit from converting low entropy energy to high entropy energy dies.

So the question is whether more intelligence will lead to faster increase in entropy. My guess is it will. Whatever recent market trends are, they are irrelevant. Whether the AI can or can't do x thing is irrelevant. What matters is whether the companies that make AI profit from making AI, and if you look closely at what's going on it sure seems like they do. If you think otherwise it may be because you're reading headlines and propaganda articles and not digging into the fundamentals.

So maybe we have a temporary correction/slow down, because these companies took on too much leverage and grew artificially too fast. That's very possible. In the big picture long term it will be a blip. The universe seems biased to increasing the volume of intelligence and I have never heard a compelling case for why this wouldn't continue for a long long time.

▲pixl97 a day ago | parent | prev [-]

>It's not like we can't turn half of global energy production or GDP to compute

That depends which part of 'we' is running the economy. Just imagine an economy that makes mass massive amounts of money but doesn't align with human needs.

▲bayindirh a day ago | parent | prev [-]

I can argue all of the reasons are valid for keeping compute at a certain speed range, or power consumption envelope.

It's not we can't make compute much more denser, there's a new standard informally called as megawatt racks. But deploying them at gigantic scales bring in massive problems as well, so we can keep total energy envelope at a fixed range.

...for the sake of the reasons you outlined.

▲ThrowawayTestr a day ago | parent | prev [-]

We can build supersonic passenger planes, no one wants them.

▲twoodfin a day ago | parent | next [-]

We could build gallium-arsenide chips, same problem.

Instead we just build more of the more economical thing.

▲kibwen a day ago | parent | prev [-]

Rather, nobody wants them at the price point necessary to make them economically viable. In the context of tech, the cost to develop a next-gen fab doubles every four years; absent some major breakthrough, we have about 20 years before even the most advanced economies balk at spending $1T, $2T, $4T, $8T on a single fab, causing hardware to settle down like airline speeds have.

▲fantasizr a day ago | parent | prev | next [-]

the best computing power used to be located at the universities and national laboratories. Now you need access - employee or otherwise - to a nasdaq listed startup. Hard not to think it's the shareholders who benefit in the short term in this new alignment.

▲contubernio a day ago | parent | prev | next [-]

I could. I mean that. But I work in Spain and have a run if the mill desktop computer and pay for ai out of pocket.

▲solyin 21 hours ago | parent [-]

You don't have any contact info in your profile.

▲hyperhello a day ago | parent | prev | next [-]

Maybe we should ask how many compute installations could reach similar breakthroughs if they had a mathematician.

▲ImHereToVote a day ago | parent | prev [-]

[flagged]