|
| ▲ | altcognito 13 hours ago | parent | next [-] |
| > compute/manufacturing capacity for current nodes which doesn’t age nearly as well I mean, compute depreciates, but I think there is zero chance that the value of inference or training is going to fall to zero. Market discovery will find the right price provided the market has the right degree of freedom. Given the type of market it is, I don't see how that won't be the case. |
| |
| ▲ | jaggederest 13 hours ago | parent | next [-] | | I'm a big fan stylistically of what https://taalas.com/ is doing, as far as models baked into silicon. If you haven't tried their chat it's absurdly fast (and also very very dumb) That implies to me that in the future we'll have models as good or perhaps better than the state of the art at the moment, but on hardware chips that can be put in places where you can't currently locate a datacenter, and operating at hundreds of times better power efficiency, which sounds pretty great. | |
| ▲ | lambdaone 13 hours ago | parent | prev | next [-] | | Algorthmic improvements in inference could make all that kit redundant very quickly - there are already moderately capable models that can be run on phones or laptops with specifications that are currently high-end but will be mainstream in another year or so. This will lead to a superabundance of power-hungry compute power in the hyperscalers, and it's not entirely clear what can be done to consume it all and still run at a profit unless they manage to make ever greater gains for ever more compute-hungry models that cannot be run on consumer devices, unless they refresh their hardware at ever faster and more expensive rates. The joke about data centers used to be that their core business was selling power at a loss; this may end up being true of the hyperscalers next. | |
| ▲ | Retric 11 hours ago | parent | prev [-] | | That hardware costs GW of electricity at scale. So barring major disruption in R&D you hit a cost curve cliff where new hardware is simply more cost effective even if existing hardware is free. Some workloads may make sense running for a few hours a day during cheap solar prices on outdated hardware, but in less than a decade the value is very much hitting zero. |
|
|
| ▲ | zer00eyz 13 hours ago | parent | prev [-] |
| > but most of the AI investment is in compute Some people thought that it was misguided when they extended the depreciation cycle of the current AI build out year(s). In terms of raw performance, there is still some headroom (maybe) but those gains are going to be marginal when you look at the amount of compute per watt (if its more than 5 percent I will be shocked). And that push is going to create a whole other set of problems (cooling is going to be an issue, it already is). It is fairly likely that this hardware buildout has more legs than one might suspect based on history. |
| |
| ▲ | Retric 11 hours ago | parent [-] | | 20 year old fiber can have another 10-20 years of useful lifespan remaining, I think compute is going to be valuable for a while but even a 7 year cycle doesn’t change much. Most compute isn’t going to be new at a potential crash and recovery takes time. |
|
