Exactly. I used to think new tokenizers for domain-specific models was the way (makes intuitive sense, better representations etc), but the added overhead of essentially training from scratch made it much more expensive than a FT.

As far as I can tell, because that is not happening.

The suggestion seems to have come from the claim that to maintain a priority queue, you have to be buying consistently. That might mean buying more than you need to ensure supply when you need it. The assumption that the excess goes into a warehouse seems to have come from someone's fevered imagination.

If there are GPUs built and not yet installed, and people who want to be using them now, money can be made getting the supply to find the demand. Most large scale companies know how to do that sort of thing. If someone has a data center and no GPUs arriving for six months, paying a few million to trade the GPUs that will exist in six months for the ones that exist now gets them an additional six months of operation, while ensuring the original buyer will have GPUs when they need them, plus a few million.

There are reports and rumours that NVIDIA are forcing the data centre companies to buy the GPUs upfront. Though there isn't a lot of clear evidence that this is the case.

[1] https://tech.yahoo.com/gaming/articles/did-nvidia-manufactur...

[2] https://www.reddit.com/r/Games/comments/1xdjbd1/multiple_rep...

[3] https://www.techradar.com/computing/gpu/is-the-gpu-stock-cri...

[4] https://www.businessinsider.com/amazon-nvidia-ai-gpu-data-ce...

I dont understand how this comment connects to the linked research paper. Anyone?

That depends on how long it’s sitting there, given they are two different things to manage it makes sense that you’d either have gpus sitting unused while a dc gets built or a dc sitting empty while you wait for gpus.

They could probably rush them to production, but it's added logistics and would lead to the parts degrading faster.

Lack of customers*. SpaceXaitwitter has some big beautiful idle datacenters with all GPUs ready to use.

Because slowing down compute purchase is collapsing the hype train which is the same as death. Vibes are a wave that must be maintained or crash down onto reality.

I hope Apple Silicon will one day be competitive with GPUs. It dominates much of the CPU market, and it has a GPU, but it's still not even remotely comparable to Nvidia for inference (factor of >5 in some cases, especially in prefill) or even for gaming ("edge computing" in Nvidia terms).

I feel like their GPU architecture is going to need a significant redesign. There are limits to this tile-based mobile architecture.

Maybe if it's worse than next-token-prediction it could still be used as a drafting model, similar to how DFlash works.

I think they think an LLM is just whatever another LLM tells them it is when they ask it to generate a script.

But what about “got taught” instead of “was taught” and the missing comma before “but”?

I wonder whether "10" in "10×" also counts as mathematical notation. Or if just "ten times" is allowed.