Remix.run Logo
▲ dofm an hour ago

> These are 90-95% cheaper than Anthropic. I'd prefer open source, too, but this has nothing to do with Windows Phone OS (which failed because of network effects).

FWIW it is worth noting that the GP's suggestion is more in line with Microsoft's own stated belief in the future of "unmetered AI".

At the moment, Microsoft is arguably meeting that claim with their support for and encouragement of Nvidia Nemotron Bolt, which is an actually pretty good, quite interesting, unusually-trained-and-sized-to-fit MoE model that was designed to go with the launch of the Nvidia/Microsoft RTX Spark machines.

(Good Sam Witteveen video about it: https://www.youtube.com/watch?v=WJpeW9lCSUM )

But it would be good to see Microsoft themselves distilling an open weights model.

Like any vendor with a profit motive they do have to decide the obvious, though: how do you make money from it, when right now it could cost several hundred million dollars to train (even if you adjust the training of a larger proprietary model so you can distill the smaller one from it)?

There is, essentially, no way to keep an open weights model from being copied/distilled/extracted, and IMO what that tends to suggest is that open weights models will essentially gravitate to sort of uncontroversial, "standard library" tasks.

So I think it also means that Microsoft's approach on open weights will ultimately be the same as Apple's: they will focus their efforts on LLMs that meaningfully add function to their OS and can be decapitated to make similarly useful decision models.

Put another way: given the geopolitical angle of the last few weeks, I think it is possible that the open weights coding models we have now may be among the best we will ever get, until the nearly inevitable GPU/tensor hardware glut arrives.

In the meantime, something else is clearly happening: much broader competition in the "good enough" models category, which is what we need to drive down the cost of inference hardware such that open source and open weights are feasible over the longer term.

▲pianopatrick an hour ago | parent | next [-]

Long term I think the economics point to having a "Linux of AI". I.e. Operating systems were the same way, expensive to make but everyone needed one and did not want to pay for one. So maybe many of the big corporations will eventually collaborate on one ecosystem of open weight models where they share the costs of training it. Then they can compete on other things like serving it or whatever.

▲alala-lala-lon an hour ago | parent | prev [-]

[dead]