Then why are they spending $20 billion dollars to handicap an inference company that giving open source models a major advantage over closed source models?

▲ gpapilion 4 hours ago | parent | next [-]

Realistically groq is a great solution but has near impossible requirements for deployment. Just look at how many adapters you need to meet the memory requirements of a small llm. SRAM is fast but small.

I would guess their interconnect technology is what NVIDIA wants. You need something like 75 adapters for an 8b parameter model they had some really interesting tech to make the accelerator to accelerator communication work and scale. They were able to do that well before nvl 72 and they scale to hundreds of adapters since large models require more adapters still.

We will know in a few months.

▲ nl 4 hours ago | parent | prev | next [-]

> handicap

Your words.

Because it's very good tech for inference?

It doesn't even do training.

And most inference providers for Open Source models use NVIDIA eg Fireworks, Basten, TogetherAI etc.

Most NVIDIA sales go to training clusters. That is changing but it'd be an interesting strategy to differentiate the training and inference lines.

▲ credit_guy 8 hours ago | parent | prev | next [-]

> to handicap an inference company

That's a non-charitable interpretation of what happened. The are not "spending $20 billion to handicap Groq". They are handing Groq $20 billion to do whatever they want with it. Groq can take this money and build more chips, do more R&D, hire more people. $20 billion is truly a lot of money. It's quite hard to "handicap" someone by giving them $20 billion.

▲ wmf 8 hours ago | parent [-]

Groq doesn't have any employees. They can't do R&D because there's no one to do it. The $20B goes to Groq's investors.

▲ credit_guy 5 hours ago | parent [-]

From the article:

  > Groq added that it will continue as an “independent company,” led by finance chief Simon Edwards as CEO.

The $20B does not go to Groq's investors. It goes to Groq. You can say that Groq is owned by its investors, and this is the same thing, but it's not. In order for the money to go to the investors, Groq needs to disburse a dividend, or to buy back shares. There is no indication that this will happen. And what's more, the investors don't even need this to happen. I'm sure any investor that wants to sell their shares in Groq will now find plenty of buyers at a very advantageous price.

	▲	wmf 5 hours ago \| parent [-]
		Let's bet on this shit. Where's the Polymarket.

▲ p1esk 7 hours ago | parent | prev [-]

they spending $20 billion dollars to handicap an inference company

Inference hardware company