Remix.run Logo
▲ jains99 3 hours ago

But hardware to run them is conveniently something IBM would be happy to sell you.

▲mbreese 38 minutes ago | parent | next [-]

I didn't think that IBM had gear that mortals would want to run. If you're not running a mainframe, what do they have to sell you? They still sell Power servers and it looks like they have an AI card that can be used on both. But, if we're talking about the IBM CEO talking about on-premises, what are they really talking about hardware-wise? It's not something they would sell, is it?

This is more of a statement for their consulting customers, right?

▲jacquesm 2 hours ago | parent | prev | next [-]

If IBM would ship a piece of affordable hardware that can run large LLMs efficiently I'd be lining up.

▲solyin 2 hours ago | parent [-]

"Affordable" and "IBM" don't belong in the same sentence together, but they're happy to sell you their inference-capable hardware if you've got the wallet for it.

▲jacquesm 2 hours ago | parent [-]

I don't agree with that. They made the PC revolution possible by making hardware that was at least affordable enough to be bought by a large number of businesses and individuals. It also set the stage for the clone market. I'm pretty sure that if IBM really set their mind to it that they could do this.

That doesn't detract from your overall point that they tend to make overpriced stuff, but I don't think they have to do that. Besides, what's overpriced? NVidia's GB300 workstation sells for 95K euros, a single RTX 6000 is now 20K. Surely even IBM can compete at that level.

▲rnd0 2 hours ago | parent | next [-]

That was completely accidental and they themselves tried to close the barn door with the PS/2 line in 1987 but at that point it was too late.

▲jacquesm 2 hours ago | parent [-]

True, even so: They still make some pretty impressive hardware. They have access to fab capacity, they know how to make chips. They have the capital. If they really wanted they could do it.

▲solyin an hour ago | parent [-]

In what way is the IBM Spyre not what you're asking for?

▲jacquesm an hour ago | parent [-]

In a way it is, but I would see that as a starting point, not as the ready solution. They'd need to do what NV has done for their eco-system: fantastic libraries, a credible future upgrade path, easy ordering, manufacturing partnerships, the works. I figure besides the performance gap that they are at least 5 to 10 years behind NV with their present offering. But NVidia is dropping stitches and IBM should take advantage of that.

I think IBM sees the Spyre as a reason to sell more Power hardware but that's just not where the industry is right now and I doubt it will be enough to create sufficient demand. Who knows, I don't have any insight in what they're doing there, but there is massive gap between A GB300 workstation and what IBM currently offers. At the same time: the RAM crunch hits all parties and that's a major obstacle as well.

▲solyin an hour ago | parent [-]

Yeah. For all the jokes we make about paperclip maximizer, we've figured it out. It's not paperclips, it's RAM.

That's the thing though. Software is cheap with AI, but Nvidia has Cuda. If software doesn't matter anymore (it does) because it's so cheap now, then Nvidia's moat evaporates to IBM's Spyre. In reality "just a little bit of software" has inertia and cannot be dislodged quickly, quietly, or easily. cf Microsoft.

▲jacquesm an hour ago | parent [-]

That. And not starting every sale with 'call us' where they first want you to tell them how much you can afford to spend. I buy a fair amount of gear and if the sales process starts with some sleazeball sales guy instead of a mouseclick on a local webshop then I'm out.

As for CUDA: I think CUDA is a formidable moat but I also think that that is precisely the kind of thing where the likes of IBM could (if they really wanted to) make things happen. They're already in the space with Redhat and they have ample knowledge about drivers and low level code. Putting together a team that either makes a drop-in replacement or alternatively, that starts supporting their custom hardware for the big players in the inference space (VLLM, llama.cpp and Torch) would make all the difference. They'd have to keep doing it though, even if the initial results were lower than expected because that market is not going to turn on a dime.

▲exac an hour ago | parent | prev | next [-]

Respectfully, those people don't work at IBM anymore. The organizational memory doesn't last that long.

▲jacquesm an hour ago | parent [-]

You'd think that with a 250K workforce and an active line of hardware including an AI accelerator that those people are still working there. I could be wrong, but someone has to be doing that work unless IBM in turn outsources it.

▲quailfarmer 2 hours ago | parent | prev | next [-]

They haven’t been _that_ IBM for decades

▲jacquesm 2 hours ago | parent [-]

I know. But: they have gobs of money, they still have some talent, they still have a hardware department that I'm pretty sure would jump at a chance to give NV a run for their money. This would require a massive long term investment and a commitment that I'm not sure they would have the stomach for.

▲alienbaby an hour ago | parent [-]

"they have gobs of money, "

They do? They are a spec in the ocean relative to big players offering inference and training their own models. Pretty sure one thing they are struggling with is the money to compete successfully in the ai game this time around.

▲jacquesm an hour ago | parent [-]

10 billion give or take, market cap 250 billion, 250K employees. That's not a 'spec in the ocean' by normal standards, and IBM is a mature company, not an inflated balloon.

▲j16sdiz an hour ago | parent | prev | next [-]

It's the "PC-compatible" hardware make those affordable, not PC.

▲dansquizsoft 2 hours ago | parent | prev [-]

> They made the PC revolution possible

Yeah they were only half a decade late to the PC revolution back in the day...

▲wmf 2 hours ago | parent [-]

They weren't really late. In many people's minds IBM validated the concept of the PC.

▲jacquesm 2 hours ago | parent [-]

When that happened I'd been working at Tandy (EU version of Radio Shack) for a while and even though we sold computers they were not taken serious until IBM came along and then suddenly we couldn't supply them fast enough.

▲spopejoy an hour ago | parent [-]

Was Tandy only EU? In the TRS-80 days the name came up a lot, although maybe just because of the acronym ... 6809E was a good time

▲jacquesm an hour ago | parent [-]

Yes, that was the first time I had a 'proper' processor to play with. A Dragon 32 (CoCo clone from the UK).

I've never seen Tandy used outside of Europe but then again, I had not traveled as much then as now so it's definitely possible they used the brand on other continents as well. I wonder how many trees were cut for those catalogs :)

▲numpad0 2 hours ago | parent | prev | next [-]

Yes but unironically? IBM should be what NVIDIA would be competing with their NVL72 and DGX products. It doesn't make sense that IBM is just watching NVIDIA take its lunch.

▲jacquesm an hour ago | parent | next [-]

I think they will need a far more aggressive management before such a thing could happen. But I can dream about it.

▲verdverm an hour ago | parent | prev [-]

> It doesn't make sense that IBM is just watching NVIDIA take its lunch.

If you have ever worked at IBM, it makes total sense. Completely dysfunctional middle management and on up.

▲anukin 3 hours ago | parent | prev [-]

Not till Jensen buys his next leather jacket.