What type of system is needed to self host this? How much would it cost?

One GB200 NVL72 from Nvidia would do it. $2-3 million, or so. If you're a corporation, say Walmart or PayPal, that's not out of the question.

If you want to go budget corporate, 7 x H200 is just barely going to run it, but all in, $300k ought to do it.

How many users can you serve with that?

For the H200, between 150-700. The GB200 gets you something like 2-10k users.

	▲	forrestthewoods 2 days ago \| parent [-]
		Whoa. How on earth can one system serve 2000 potentially concurrent users?

disiplus 2 days ago | parent | prev | next [-]

Depends how many users you have and what is "production grade" for you but like 500k gets you a 8x B200 machine.

p1esk 2 days ago | parent | prev | next [-]

Depends on fast you want it to be. I’m guessing a couple of $10k mac studio boxes could run it, but probably not fast enough to enjoy using it.

CamperBob2 2 days ago | parent | prev [-]

$20K worth of RTX 6000 Blackwell cards should let you run the Flash version of the model.