It’s a 32 node gb200 cluster across 3 az (with one sitting in house), running 3 different open weight models. The remote az are subjected to preempt all the time too. So it’s much more than a day 0.

My point is that no one should actually read a guide like this and rollout the stack manually. A bootstrap agent can do all these by reading the engine-model cookbooks and combine with distributed system common practices (which the models know quite well).

The “cutting edge” issues such as engine bugs for new model releases, topology and spec decode conflicts, are what the models don’t necessarily directly know and needs human collaborations. They are very situational and is not covered anywhere (and shouldn’t be).

So you think learning about inference engines is a waste of time as a career move?