> and figuring out how to train a 2T parameter model

Why train your own model? You could use one of the many models already available on HF.