You can fully train a 1.6b model on a single 3090. That’s a reasonably big model.

you can train it, but not fully