Didn't set it up myself so my info will be meagre, but I think it's litellm and a big enough nvidia card. We use it for development but also for internal applications (where smaller models might escalate a task to it when needed).

Obviously not feasible for (just) personal use, the model is too large for consumer hardware.