Would running vLLM in docker work for you, or do you have other requirements?

its not an image and audio model, so I believe it wouldn't work for me by itself

would probably need multiple models running in distinct containers, with another process coordinating them