lm studio is not opensource and you can't use it on the server and connect clients to it?
LM Studio can absolutely run as as server.
IIRC it does so as default too. I have loads of stuff pointing at LM Studio on localhost