I just use llama.cpp. It worked out of the box.