checkout llama.cpp, the entire point of the project is for us mere mortals and GPU poor.