> I had older NVIDIA card (RTX 3070) and llama.cpp instalation was smooth

what model was it that you were able to run with the rtx 3070?

I was able to fit only small models Qwen3.5-4B in RTX3070 which is not very useful for Python and SQL generation thought. When I wan to test larger open LLM models I often just use cloud resources.