It sounds great and I hope local LLM/AI will be more powerful and available even on old hardware.

I tried llama.cpp with offloading "all what could be" to my GPU (GTX 750 Ti 2 GB). I could run local Qwen2.5-Coder quite fast (I do not remember but something like 25-30 t/s). However this model generates bad code (I tried Python and JavaScript). So, for my setup it is better to chat with free ChatGPT and do copying-pasting between web chat and code editor.

Is it even worth for trying something on my GPU? What do you think?

And yes, I use Claude Code (Opus 5.5) and Codex in my job but for personal stuff I would prefer trying free solutions (it is the reason I tried local LLM). I do not want to buy subs, at leats for now.

Models and inference engines are getting better and more optimized everyday so it is definitely worth experimenting with your GPU. You should probably check on newer finetuned and or REAP models Huggingface that could fit on your system.

I believe in your case the model quality matters a lot. To my (trust me bro) experience any MOE model, finetuned or not, with params < 19B even with Q4-quant always give lackluster results. So any Qwen2.5 wouldn't suffice to even simple coding tasks. For your case maybe you can bump it up to a Gemma4 E2B/E4B or Qwen3.5-9B. Also make sure that your current build of llamacpp actually uses your GPU with CUDA.