Interesting! I think the advantage of style fine-tuning is that you might not have to provides that much context upfront. Also, it's kind of magical to have an LLM just do something out of the box. I'll compare my local fine-tuned models against the baseline with instructions and see how they fare.

▲

badsectoracula 3 hours ago | parent [-]

Yes, TBH one of the reasons i want to try and finetune my own is to teach it stuff that now i have to explain before anything can be done :-P.

Unfortunately i only have a 24GB GPU - and an AMD one at that - so there isn't much i can do on that front. Supposedly a 24GB GPU is enough for finetuning a 24B model with 4bit QLoRA, though when i tried it with some finetuning app (in an official docker container) it barfed at Mistral's weird template or something and i lost interest after that.

▲

theletterf 2 hours ago | parent [-]

Try Runpod or similar services: you can fine-tune stuff for the price of a latte. Stanford's NLP course recommends them: https://cs336.stanford.edu/

	▲	badsectoracula an hour ago \| parent [-]
		I've heard about this but i prefer to avoid anything cloud based (not just for AI but in general). I try to avoid relying on stuff i have little to no control over.