I’m trying to do something similar but hyper fine tune a model of choice for my specific local data source. For example, use existing code models to answer dquestions with code examples based on my private source files and documentation.

I tried doing it with using Huggingface and Unsloth but keep getting OOM errors.

Have anyone done this that runs locally against your own data?

I was just writing up a plan this morning. I use local models a fair amount, especially on trips.

My plan is to build a $150 AI bot, host it in my bedroom, give it access to all my writing, and let the world access it.