Very interesting - I have a tangential question.

What motivated you decide to release this. OpenAI or Anthropic will just hoover it up, maybe scale it up and use it if they are interested.

You probably won't know if they do, and the chance they will give you something back is near zero. Why did you release rather than try to scale and build yourself?

(I've been working on some thing, not similar, but not dissimilar in goal - and I just can't get over the fact that tech will steal without giving back)

I have no practical means of scaling it up at any compatible scale. I will not make any money on it either way as well. So there is absolutely no reason for hoarding it. And as I said I did use Claude in the process, so Anthropic already has full access to it anyway and could steal it just as easily if they really want to.

I also doubt it is really that valuable on the OpenAI/Anthropic scale, at the same time if people will use it and it will work for them on the personal scale it is already a major win for me. New ideas and optimizations I could never have thought of might bring this up from a toy model to an actually useful model trained locally. Then people could add RL and RLHF and other cool things to it to make it even better.