You can't claim it "works" if it hasn't produced any coherent responses and is still early in your first training attempt.

▲

volotat 3 hours ago | parent [-]

It is a goalpost that is easy to move. By "works" I mean learning from a continuous single (meaning batch-1) stream of data. The fact that it produces full words and full coherent phrases instead of a random stream of characters that would any typical LM produce if trained under the same training regime.

▲

ilaksh 2 hours ago | parent [-]

I would be okay if you shared it as a potential idea and possibly interesting early result, but the language you are actually using to characterize it is misleading or delusional.

Please get a model to the point where it seems like it has some natural language understanding and then share again with reasonable characterization.

	▲	volotat 2 hours ago \| parent [-]
		For sure. As it will pass through the whole corpus I will share the weights, run it through established benchmarks for small models and share all of this as an update. I am also planning on making a Youtube video explaining in detail how it works on a deeper level and the whole reasoning behind why it is built the way it is. But no promises here.