I’m work in ML and I don’t understand your point. Transfer learning usually refers to leveraging data for a different task to help with a task for which you have limited data.

You’re saying that the knowledge gained from the other languages transfers to English? I don’t think for a 270M parameter model the bottleneck is the availability of enough English language training data.

▲

tucnak 6 days ago | parent [-]

> You’re saying that the knowledge gained from the other languages transfers to English?

Yes, there has been many results circa 2020 or so, that have shown this to be the case. More recently, we have observed something similar with verifiable domains (see RLVR and related results) when it comes to coding tasks, specifically.

	▲	bigmadshoe 2 days ago \| parent [-]
		Right, but my point is that a 270M parameter model will not be bottlenecked by the availability of data for the entire English language.