The problem is the amount of data with that cutoff is really minuscule to produce anything powerful. You might be able to generate a lot of 1700s sounding data, you’d have to be careful not to introduce newer concepts or ways of thinking in that synthetic data though. A lot of modern texts talk about rates of change and the like in ways that are probably influenced by preexisting knowledge of calculus.

▲

NewJazz 2 hours ago | parent [-]

Doesn't it prove GP's point then, that LLMs themselves simply aren't capable of creating/proving new theories and axioms?

	▲	codebje an hour ago \| parent [-]
		Without passing opinion on GP's point, I think that just proves it's hard to establish a data set that doesn't bias toward the result you're hoping to find.