I guess the fundamental question would be, for what domains can we pretrain with “good views”?

Part of the magic of next token prediction is it works to extract intelligence from the unstructured web corpus.

Can JEPA formulate a similar scale training set of “good views”?