> I think a potentially better way would be to segment the market up to today but take half or 10% of all the stocks and make only those available to the LLM.

Those stocks are going to be coupled. Let's take an easy example. Suppose you include Nvidia in the training data and hold out AMD for test. Is there information leakage? Yes. The problem is that each company isn't independent. You have information leakage in both the setting where companies grow together as well as zero sum games (since x + y = 0, if you know x then you know y). But in this example AMD tends with Nvidia. Maybe not as much, but they go in the same direction. They're coupled

Not to mention that in the specific setting the LLMs were given news and other information.