| ▲ | itake 19 hours ago | |
> We time segmented the APIs to make sure that the simulation isn’t leaking the future into the model’s context. I wish they could explain what this actually means. | ||
| ▲ | nullbound 19 hours ago | parent | next [-] | |
Overall, it does sound weird. On the one hand, assuming I properly I understand what they are saying is that they removed model's ability to cheat based on their specific training. And I do get that nuance ablation is a thing, but this is not what they are discussing there. They are only removing one avenue of the model to 'cheat'. For all we know, some that data may have been part of its training set already... | ||
| ▲ | devmor 19 hours ago | parent | prev [-] | |
It's a very silly way of saying that the data the LLMs had access to was presented in chronological order, so that for instance, when they were trading on stocks at the start of the 8 month window, the LLMs could not just query their APIs to see the data from the end of the 8 month window. | ||