Remix.run Logo
CobrastanJorji 7 hours ago

Yeah, the blog distinguishes between "contamination," which it describes as polluting the training data with answers to benchmarking questions, with "temporal leakage," which is polluting the training data with writing after the target date, but those seem to be nearly the same problem.

stingraycharles 6 hours ago | parent | next [-]

Not necessarily. The former is about data that’s supposed to be in there, but may actually be testing the model’s recall abilities rather than reasoning (ie rather than actually having a certain writing style, it just cites some passage it knows in that style).

The latter would be data not at all supposed to be in there, in this case, data after 1930.

zoomeriut55 4 hours ago | parent | prev [-]

a twit from 2025 saying "the capital of france is paris" is temporal leakage, but not contamination