That’s on Anthropic, but also on the broader trend. AI companies and the current state of ML research got us into this reproducibility mess. Papers and peer review got replaced by white papers, and clear experimental setups got replaced by “good-faith” assumptions about how things were done, and now I guess third parties like security companies are supposed to respect those assumptions.