> human raters comparing watermarked and unwatermarked answers side-by-side saw no difference in quality

Yes - maybe saying "vibes" was minimizing the effort but what I am trying to say is that even the controlled testing is just asking users whether quality is impacted or not. Which is subjective and thats what I meant by when I said "vibes"

Don't get me wrong - I have no idea how one would go about testing this with other methods; I was just stating my assumption.

Since they rolled this out to all users I had assumed there would be other testing involved.