>How do you test for ad effectiveness vs annoyance?

In a walled garden like apple? You simply don't, just make the test gradual and long enough until people get used to it.

That’s the way it appears, sure. But my question is how would you do it if you did care. What metric would there be you could measure if they have no choice but to use the product.