I don't think that's true. You'll just look like someone who already has it cached.

It depends on how the browser rejects favicons? If the browser reports the icon is already cached, I agree (assuming the reports are indistinguishable). But maybe it just never downloads the icon, for example.