The scenario you're describing seems like more of a language thing than a perception thing. We generally learn names of colors by references to common objects. I would argue that if people agree something is "Red, like a strawberry, tomato, or apple" then it doesn't really matter what you're seeing, that color is red.