It’s more that both the club and the area are independently named after the site of the stadium where the club used to play. The club would still be called Crystal Palace even if the name had died out for the area of town.