Of course, any two traits can be correlated. But unless there is an obvious causal mechanism ("whiter skinned people get sun burns more easily", "darker skinned people tend to have vitamin D deficiencies at higher latitudes"), there's no reason to assume it and start looking for it to prove or disprove. It's a priori just as likely that there is a genetic link between toe shape and intelligence as skin color and intelligence. Yet no one is doing the science of measuring toe length correlation to intelligence, just skin color ("race").

As far as I know phrenology measured all kinds of things and claimed correlations. But they used highly dubious data (even for their time) and the field is now too unpopular to try (something similar) again with better methodology.

And I would say that the racist claim is that there is observable correlation and that genetic explanations are about as probable as other explanations. But I suppose many non-genetic explanations (eg cultural differences) would also be counted as racism, since in these kinds of discussions race and culture are often jumbled together.