You could say the same thing about diagnosing cancer via astrology.

A test that with low specificity is simply not a useful test.

> we don't always know at the individual level if a specific tumor is harmless at the moment of diagnosis.

You didn't know prior to the test either.