Right - we don't know. Even with mass-surveillance, you don't know if they're a criminal, because criminals can look innocent, and innocents can look criminal. That's why you need to do more than just look at the whole world and look for any pattern of criminality - you need targeted detection and analysis.
An example is cops stopping and searching you. If they're looking for a murderer, and they stop people coming out of Home Depot, and find a shovel, lye and rope in your trunk - hey, looks like a murderer! But it also could just be any customer going to Home Depot to get gardening supplies. So they can only stop people if they have probable cause: a reason they specifically think this specific person is the murderer.
(The other reason for probable cause, which is just as valid today as it was during the Framers' time, was to protect innocent civilians from unreasonable search and seizure by the Government, as well as protecting personal privacy. Mass surveillance violates all those principles.)
