Yes. Even dealing with "what does it look like in rain? What does it look like in snow?" is hard. Hell... "What does it look like at night" is hard. Hell.... what does it look like at noon vs sundown (no shadow vs long shadows) is hard.

Have you ever seen a commercial use of anything like this? That should give you a hint about how reliable these systems get.