I suspect it is not just when fruit is scarce many if not most primates are omnivores. Fruit is likely easier than meat most of the time but meat contains dense nutrients. Macaques e.g crab eating macaques routinely eat meat.