If you look at the rate of "new" word use after the first spoken word its very clear that word acquisition and categorizing occurs for a long period before that first word is ever spoken.

Speaking to babies is incredibly important for linguistics but probably for all types of complex brain function, I don't think there is an upper bound on how many words we should expose children too.