The point of the metaphor is not to say "spending time is mechanically similar to putting things in a container". It is to look at spending time from a new angle, and see if it helps you understand it better. A wise person sees a metaphor as a launching point for thought, not as an expression of a metaphysical connection.

Yes, there are bad metaphors, and people who take metaphors too seriously. That you can conjure a bad metaphor with somewhat similar to semantics to some other metaphor does not mean that said metaphor is bad.