It's a rather obvious allegory for the loss of childhood innocence. We all have to leave The Garden at some point. See also Puff the Magic Dragon.