Empires have often grown without any desire to grow. Quite often empires have grown by moving against a threat close to its borders or trade routes and ends up in control of new areas, which then become part of its territories, which are then threatened....

Expansion is not uncontroversial either. The Roman Empire had periods of a non-expansionist policy, and the British Empire in the 19th century had a fairly popular Little Englander movement that opposed expansion of the empire.

The desire to grow is simply hidden behind the supposedly defensive posture of wanting to have a peaceful "buffer zone".