Taking your stuff is indeed infringing upon freedom, and money is indeed "stuff". And money is how you pay for food, and housing, so doubly infringing.