The common ground Republicans and Democrats can find is that neither wants the power of government used against them or their rights. The best way to stop the government from being used against either party is to shrink the government until it is a threat to neither party. Lower taxes, less spending, and no regulations infringing on rights or freedoms.

This is one solution to the problem. But it isn't as if its the only solution or one that has no downsides.

I mean from a very trivial point of view, government spending constitutes a large amount of GDP.

What this comes down to, in my opinion, is the question of democratic allocation of resources and labor. Most people believe that there is a role for democracy in the allocation of resources and labor, which is to say that we think that certain societal goals (for example, defense, the care of the elderly or the poor, etc) should not be allocated to by markets but by democratic will.

This seems to be something almost all Americans agree on (though what things should be handled this way and how is contentious).

But to simply shrink the government away has the effect of decreasing the power of democracy to allocate resources, transferring that power to (in an ideal case, anyway) markets.

The fact is, most people do not want to live in a pure free market society, as far as I can tell. They want government services, they want safety nets, they want the air they breathe to be clean and safe. They want the power to decide that sometimes its worth spending money on stuff even if no one accumulates profits in the process.