You can, but then those people have more money, and you have less. The money supply is much smaller than the supply of goods and services, because the same money circulates repeatedly. So simply doubling your money can tilt things a little more your way, but does not make the situation completely lopsided, because you can only spend the extra money once.

Also, to quote Jastram [0], "In spite of the romanticism of the Spanish Main and treasure ships, the supply of gold from the New World was just a trickle by later standards. Less than 1% of what was to be 1930 world production was produced in each of the years from 1500 to 1520." Over the next 80 years, it never got much above 1.3%.

[0] Roy W. Jastram, The Golden Constant (1977), pg. 41.