It would if the saying was "vote with your dollars" or "vote with the dollars in your wallet". A literal reading of the term means you signal your vote/opinion by choosing what to pay for and it can hurt businesses since they have to generate revenue, not that $1 = 1 vote.

I disagree. The wallet is a term that can be augmented by 'fat' or 'full' or 'heavy', which means that a wallet can be different sizes. From this you would get that poor people would have thinner wallets and thus less effect on outcomes where money is a factor.