Yeah, most of us absolutely are not getting our money back.

The importers pay the tariffs, and they might get a refund, but it's unlikely they can distribute the money back to the people who they passed the price increase onto.

Imagine I imported 1 ton of rice and paid the tariff. Then I split that ton of rice into 2000 one pound bags and sold them to two super markets, with a higher price accounting for the tariff. Then one super market decided to absorb the price from their margins and sell it at the same price as before to avoid price shocks. Can I track down the other 1000 purchasers who paid a higher price? Is it even worth it?

Not only can’t you track down the 1,000 rice buyers, you don’t have any legal obligation to, so you 100% wouldn’t. (Not speaking of “you” just the general case of all importers who get refunds).

If you wanted to pass the refund on wouldn't the most sensible way be to pass it on to the two supermarkets since they were the actual buyers from you?

If the supermarket that raised prices wants to pass that on to their 1000 buyers that would be for them to deal with, not you.

The other important point is that those 2000 one-pound bags sold, so the market accepted the new higher price. Even after the tariffs are removed, the higher prices are here to stay.