What I’d really like is to run the admin interface for an app on a self hosted system behind firewalls, and push read replicas out into the cloud. But I haven’t seen a database where the master pushes data to the replicas instead of the replicas contacting the master. Which creates some pretty substantial tunneling problems that I don’t really want on my home network.

Is there a replica implementation that works in the direction I want?

Use NAT hole punching if you're advanced, or you could fall back to IP/port filtering

Tailscale will take care of the networking if you install it in both locations.