doesn't there have to be a NAT64 gateway at the router?

There has to be one somewhere. In my case it's running on my router, but your ISP can do it at their end, and since you can use any /96 including public ones, it's also possible for anyone to run one:

  $ dig A +short github.com
  140.82.112.4
  $ dig AAAA +short github.com
  64:ff9b::8c52:7004
  $ dig AAAA +short github.com @2a01:4f8:c2c:123f::1
  2a01:4f8:c2c:123f:64:5:8c52:7004
  2a00:1098:2c::5:8c52:7004
  2a00:1098:2b::1:8c52:7004

  $ getent ahosts 2a00:1098:2b::1:140.82.112.4
  2a00:1098:2b::1:8c52:7004 STREAM 2a00:1098:2b::1:140.82.112.4