What throughput do you get on those? I worry latency, speed, and whatnot will be impacted without dedicated silicon.

Systems like the Banana Pi 4 can easily route traffic at near 10 gigabit wire speed. Granted, this particular system has very good hardware offloading capabilities.