Just a single data point, but the BSDs in general, as much as people like to jerk them off, having tested both recent FreeBSD (which should be much faster than OpenBSD) and Debian on I guess the now kind of elderly APU2s I have, netfilter is noticably faster (and I find nftables to be frankly less challenging than pf) and gets those devices right at gigabit line speed even with complex firewall rules, where as pf leaves performance on the table. It probably has to do with the fact it's an older 4 core design that wasn't super high power to begin with (does still does its job extremely well), but still.

One issue I've seen from a fair number of people on the APU2s running FreeBSD is if they've got PPPoE; inbound traffic (at least) all hashes to the same RX queue, and as a result there's no parallelism... if you're on gigE fiber with PPPoE, the APU2 can't really keep up single threaded. The earlier APU (1) boards use realtek nics that I think only have a single queue, so you won't get effective parallelism there either. If I'm finding the right information, APU2s with i210 have 4 rx queues which is well matched with a quad core, but those with i211 only have 2 rx queues, which means half of the processors will have nothing to do unless your kernel redistributes packets after rxing, but that comes at a cost too.

Linux may have a different packet flow, or netfilter could be faster than pf.

> I find nftables to be frankly less challenging than pf

I also don't really care for how pf specifies rules. I would rather run ipfw, but pf has pfsync whereas ipfw doesn't have a way to do failover with state synchronization for stateful firewalls/NAT. So I figured out how to express my rules in pf.conf; because it was worth it, even if I don't like it :P