Something I did not see in the article are router specific tuning such as

    net.ipv4.ip_early_demux = 0
    net.ipv4.tcp_early_demux = 0
    net.ipv4.udp_early_demux = 0

in /etc/sysctl.d/10_router.conf to slightly reduce overhead when being used primarily as a router. There are many other router related knobs but those I would always set especially if trying to reduce overhead for VoIP/Gaming setups. There are many other knobs I tune such as gro_flush_timeout and napi_defer_hard_irqs, sch_cake tuning, lowat and output limits and hundreds more but those rabbit holes would require a large write-up. My overall goal is to give family members latency, jitter and throughput numbers that improve their quality of life and gaming scores of course.

Such things do not preclude additional tuning on the client and server sides as well but those are even bigger topics.

▲ jcalvinowens 2 days ago | parent | next [-]

How much does disabling early_demux actually matter? I do this too, but honestly I've never tested what difference it makes.

I've always found fq_codel to be good enough for gaming through my router. But I have a 10gbps uplink.

▲

Bender 2 days ago | parent [-]

How much does disabling early_demux actually matter?

It depends on what kind of packet rate one is pushing through the device, what type of hardware is being used as the router and how sensitive ones applications are to such things but if you want a better scientific answer run as many load tests as you can that measure packet rates, lag and jitter then disable them and run your tests again. I should add this is just one tunable related to routing. If bored dig into this a lot deeper and one will find there are many things that can be adjusted at the NIC, in the OS network stack and so much more.

Some people like defaults because it is less cognitive load and some like to tweak things until they can't get better results. I believe that everyone should be able to choose their own path to satisfaction. If the defaults and fq_codel work for you then that is probably the right answer.

▲

jcalvinowens 2 days ago | parent [-]

I was just curious if you'd benchmarked it. I've always guessed it makes no measurable difference for packet rates achievable on a residential internet connection, but never bothered to check.

▲

Bender 2 days ago | parent [-]

Per the documentation it compensates for some overhead. The numbers on my hardware based on my usage patterns will not match the numbers on your hardware and your usage patterns. I have tested it on several different routers and saw a difference but by itself nothing to write home about in my testing. Just as there is the concept of a death by a thousand cuts, there are also notable improvements by adjusting each layer of the stack to be optimized for the workload. In the big picture it is enough difference for me to bother with along with a few dozen other settings especially for gaming.

I predict some day this will be one of many things that will get adjusted by the "tuned" daemon if they add a profile for routers.

▲

jcalvinowens a day ago | parent [-]

On the other hand, I can think of a lot of things I've seen done and done myself because they were "obviously" beneficial, which turned out to make no difference or be slightly harmful when I rigorously tested them.

Finding the balance is hard. I find as I get older I'm less inclined to twiddle knobs, but that isn't necessarily a universally good thing :)

▲

Bender a day ago | parent [-]

Well like I said, some people like defaults because it is less cognitive load and some like to tweak things until they can't get better results. I believe that everyone should be able to choose their own path to satisfaction. If the defaults and fq_codel work for you then that is probably the right answer.

▲

voltagex_ a day ago | parent [-]

I am more curious that OpenWrt doesn't set these (demux) settings. It feels a bit like looking for advice on ZFS tuning and finding out 10 years later that what you were told was completely wrong.

	▲	Bender a day ago \| parent [-]
		For what it's worth in your example there is a 10 year time span. What anyone was told 10 years ago about most things will have evolved with time, patches, major/minor code changes so if a configuration remains static for 10 years there will ultimately be room for some improvement just as the code itself and the hardware it is running on may have also changed. Where this concept gets a little unnerving is the medical industry. Imagine if your doctor did not stay on top of the evolution on medicine and technology. They would confidently perform the same actions and prescriptions they did a decade ago even if it was now know that those things were not only wrong but also wildly dangerous and ineffective. At least with routers and ZFS we are hopefully not using those to save lives and in the example in this thread it is just tiny change among hundreds of changes that have evolved with only marginal gains.

▲ BenjiWiebe a day ago | parent | prev | next [-]

If you write the large write-up explaining those rabbit holes, let me know. I'd like to read it.

▲ floralhangnail a day ago | parent | prev [-]

Got any more details about your setup? OpenWRT or something else?