You are confusing the audio latency with the processing latency. I see this mistake a lot.

One way audio latency is about 3.6 ms divided by two = 1.8 ms. This type of latency is audible.

Processing latency is just the amount of time it takes to complete all processing for each audio cycle. From the histograms, the typical processing latency is about 625 us and of course there is some jitter (almost all of the jitter comes from the Linux host BTW). Processing latency is not audible. However if the processing latency exceeds the deadline on a given audio cycle, there will be an underrun which will cause an audible glitch.