Remix.run Logo
jiggawatts 9 hours ago

Meanwhile the best network I’ve ever benchmarked was AWS and measured about 55µs for a round trip!

What on earth are you using that gets you down to single digits!?

Galanwe 2 hours ago | parent | next [-]

> the best network I’ve ever benchmarked was AWS and measured about 55µs for a round trip

What is "a network" here?

Few infrastructures are optimised for latency, most are geared toward providing high throughput instead.

In fact, apart from HFT, I don't think most businesses are all that latency sensitive. Most infrastructure providers will give you SLAs of high single or low double digits microseconds from Mahwa/Carteret to NY4, but these are private/dedicated links. There's little point to optimising latency when your network ends up on internet where the smallest hops are milliseconds away.

crest 6 hours ago | parent | prev | next [-]

I assume 1-3 hops of modern switches without congestion. Given 100Gb/s lanes these numbers are possible if you get all the bottlenecks out of the way. The moment you hit a deep queue the latency explodes.

jiggawatts 6 hours ago | parent [-]

So, are you talking about theoretical latencies here based on bandwidths and cable lengths, or actual measured latencies end-to-end between hosts?

I know that "in principle" the physics of the cabling allows single digit microseconds, but I've never seen it anywhere near that low even with cross-over cables with zero switches in-path!

eqvinox 4 hours ago | parent [-]

You need high bandwidth links (time to get the entire packet across starts to matter), run on bare metal (or have very well working HW virtualisation support), and tune NIC parameters and OS processing appropriately. But it's practically achievable.

Switches in these scenarios (e.g. 25GE DC targeted) are pretty predictable and add <1μs (unless misconfigured)

6 hours ago | parent | prev | next [-]
[deleted]
blibble 3 hours ago | parent | prev [-]

that's because cloud networks are complete shit

this is xilinux/mellanox cards with kernel bypass and cut-through switches with busy-waiting

in reality, in a prod system