Remix.run Logo
jmull 2 days ago

> Roughly speaking, the latency of systems like object storage tend to have a lognormal distribution

I would dig into that. This might (or might not) be something you can do something about more directly.

That's not really an "organic" pattern, so I'd guess some retry/routing/robustness mechanism is not working the way it should. And, it might (or might not) be one you have control over and can fix.

To dig in, I might look at what's going on at the packet/ack level.

nkmnz 2 days ago | parent | next [-]

I don't know what you mean by the word "organic", but I think lognormal distributions are very common and intuitive: whenever the true generative mechanism is “lots of tiny, independent percentage effects piling up,” you’ll see a log‑normal pattern.

jmull 2 days ago | parent [-]

You can think of a network generally as a bunch of uniform nodes with uniform connections each with a random chance of failure, as a useful first approximation.

But that’s not what they really are.

If you’re optimizing or troubleshooting it’s usually better to look at what’s actually happening. Certainly before implementing a fix. You really want to understand what you’re fixing, or you’re kind of doing a rain dance.

pyfon 2 days ago | parent | prev [-]

How do you do that for an abstract service like S3? I see how you could do that for your own machines.