Remix.run Logo
Shank 2 hours ago

I’m a huge fan of fiber optic cables instead of DAC. For one thing, the distance you can run is far greater. For another, they don’t get as hot as metal cables, particularly at high speeds. OM3 and OM4 is actually quite affordable and SFP+ is nice.

Yes on an absolute sense, DAC can be cheaper, but for a homelab or something, it’s not that much different. Also, it’s much cooler to run fiber optic to things (imho).

GoofGarage 21 minutes ago | parent [-]

In a homelab scenario when asking “DAC vs. fiber” the answer is usually, “yes.” :) Basically, it’s a tradeoff as with everything else.

DACs will usually be even (slightly) lower power per port, and slightly lower latency[1] (we’re fighting over microseconds here!), with excellent durability. The tradeoff is for passive DACs you’re limited on range, cost is often higher, and they may need to be encoded for your interfaces. Moreover, the range is very limited.

Fiber (the cable) is immune to electrical noise, can run long distances, advances in wave division multiplexing extends the life of the fiber by changing what’s the fiber connects to. The downside is you pay slightly in latency for media interface changes (the electrical-to-optical conversions), the limits of bend radius of the cable to not break the cable or reduce bandwidth, and the relative complexity of field terminations compared to twisted pair. I’ve 25+ years experience with fiber, and trust me, it’s great.

————

Outside of cost, both crush twisted pair like an ant. The power consumption per port is also far lower. However, this is only going to matter if you focus on limiting power consumption (not for cost, on principle), have very high-bandwidth applications where latency matters (I do!), and/or just want field experience with things other than twisted pair.

I use DAC and fiber for some things as I try to get every scrap of capability out of my hardware. For example, I have VERY low power (silent or near silent) hardware where I can push 5GB (so ~40Gbps) / sec storage. Not just sending it over the wire, but actually committing it to disk without buffering in RAM. So I have the capability of “PCIe 3.0/4.0 x4 NVMe” speeds across the network… from the (mostly silent) storage server, to anything else that can send or ingest the data that fast. Despite the storage server having very little flash (a few TB vs 100TB+ disk). That’s harder to do with twisted pair, or at least the power consumption of the network connectivity itself starts to add up for a few virtualization cluster nodes.

———-

Generally, “DAC in the rack, fiber to out back” is a reasonable approach. Though “fiber-only” works if you want to limit complexity!

[1] Fiber and DAC tend to trade places on latency every generation or so. It’s a very close race, but they crush twisted pair.

mwpmaybe 7 minutes ago | parent | next [-]

Couldn't have said it better myself.

sedimannapoleon 21 minutes ago | parent | prev [-]

[dead]