Remix.run Logo
10Gb/s Ethernet: switching to a Broadcom SFP+ module(gilesthomas.com)
84 points by gpjt 6 hours ago | 64 comments
PaulKeeble 4 hours ago | parent | next [-]

I feel like we have moved into the era now where if you were putting cabling in the walls for networking you should be choosing fibre now. Not necessarily because we are definitely at the stage where the home needs it, but because the off ramp is clearly happening for ethernet at 10gbit/s and its really high consumption and heat. Switching to fibre after 2.5gbit/s seems like the thing to do now and plenty of us now have access to internet speeds that can exceed 2.5gbit/s.

Keyframe 2 hours ago | parent | next [-]

I did just that relatively recently in a house we bought. OS2 single mode duplex throughout the house, all converging to a trunk which is available in three locations for equipment. It's basically future proof, but also has its own well, things. You can't really plug into a duplex (I wish though), you have to put a small switch to it with SFP+ or 28 or whatever the speed you want. Higher speed switches are also a tad expensive. And then, there's the big one - PoE. That's why I also ran CAT6A next to each duplex to rooms and they're more or less for APs in the house. Overall it's definitely future proof and fantastic, but also a bit expensive if you wanna engage that fiber through the house. Pulling the cable itself isn't much of a cost at all and I recommend it.

brianwawok 26 minutes ago | parent [-]

I did this and used little $129 Zyxel SFP+ to 2.5G copper switches to get to my access points. Has been running smooth!

readingnews an hour ago | parent | prev | next [-]

Totally agree, I went to fiber years ago, and the decrease in latency makes it _feel_ so much faster than 10G copper, it is not funny. Besides, if you put in the "good stuff" them moving to 40G and beyond is not a problem later on. Like others said, just add a copper line for POE devices, but for systems... its fiber all the way.

drnick1 4 hours ago | parent | prev | next [-]

What for? Ethernet is what you ultimately need, because that is what devices such as PCs and WiFi access points use. I experimented with SFP for a while, but ultimately concluded that it isn't worth the effort to add SFP cards to PCs now that that low-power 10G Ethernet chips like the RTL8127 are available. High-end motherboards already have 10G Ethernet and soon lower-end models will too. 2.5G is practically standard already.

thefz 3 hours ago | parent | next [-]

Ever ran a single mode bidi fibre in a conduit? Push a wire puller, cleave and terminate ends, done. Zero effort unlike pulling a jacketed CAT7 cable, zero worries from electrical interference too, future proofing up to 40GBps. I ran double strands in my house so in case one breaks, there's another.

The floors where native fibre is not needed have a cheap ethernet media converter from fs.com, everything else (3 floor switches) are interconnected with 10Gbps SFP+ modules and 2.5G ethernet for the hosts.

All done thanks to the great https://michael.stapelberg.ch/posts/2020-08-09-fiber-link-ho...

(if you are reading this, I owe you several beers)

throw0101c an hour ago | parent | next [-]

If your walls are ripped open, then sure run some OS2 everywhere, but Cat 6A gets you 10GbE at 100m, but even 'only' Cat 5e or (plain) 6 gets you that speed up to 55m (per 802.3an):

* https://en.wikipedia.org/wiki/10_Gigabit_Ethernet#10GBASE-T

* https://en.wikipedia.org/wiki/Ethernet_over_twisted_pair#Var...

Why the need for faffing about with media convertors, at least with-in your domicile? (Fibre outside / to the garage certainly makes sense.)

drnick1 3 hours ago | parent | prev [-]

The issue with this setup is that you need an extra switch with an SFP+ uplink or media converter in each room or place where Ethernet will be used. And then you still need Ethernet cables anyway for the end devices. I can't justify this complexity for 40Gbps when I can now get 10Gbps inexpensively and conveniently.

esseph 2 hours ago | parent | prev [-]

My APs are fiber now.

cbdumas 4 hours ago | parent | prev | next [-]

There are probably still a lot of cases where you would want PoE though right? Cameras, WAPs, etc.

brianwawok 24 minutes ago | parent [-]

Waps are next to power, some even have SFP+ ports

throw0101c an hour ago | parent | prev | next [-]

> I feel like we have moved into the era now where if you were putting cabling in the walls for networking you should be choosing fibre now.

How many consumer devices have an ((Q)SFP(+)) optical cage?

If you're in their pulling stuff anyway, sure, do some OS2, but for most people, for most devices, Cat 5e/6 is more useful, especially since you can do POE(+(+)) over it as well. 5e/6 gets you 10GbE to 55m, and 6A to 100m.

protocolture an hour ago | parent [-]

Personally, if I was building a house, anywhere I want significant non wifi hardware would have a switch anyway.

collabs 4 hours ago | parent | prev | next [-]

I would be grateful and happy to have gigabit Ethernet with cat 6A in every room instead of this single landline phone jack and/or coaxial cable. The most important thing is a good conduit in place when the house is built.

tombert 3 hours ago | parent | prev | next [-]

I have ten gigabits throughout most of my house, and you're right: copper is not happy pushing ten gigs.

My 10 gigabit thunderbolt dongle weighs about a pound, and I think 90+% of that weight is just heatsink. If I've had it plugged in for awhile, if I accidentally touch that dongle it actually hurts because it's so hot. I cannot image that much heat is good for, well, anything.

I have another Thunderbolt dongle that has an SFP+ module, so I ran a fiber line from my switch to my computer, and that runs considerably cooler. That's what I use nowadays.

myrandomcomment an hour ago | parent [-]

What standard of cable? When I rewired I ran Cat6a everywhere. My longest 10G run is ~70 meters and works just fine. Anytime I had a link issue it was because I did not do the best job in termination on the keystone jack.

To be clear the Cat6a is thicker than Cat6 and harder to work with. It makes termination a bit more tricky.

tombert an hour ago | parent [-]

I use Cat 8.

The cables themselves don't get too hot, but the dongles themselves seem to get really hot. I'm assuming that's a known issue given the size of the heatsinks on them.

Hasz 4 hours ago | parent | prev | next [-]

you should be putting in conduit -- either smurf tube, emt, sch40, or similar. can pull whatever, and more importantly, if a cable is damaged by an overly zealous gorilla during installation, it can easily be fixed and replaced.

tiffanyh 2 hours ago | parent | prev | next [-]

> ethernet at 10gbit/s and its really high consumption and heat

Do you mean Ethernet cables get hot? Or just the networking equipment pushing that data.

I ask because I’ve never heard of Ethernet cables getting hot.

gerdesj 2 hours ago | parent [-]

PoE++ can get warm but you wont be doing that with fibre!

Before specifying fibre everywhere I suggest you note that a CAT 6 cable can manage 10G and PoE++. Its a lot more resilient to breakage too, especially outside the data centre.

If you really want to blow some cash there is CAT6A, which is probably not indicated unless you want cable lengths of more than say 50m.

zer00eyz 4 hours ago | parent | prev | next [-]

> the off ramp is clearly happening for ethernet

You should be running both.

If you are being smart about it your planning distributed switching (fiber to media boxes with power).

From a pure networking stance, fiber is the way to go. But POE continues to have more and more uses (doorbells, cameras, sensors, lighting controls).

whalesalad 2 hours ago | parent | prev | next [-]

You cannot do PoE over fiber.

esseph 2 hours ago | parent | prev [-]

You still need to power things.

Often that will mean running both Cat6A and fiber.

wingmanjd 5 hours ago | parent | prev | next [-]

Speaking of modules that lie about themselves, unifi has an interesting little device called the "SFP Wizard" that can reprogram sfp modules.

https://www.ui.com/us/en/integrations/accessory-tech/sfp-wiz...

Previously seen: https://news.ycombinator.com/item?id=45732874

kohlschuetter 5 hours ago | parent | next [-]

You can also use a BananaPi BPI-R3 for reprogramming. And you can “upgrade“ 10 Gbit/s DACs to 25 Gb/s. Details here https://kohlschuetter.github.io/blog/posts/2026/03/22/unlock...

hdgvhicv 5 hours ago | parent | prev | next [-]

I tend to use fs.com optics, but I’ve heard in high rates (100g plus) that flex optic tend to be more reliable

secabeen 5 hours ago | parent [-]

FS has a reprogrammer, called the FS Box. Works well.

theMMaI 5 hours ago | parent | prev [-]

fiberstore has them as well, plus you can buy modules, DACS and everything programmed to the vendor of choice, including different vendors on each end

Especially handy for specific Intel NICs where they refuse to link up if the module isn't in the driver-allowed list and those modules are hard to come by

tcdent 5 hours ago | parent | prev | next [-]

If you are implementing 10 GBE at distances less than 5-7 m, I highly recommend standardizing on DAC cabling. It removes the need for these kinds of conversions that create these kinds of heat signatures.

tuetuopay an hour ago | parent | next [-]

So much this. The rule of thumb is: avoid SFP-RJ45 converters at all costs, you'll be burned by them (literally and figuratively).

They all are little snowflakes. Compatibility is hit-or-miss. They run hot. They eat more power. They're finnicky. Heck, they plain out lie about what they are (I've got some that pretend to be fibre with 3m of copper, sure).

So yeah, DAC it is for patch, fibre for anything more.

iknowstuff 4 hours ago | parent | prev [-]

Wish apartments would come with those

happyPersonR 4 hours ago | parent | prev | next [-]

I know everyone is going to say I’m crazy

But for cabling, OS2 clear bend rated cable … pre-terminated is like the same price and currently have 25gb optics but I’m able to run over 100gb in my house without having to drill holes etc. (runs along the baseboards)

The cables are super thin… and clear/transparent

And I never have to replace the cable again I’m pretty sure haha

The bidi sfp28s $25 are awesome :)

And worst case if your service loop just … loops …. Eh haha

Gonna try using it for other things like hdmi etc too with a cassette :)

poisonborz 5 hours ago | parent | prev | next [-]

Cheap, low-heat 10G copper is already here. RTL8127 NIC is under $50, $200 nets you a quality 4x switch (CRS304-4XG-IN).

Havoc 4 hours ago | parent | next [-]

I've got a mix of both running here and definitely prefer the SFP+ part of the world. Couple of neat tricks it enables like the new "invisible" fiber - looks the same like fishing line basically. Unless you're 30cm away and actively looking you can't see it.

Replaced a wifi bridge that way...30m run across multiple rooms & hallway...zero drilling.

undersuit 4 hours ago | parent | prev | next [-]

I have a RTL8127 NIC from Aliexpress that uses 1 PCIE 4.0 lane, finally a use for those 1x slots and it does pretty good on 10G speeds.

ciupicri 2 hours ago | parent | prev | next [-]

What about performance and driver support for Linux? Any issues with suspending to memory or disk?

drnick1 4 hours ago | parent | prev | next [-]

I wish multiple-port 10G PCIe cards with this chip were available. I would immediately upgrade by Debian router from 2.5G to 10G. At the very least I would need a dual NIC.

boredatoms 3 hours ago | parent | prev [-]

and the QSW-L3208-2C6T-US is a cheap 8port switch

xxpor 2 hours ago | parent | prev | next [-]

I think this is the first time one of my comments has ever gotten mentioned in a subsequent post, so that's cool :)

Glad it was helpful and not me being an idiot. That's a shame about the temp read out. I just checked my MikroTik and can see the same thing. In fact, the only SFP module reporting a temperature at all is the real fiber one, all of the DACs/converters report nothing. No voltage either.

bhouston an hour ago | parent | prev | next [-]

My lessons on 10G networking at home is here: https://ben3d.ca/blog/home-network-lessons

debayande 5 hours ago | parent | prev | next [-]

Does anyone have any recs for GPON/XGS-PON SFP/SFP+ ONTs-on-a-stick that run cooler than average (say 50-65 °C)?

qurren 5 hours ago | parent | next [-]

I'm also using the WAS groupon stick with a huge fan on it. I really wish they would build a proper fan and cooling duct into the stick and power it with the stick itself. It very much seems like a half-assed solution.

debayande 5 hours ago | parent [-]

I'm considering that and the Nokia G-010S-A on my Uni-Fi Dream Router 7 (my area is serviced by multiple wholesale network access providers who operate either over GPON or XGS-PON, hence the need for both). However, I've heard enough horror stories that I'm a bit concerned about temperature issues...

qurren 3 hours ago | parent | next [-]

What horror stories have you heard? Just hot temperatures or smoke and fire?

matt-p 4 hours ago | parent | prev [-]

all XGS models I've tried run super hot. I don't know the standard for your area but here 99% of the time here you just get a 'media converter' style layer 2 ONT which I would just keep.

colechristensen 4 hours ago | parent | prev [-]

fs.com displays wattage ratings on all of their products, pick low watts.

https://www.fs.com/c/gpon-xgspon-sticks-5607 (I think this is what you're looking for?)

goolz 6 hours ago | parent | prev | next [-]

I am in the market for an SFP+ module and was looking at this exact model! The serendipity made me smile. Cheers mate.

gh02t 4 hours ago | parent [-]

I've bought a good bit of 10Gtek stuff over the years. Not sure to what extent they actually are designing their products vs. just acting as a reseller (I think the latter), but either way everything I've bought from them has been quality kit that lasted for years, at a great price.

cliftonk 4 hours ago | parent | prev | next [-]

i have gfiber 8gb put in my house. a cost-effective high performance setup im using is unifi cloud gateway fiber + 2x microtik CRS305 SFP+ (for all my other 10G devices) + 2x unifi flex 2.5G. this setup gives me a lot of 10G ports for little cost. copper DAC cables are great (optical transceivers and cables also work for longer runs). another great hack for older houses is using goCoax MoCA 2.5 Adapter to run 2.5G around the house via coax cables to your wifi access points.

jauntywundrkind 6 hours ago | parent | prev | next [-]

Recommendations for 25Gbit next please!

mittensc 5 hours ago | parent | prev | next [-]

why not use fiber directly and use whatever sfp for much cheaper without worry of heat

Tuna-Fish 5 hours ago | parent | next [-]

For a lot of people, because they already have copper in the walls.

You are correct that 10GBASE-T really shouldn't be the default choice, fiber and DAC both have advantages over it. But compatibility is important, and there are a lot of situations where 10GBASE-T is just more convenient.

nubinetwork 5 hours ago | parent | prev | next [-]

DAC cables can get expensive, and nobody knows what to buy when it comes to fiber, unless you're running entire spools of the stuff inside buildings... OM3/OM4/OM5? Single mode/Multi mode? LC/SC? Regular people don't know this stuff...

baby_souffle 5 hours ago | parent | next [-]

> Regular people don't know this stuff...

Regular people also are not buying DACs.

If you are in the line of work where you need to know what SFP is and the difference between DAC and Optical, a quick "what's OM3 vs OM5 and when do I use either?" to your favorite LLM/Search engine will get you sorted.

simoncion 3 hours ago | parent | prev [-]

> Regular people don't know this stuff...

Regular people don't know whether to get Cat5, Cat5e, Cat6, or Cat7. So... yeah.

> ...OM3/OM4/OM5? Single mode/Multi mode? LC/SC?...

My answer is OM4, Multi-mode [0], LC. OM3, 4, and 5 will all work at 10gbit for any run you'd expect to make in most houses. I chose cable grade based on what was in stock at the local store. I chose connector type based on what fit into my NICs. I went with multi-mode because it was cheaper than single-mode and I wasn't going to be making multi-km runs.

[0] That's what the "M" in in the cable designation means.

bigfatkitten 3 hours ago | parent | next [-]

Go with single mode only for new installs.

Biggest install cost is labour. The cable and optics are cheap now, and with the future (200Gbps+) being multiple wavelengths in parallel[1], we’ve pretty much hit the end of the road for MMF.

[1] https://www.tiafotc.org/ieee-802-3-ethernet-standards-update...

sekh60 2 hours ago | parent | next [-]

This is the correct answer, always single mode. It's been the most future proof to date, people just keep figuring out how to cram more and more wavelengths into it.

simoncion 3 hours ago | parent | prev [-]

> Biggest install cost is labour.

Okay? If I had to run cabling through a wall, I'd make sure the guy sets it up so that I can use the cable he installs to pull new later. My time's free when I'm doing something that I don't mind doing, and I don't mind easy cable pulls.

> ...(200Gbps+)...

Don't you need 16x PCIe 4.0 for those guys? With everything other than workstation and server boards having exactly one 16x slot, you're "never" hooking that up to a gaming PC.

ciupicri 2 hours ago | parent | prev [-]

> Regular people don't know whether to get Cat5, Cat5e, Cat6, or Cat7.

What are you talking about. It's right in the manual for some switches like the TP-Link TL-SX105 V4 [1]. It's not even an expensive or fancy one.

    Network Media (Cable)
    100BASE-TX: 2-pair UTP/STP of Cat. 5 or above (maximum 100 m)
    1000BASE-T: 4-pair UTP/STP of Cat. 5e or above (maximum 100 m)
    2.5GBASE-T: 4-pair UTP/STP of Cat. 5e or above (maximum 100 m)
      5GBASE-T: 4-pair UTP/STP of Cat. 5e or above (maximum 100 m)
     10GBASE-T: 4-pair UTP of Cat 6 (maximum 55 m) or STP of Cat 6, 6a, 7 (maximum 100 m)
If you're too lazy to read the manual you could probably ask chatgpt, gemini whatever. Or you could ask the guy from a store. A run of the mill store, not some crazy hobbyist store.

In the worst case you'll buy some overboard Cat 7 cable, but at least things are simple unlike with fiber optics last time I've asked [2]. With cable all you need to know is the speed. You don't have to ask yourself what kind of module you have or maybe you don't even have one. All you need to know is the speed and perhaps the length although I think only "the 1%" will need more than 55 meters :-)

[1]: https://static.tp-link.com/upload/manual/2025/202501/2025012...

[2]: https://news.ycombinator.com/item?id=47901136

simoncion 20 minutes ago | parent [-]

   Regular people don't know whether to get Cat5, Cat5e, Cat6, or Cat7.
> What are you talking about. ... If you're too lazy to read the [rare] manual [that contains advice on the topic] you could probably ask chatgpt, gemini whatever. Or you could ask the guy from a store. A run of the mill store, not some crazy hobbyist store.

I can query Google, an LLM, or a run of the mill cancer doctor for information on how to treat my stage 1 melanoma. That I can learn how to treat stage 1 melanoma doesn't mean that I know how to treat stage 1 melanoma.

> [Fiber optics are so complicated.] [With copper, all] you need to know is the speed and perhaps the length although I think only "the 1%" will need more than 55 meters :-)

For a 55 meter run, all you need to know is "Buy the cheapest multimode two-strand fiber your vendor has in stock. It's going to have LC ends, so get LC multimode optics.". You don't even have to worry about the speed of the transceivers to use this advice.

As an aside: Wow. That's [0] pricey for a dumbswitch that you also can't ever switch over to fiber. You can get a managed switch with four 10gbit cages, five 1gbit cages, and one 1gbit port for fifty bucks less [1], or a (physically much smaller) managed switch with four 10gbit cages and one 1gbit port for about the same price as that five-port TP-Link dumbswitch. [2]

TP-Link is absolutely raking in the dough on that unit.

[0] <https://www.newegg.com/tp-link-tl-sx105/p/0XP-001U-007G7> (apparent MSRP of 280->300 USD)

[1] <https://www.newegg.com/p/0XP-002R-000Y8?Item=9SIAEFKHB37914> (MSRP 200 USD)

[2] <https://www.newegg.com/p/0XP-002R-00108?Item=9SIB7VEJJD1334> (MSRP 150 USD)

seabre 5 hours ago | parent | prev | next [-]

For connecting via say a Macbook Pro, there used to not be Thunderbolt 4 SFP+ interfaces. So, you were pretty limited to some ethernet SFP+ module that you hope would actually work.

Also personally, if you can get away with a copper DAC, I would rather use that instead of fiber because you don't need any special modules.

drnick1 4 hours ago | parent | prev [-]

Access points, for example, expect PoE.

jmyeet 5 hours ago | parent | prev [-]

Some time ago I was playing around with 10GbE using a Macbook Pro. At the time that meant a Thunderbolt adapter (and still does). Thing is, the one I got was essentially just a giant heatsink [1]. It was a beast and belied just how much of a problem heat distribution was. I'm not an EE so I'm not really sure why, other than by looking at what high bandwidth cables have done since.

10baseT (!0Mbps) came out in 1990 (there were non-twisted pair earlier versions). "Fast Ethernet" (100Mbps) came out in 1995. Copper 1GbE came out in 1999. Copper 10GbE came out in 2006. Ethernet seemed addicted to 10x'ing every version and 10GbE is really where everything fell apart. Or at least, it's where it got hard. We never really got mass market 10GbE. The controllers were too expensive. The cable requirements were quite high. And heat was an issue.

1GbE really was fast enough and 10GbE was a massive jump that I even remember thinking at the time that there should've been intermediate steps, which is what happened in 2016 with 2.5GbE and 5GbE.

Now compare to Thunderbolt, introduced in 2011, which has completely surpassed Ethernet bandwidth, in part by putting chips in the cables, but of course the big difference is cable length. A copper cat 6/7 cable can get to ~100 meters, which is also why the power is so high: attenuation.

but I guess my point is that 10GbE over copper was a mistake. We'd reached the point where you really had to swap over to fiber.

[1]: https://www.ebay.com/itm/127178476193

xenadu02 4 hours ago | parent [-]

I don't think that's quite true. Unifi 10GbE switches are cheap enough I have a 24 port PoE+++ one in my house and my 3 main WiFi APs are 10GbE connected. My MBP uses a 5Gbps Thunderbolt adapter that runs relatively cool as well. All of this is over the existing Cat5e wiring.

I'd say 10GbE has arrived. It is relatively cheap, most of the time works over existing 1GbE cabling, and gracefully degrades to 5/2.5/1Gb based on conditions when it can't reach 10Gb.

Yes to be 100% guaranteed of getting 10Gb even in bundles of 100 cables running over noisy fluorescent ballasts to a full 100m you need Cat6A but in many environments Cat5e or Cat6 is more than sufficient. It works so well if you fail to get the full 10Gb I humbly suggest you re-do the terminations on both ends before considering replacing the cable.