| ▲ | aschleck 5 days ago |
| It's been a while since I thought about this but isn't the reason providers advertise only 3.2tbps because that's the limit of a single node's connection to the IB network? DGX is spec'ed to pair each H100 with a Connect-X 7 NIC and those cap out at 400gbps. 8 gpus * 400gbps / gpu = 3.2tbps. Quiz 2 is confusingly worded but is, iiuc, referring to intranode GPU connections rather than internode networking. |
|
| ▲ | charleshn 5 days ago | parent [-] |
| Yes, 450GB/s is the per GPU bandwidth in the nvlink domain.
3.2Tbps is the per-host bandwidth in the scale out IB/Ethernet domain. |
| |
| ▲ | jacobaustin123 5 days ago | parent [-] | | I believe this is correct. For an H100, the 4 NVLink switches each have 64 ports supporting 25GB/s each, and each GPU uses a total of 18 ports. This gives us 450GB/s bandwidth within the node. But once you start trying to leave the node, you're limited by the per-node InfiniBand cabling, which only gives you 400GB/s out of the entire node (50GB / GPU). | | |
| ▲ | xtacy 5 days ago | parent [-] | | Is it GBps (gigabytes per second) or Gbps (giga bits per second)? I see mixed usage in this comment thread so I’m left wondering what it actually is. The article is consistent and uses Gigabytes. | | |
|
|
