You are architecting a multi-node GPU cluster to train a trillion-parameter Large Language Model. Because the model must be distributed across hundreds of GPUs in different physical servers, the nodes must continuously synchronize billions of model weights during backpropagation. Which networking characteristic is absolutely vital to prevent inter-node communication from bottlenecking the entire training cluster?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Let me show you how this works in the real world. When you are training a massive model across multiple servers, the GPUs spend a huge chunk of their time talking to each other. They need to swap gradients and sync weights after every single batch. If your network is slow, it doesn't matter if you have the fastest H100 or B200 GPUs on earth—they will just sit there twiddling their thumbs waiting for packets. We call this the communication bottleneck. To prevent this, you need a specialized network with massive throughput and ultra-low latency—think InfiniBand or high-speed Ethernet with RoCE (RDMA over Converged Ethernet). This allows the GPUs to write directly to each other's memory across the network. Now, wireless networking is a total joke for high-performance computing, and standard redundancy protocols like Spanning Tree are too slow and inefficient. You need raw, low-latency speed to keep those GPUs fed. Got it? Let's keep rolling!
Full explanation below image
Full Explanation
Distributed deep learning training uses techniques like data parallelism, model parallelism, and pipeline parallelism to train large models across multiple physical nodes. During backpropagation, nodes must continuously perform collective communication operations (such as AllReduce, AllGather, and ReduceScatter) to synchronize gradients and model parameters.
Because these operations occur in synchronous loops, the training speed of the entire cluster is bottlenecked by the slowest network link (the 'tail latency' problem). To achieve efficient scaling, the network interface must support: 1. High Bandwidth: Typically 400 Gbps to 800 Gbps per link using technologies like InfiniBand or high-speed Ethernet to move gigabytes of weight parameters quickly. 2. Ultra-Low Latency: Latency in the sub-microsecond or low-microsecond range to minimize synchronization wait times. 3. RDMA (Remote Direct Memory Access): Allowing GPUs to transfer data directly from their memory to the memory of a remote GPU without involving the operating system or CPU on either end (using GPUDirect RDMA).
Let's analyze the incorrect options: - Option A: Wi-Fi lacks the bandwidth (caps at a few Gbps), has high latency, and suffers from packet collision and signal degradation, making it completely unsuitable for high-performance computing clusters. - Option C: While physical path redundancy and loops are concerns in traditional enterprise networking, standard Spanning Tree Protocol (STP) shuts down active paths, reducing usable bandwidth. Modern AI fabrics use active-active multi-pathing or InfiniBand routing (which does not use STP) to maximize bandwidth. - Option D: Network segmentation isolates administrative traffic for security, but it does not address the core data transfer performance requirements of the model training workload.
Therefore, high network throughput with low latency represents the most critical networking requirement for distributed AI training.