What InfiniBand is
InfiniBand is a network technology built for high-performance computing. It links servers through dedicated switches and adapters, and it is designed to keep delay low and throughput high. The whole network of linked machines is called a fabric. Throughput means how much data moves through the network each second.
InfiniBand also includes remote direct memory access, usually shortened to RDMA. This feature lets a machine read or write memory on another machine with little help from the processor on either side. Transfers like this avoid much of the overhead that slows ordinary network traffic.
What Ethernet is
Ethernet is the general-purpose network standard found in most data centers and offices. It runs on a very wide range of hardware, and most engineers already know how to operate it. Its traffic normally passes through the operating system stack, which adds delay and uses processor time.
RDMA over Ethernet, often called RoCE, brings the same direct memory model to standard Ethernet links. It needs careful network setup, because a lost packet, a small chunk of data on the wire, can slow a transfer sharply. Both options can carry RDMA-style traffic, so the real question is how well each network is built to avoid loss.
Why the interconnect matters for multi-node training
Training a large model across several machines means the GPUs must exchange data many times. GPUs are the graphics processors that do most of the math in training. After each step, the GPUs share gradients, which are the numbers that say how to adjust the model. Many of these exchanges are collective operations, where every machine joins a combined calculation, such as an all-reduce.
The interconnect is the link that carries this traffic. Latency and bandwidth are the main measures of that link. Latency is the delay before a message starts to arrive, and bandwidth is how much data the link can carry each second.
Small messages depend mostly on latency, while large transfers depend mostly on bandwidth. When the link is slow or adds delay, the GPUs sit idle while they wait. Idle hardware is costly, and the training run takes longer than it should.
How the network behaves under load
The difference that decides most cluster choices is how the network behaves under load. InfiniBand is built as a lossless fabric. Flow control, where senders wait for room on the receiving side, is built in from the start. Ethernet is a general network, so its behavior under load depends on configuration and tuning.
Congestion happens when too much traffic competes for the same links. Under congestion, a well-tuned Ethernet network can come close to InfiniBand, but it takes more setup work to stay predictable. For multi-node training, the gap shows up as how steady the latency stays when many machines send at once.
When InfiniBand wins
InfiniBand usually wins when a job is tightly coupled and sensitive to delay. Large training runs that split a single model across many machines, with frequent collective operations, benefit most. Teams that want predictable behavior without heavy network tuning also tend to prefer it.
It also suits operators who already run a dedicated high-performance fabric. Their staff can keep the cluster running with tools they already know. Mixing general traffic into that fabric is usually a poor idea, so plan it as a separate network.
When Ethernet is enough
Ethernet is often enough when jobs are small, loosely coupled, or mostly confined to a single machine. Inference serving, fine-tuning on a few GPUs, and batch work that rarely crosses machine boundaries can all run well on it. The surrounding tools and skills are also broad, which keeps parts and staff easy to find.
If the cluster is modest and the workload tolerates delay, a carefully built Ethernet network with RDMA support can be a sensible choice. Check the provider's own documentation before you commit. Network options, RDMA support and the exact fabric on each machine type vary by provider, and they can change over time.
Which to pick
The right choice depends on your setup. Match your situation to the groups below.
- Teams running large, tightly coupled training should lean toward InfiniBand, and plan for a dedicated network.
- Developers training on a few machines should start with Ethernet and measure how much time the GPUs wait.
- Operators running mainly inference or independent jobs should favor Ethernet, with RDMA support where it helps.
Measure before you spend on a network upgrade. Watch whether the GPUs spend much of their time waiting on the network. If they do not, a faster network is unlikely to change your results much.
For the hardware side of the decision, see the GPU provider comparison and read the GPU selection guide.