How do I inspect GPU topology and PCIe links on a dedicated server?
GPU placement and interconnects can affect transfers between CPUs, GPUs and network adapters. Inspect the actual topology before attributing slow multi-GPU work to compute capacity. A topology report describes connections; it does not by itself measure application bandwidth.
Before you start
- The intended GPU identities and permission to inspect the host hardware view.
Instructions
1. Read the connection matrix
Run the host-level topology query and inspect its legend, CPU affinity and device paths. Some fields or adapters may be unavailable on a platform or in a guest, so compare the output with the actual server design.
nvidia-smi topo -mDisplays supported GPU/NIC interconnect and affinity information. The output is descriptive and does not reconfigure device placement.
2. Inspect the selected link details
Use a verified device index or stable identity. In the query output, review PCI bus identity and current versus maximum link information where applicable. Some integrated platforms use a different primary interconnect.
nvidia-smi -i 0 -qReads detailed information for the selected GPU. Keep unrelated identifying details private when sharing an extract.
3. Compare under the intended workload
Current PCIe generation or width may reduce while idle, so a lower idle value is not sufficient evidence of a fault. Correlate the topology with measured transfer and application performance during an authorised representative job. Escalate persistent unexplained limits with device and timestamp evidence.
Check the result
- The reported topology matches the expected hardware arrangement and explains the relevant transfer path.
Troubleshooting
- Adding another GPU does not improve throughput
- Check inter-GPU communication, CPU affinity, model partitioning and input delivery. Additional devices can add synchronisation overhead when the workload or topology cannot use them efficiently.
Official references
Use the documentation for your installed version when a flag, service name or platform feature differs.