NVLink This is a concept that Hemal Shah introduced me to, developed by NVIDIA. NVLink is allows multiple GPUs to communicate directly via a high-speed interconnection. It’s essentially faster PCIe. NVLink Switch https://www.nvidia.com/en-us/data-center/nvlink/