GPU-to-CPU Ratio?
Over on the NVIDIA CUDA Computing forum I saw that Dell is now shipping a 16 GPU PCIe Expansion Chassis.
If you dig around a little you’ll find a great video by the chassis architect that starts by describing the impetus for the product.
I thought it was really interesting that when an oil and gas customer came to Dell and asked for a chassis solution for GPUs, their “GPU-to-server” ratio requirement went from 2:1 in the beginning all the way up to 4:1 (4 GPUs per server).
Presumably this ratio was determined by testing and maybe tuning their GPGPU application. Or it simply might’ve been because the chassis made it practical to access 4 GPUs.
A ratio of 4:1 sounds high to many developers because it’s a challenge to install, power and cool that many GPUs in a standard chassis.
If the GPU:CPU limit is going to be loosened then it raises several questions:
- Which applications scale to high GPU-to-CPU ratios?
- How can developers practically find this limit?
- What GPU coordination “patterns” are there for scaling up?
- What is the next bottleneck: PCIe transfer speed or the need for device-to-device transfers?