GPU Hack: LLVM for pre-Fermi Kernels

I’ve been cleaning up a set of kernels so that they will run optimally on GT200 devices (sm_1x). The kernels run extremely well on Fermi so I was disappointed when the opencc compiler struggled to use a reasonable number of registers despite having access to the same number of registers per thread.

I was getting over 200 bytes of spills in a critical kernel that had no spills at all on Fermi.

Not good! So what could I do?

In my case, the answer was to force use of the LLVM compiler with the --nvvm switch. This produced kernels with either zero or at most 8 bytes of locals.

My understanding is that this switch is unsupported for pre-Fermi devices but it worked very well for me and all of the kernels passed their verification tests.

On an old GT215 @ 550 MHz:

opencc:   29.14 MKeys/sec
4.1-nvvm: 42.56 MKeys/sec
5.0-nvvm: 43.30 MKeys/sec

Almost a 50% improvement… I’ll take it!

Update Aug. 17, 2012:

CUDA 5.0 RC finally removed the --nvvm compiler switch.

The workaround is to generate sm_1x PTX with a 4.x compiler and generate the cubin with the bug-fixed 5.0 ptxas.

It wasn’t mentioned above, but 4.x and 5.0 Preview had a bug where sm_12 devices were treated as if they were sm_11 devices with only 8192 registers. That bug is fixed in 5.0 RC thus the baroque workaround I’m suggesting.