https://dispatch3.com/tags/gpu-hacks/Recent content in GPU Hacks on Dispatch3 Inc.Hugo -- gohugo.ioen-usThu, 03 Mar 2016 00:00:00 -0500https://dispatch3.com/posts/cuda_macro/Thu, 03 Mar 2016 00:00:00 -0500https://dispatch3.com/posts/cuda_macro/I just switched over to a new CUDA error checking macro and really like how it works. This error checking macro will transform a CUDA Runtime API function into its error-checked equivalent simply by parenthesizing everything after the lower case cuda function prefix. Examples: cuda(GetDeviceProperties(&props,device)); cuda(SetDevice(device)); cuda(Malloc(&vin_d, bytes)); cuda(Memset(vin_d,0,bytes)); cuda(Free(vin_d)); cuda(EventCreate(&end)); cuda(EventRecord(end)); cuda(EventSynchronize(end)); cuda(EventElapsedTime(&elapsed,start,end)); cuda(EventDestroy(end)); cuda(DeviceReset()); The C99/C++ macro and assert look like this: #define cuda(...) cuda_assert((cuda##__VA_ARGS__), __FILE__, __LINE__, true); cudaError_t cuda_assert(const cudaError_t code, const char* const file, const int line, const bool abort) { if (code !https://dispatch3.com/posts/smem_idiom/Thu, 18 Jul 2013 00:00:00 -0400https://dispatch3.com/posts/smem_idiom/It’s common to declare a number of shared memory variables at the top of a CUDA kernel file. You often see these declarations with Hungarian’ish names that imply the variables somehow reside in shared memory. But as someone who likes to write highly readable code, I’ve found that naming, referencing and maintaining independently declared shared variables can sometimes turn into an endeavor. Some example issues: I would like to use the same meaningful variable name for both the shared and in-register instances.https://dispatch3.com/posts/kernel_args_vs_constants/Thu, 09 May 2013 00:00:00 -0400https://dispatch3.com/posts/kernel_args_vs_constants/It’s not uncommon to have a GPU kernel where a number of the kernel parameters are left as constants throughout the entire call chain and, if the target were a CPU, would have been implemented as traditional static const file scope variables. For this reason, you might think that exploiting the __constant__ qualifier and its associated cudaXXXSymbol() routines is a good way of reducing the number of kernel arguments and generally cleaning up your code.https://dispatch3.com/posts/fast_matrix_transpo_kepler_2/Sun, 07 Apr 2013 00:00:00 -0400https://dispatch3.com/posts/fast_matrix_transpo_kepler_2/Last time I presented a clever way of transposing that exploited the GPU’s support of 32-byte stores as its smallest “100% efficient” transaction size. An open question was whether designing a transpose that performed 64-byte stores would achieve better performance at the cost of more instructions. The answer is yes. Performing 64-byte transactions improves throughput by ~8%. The new 64-byte transpose kernel reaches ~148 GB/sec. on a 1024x1024 matrix of 32-bit elements on a K20c (758MHz).https://dispatch3.com/posts/fast_matrix_transpo_kepler/Tue, 26 Mar 2013 00:00:00 -0400https://dispatch3.com/posts/fast_matrix_transpo_kepler/I needed a fast and minimal routine that could transpose a warp with 32 elements per lane and store the result to device memory as the final step in a complex kernel. However, I had a hypothesis that Kepler SHFL instructions and 16-byte stores by pairs of lanes would be functionally equivalent to the standard matrix transpose approach but not require any shared memory or thread block synchronization yet still achieve 100% memory store efficiency.https://dispatch3.com/posts/experiments_with_shfl/Thu, 14 Mar 2013 00:00:00 -0400https://dispatch3.com/posts/experiments_with_shfl/I wanted to double-check my understanding of Kepler’s shfl operation when using negative indices or negative index offsets. The PTX documentation on this instruction is accurate but a little terse so a micro-test was in order. The summary results are: shfl.idx handles negative indices without any problem which means “shuffle rotations” are feasible. shfl.up has no chance to mask the lane - bval value so it sets the in-range predicate to false when negative indices are produced and the lane’s current value is assigned.https://dispatch3.com/posts/gpu_hack_pre_fermi/Sun, 27 May 2012 00:00:00 -0400https://dispatch3.com/posts/gpu_hack_pre_fermi/I’ve been cleaning up a set of kernels so that they will run optimally on GT200 devices (sm_1x). The kernels run extremely well on Fermi so I was disappointed when the opencc compiler struggled to use a reasonable number of registers despite having access to the same number of registers per thread. I was getting over 200 bytes of spills in a critical kernel that had no spills at all on Fermi.https://dispatch3.com/posts/why_warpsize/Thu, 19 Apr 2012 00:00:00 -0400https://dispatch3.com/posts/why_warpsize/In CUDA C, the built-in variable warpSize is initially treated as a variable at compile-time and doesn’t appear to be recognized as a constant until the PTX generation phase. This could be an issue if the warp width is part of some tricky preprocessing early in the compilation. The simple line: const unsigned int w99 = warpSize * 99;` is resolved to the following PTX: mov.u32 %r5, WARP_SZ; mul.lo.s32 %r6, %r5, 99; Yes, the stage after PTX will do a great job folding/propagating away WARP_SZ but sometimes you need to resolve logic in the preprocessor.https://dispatch3.com/posts/gpu_hack_high_lane_wins/Wed, 27 Jul 2011 00:00:00 -0400https://dispatch3.com/posts/gpu_hack_high_lane_wins/A useful GF100 GPU hack is revealed on page 6 in the Laine & Karras' paper “High-Performance Software Rasterization on GPUs”. They state that: When there are shared memory write conflicts within the warp, the write from a thread on a higher lane, therefore containing a later triangle, will override a write from a thread on a lower lane, containing an earlier triangle. The CUDA programming guide explicitly leaves it undefined which thread will succeed in the write, but at least on GF100 the behavior is consistent and can be exploited.