https://dispatch3.com/tags/cuda/Recent content in CUDA on Dispatch3 Inc.Hugo -- gohugo.ioen-usThu, 03 Mar 2016 00:00:00 -0500https://dispatch3.com/posts/cuda_macro/Thu, 03 Mar 2016 00:00:00 -0500https://dispatch3.com/posts/cuda_macro/I just switched over to a new CUDA error checking macro and really like how it works. This error checking macro will transform a CUDA Runtime API function into its error-checked equivalent simply by parenthesizing everything after the lower case cuda function prefix. Examples: cuda(GetDeviceProperties(&props,device)); cuda(SetDevice(device)); cuda(Malloc(&vin_d, bytes)); cuda(Memset(vin_d,0,bytes)); cuda(Free(vin_d)); cuda(EventCreate(&end)); cuda(EventRecord(end)); cuda(EventSynchronize(end)); cuda(EventElapsedTime(&elapsed,start,end)); cuda(EventDestroy(end)); cuda(DeviceReset()); The C99/C++ macro and assert look like this: #define cuda(...) cuda_assert((cuda##__VA_ARGS__), __FILE__, __LINE__, true); cudaError_t cuda_assert(const cudaError_t code, const char* const file, const int line, const bool abort) { if (code !https://dispatch3.com/posts/hotsort_2_cuda/Tue, 23 Feb 2016 00:00:00 -0500https://dispatch3.com/posts/hotsort_2_cuda/I had some major spilling issues with CUDA 7.0 and my HotSort sorting library due to some NVCC bugs. CUDA 7.5 didn’t resolve the spills and I couldn’t wait any longer… it was unfortunately time for a heavy rewrite. Some of the work was done months ago but it took the last few weeks to get the higher-level kernels completed. I also managed to generalize the implementation so it can run on architectures that aren’t as programmer-friendly as CUDA multiprocessors.https://dispatch3.com/posts/hotsort_2/Thu, 07 May 2015 00:00:00 -0400https://dispatch3.com/posts/hotsort_2/I hadn’t touched my HotSort codebase in a long time but recently was able to spend a couple weeks developing a generalized version of the sorting algorithm that can be “tuned” for a specific GPU architecture. After more than a couple years of thinking about the algorithm, I had accumulated a number of performance and portability ideas. The most important idea generalized the HotSort sorting algorithm so that it could run well on some of the newer resource-rich but architecturally different “shaped” OpenCL GPUs.https://dispatch3.com/posts/hotsort_tk1/Fri, 30 May 2014 00:00:00 -0400https://dispatch3.com/posts/hotsort_tk1/The ~5 Watt Jetson TK1 can sort 16K 32-bit keys in 0.07 msecs or ~220 Mkeys/sec! You can see the TK1 results for sorting 32-bit, 32+32 and 64-bit keys here. I’ve also added these results to the end of the HotSort benchmark PDF. My expectation was that the K1 would be able to sort 16K keys in ~0.05 msecs. instead of ~0.07 so I’m curious if there are architectural differences other than the reduced size register file.https://dispatch3.com/posts/hotsort_gtx750ti/Fri, 21 Feb 2014 00:00:00 -0500https://dispatch3.com/posts/hotsort_gtx750ti/I’ve added GTX 750 Ti results to the HotSort benchmark PDF. I am pleased to see that the entry-level 60W 750 Ti performs well against the once flagship GTX 480 up until ~512K 32-bit keys. You can see a direct comparison of the two cards here when sorting 32-bit and 64-bit keys. Note that the GTX 480 still has more than twice the aggregate shared memory as well as 40% more 32-bit registers than the GTX 750 Ti.https://dispatch3.com/posts/hotsort_gk208/Thu, 08 Aug 2013 00:00:00 -0400https://dispatch3.com/posts/hotsort_gk208/NVIDIA’s new GK208 GPU is a low power device with very high end sm_35 compute capabilities. At this point, only the flagship GTX 780, GTX TITAN, TESLA K20 and Quadro K6000 GPUs support sm_35. It’s also rumored that Tegra 5 (“Logan”) will be an sm_35 compute capability device. My guess is that it will run at ~300 MHZ, have 192 cores (1 SMX) and 12.8 GB/s of bandwidth (64-bit LPDDR3@800MHz).https://dispatch3.com/posts/smem_idiom/Thu, 18 Jul 2013 00:00:00 -0400https://dispatch3.com/posts/smem_idiom/It’s common to declare a number of shared memory variables at the top of a CUDA kernel file. You often see these declarations with Hungarian’ish names that imply the variables somehow reside in shared memory. But as someone who likes to write highly readable code, I’ve found that naming, referencing and maintaining independently declared shared variables can sometimes turn into an endeavor. Some example issues: I would like to use the same meaningful variable name for both the shared and in-register instances.https://dispatch3.com/posts/hotsort_oc_k20c/Mon, 20 May 2013 00:00:00 -0400https://dispatch3.com/posts/hotsort_oc_k20c/A mysterious stranger just ran the HotSort benchmarking utility on an overclocked Tesla K20c and sent me the results. The K20c was overclocked to 1058 MHz x 6.0 Gbps (GPU x MEM). A stock K20c normally runs at 758 MHz x 5.2 Gbps. The results are stunning! You can see all the benchmarks here.https://dispatch3.com/posts/hotsort_gk110/Sun, 19 May 2013 00:00:00 -0400https://dispatch3.com/posts/hotsort_gk110/Last week I returned to working on HotSort in order to add a few new features. One of the “free” features on my list was to implement high register-count merge kernels on GK110 and GT200 architectures. The merge kernels in HotSort minimize global loads and stores by maximizing the number of element comparisons performed per thread. Up until now, the same merging algorithm and register configurations were being used across all CUDA architectures and the resulting merge kernels were approaching the Fermi and GK104 63 register-per-thread limit.https://dispatch3.com/posts/kernel_args_vs_constants/Thu, 09 May 2013 00:00:00 -0400https://dispatch3.com/posts/kernel_args_vs_constants/It’s not uncommon to have a GPU kernel where a number of the kernel parameters are left as constants throughout the entire call chain and, if the target were a CPU, would have been implemented as traditional static const file scope variables. For this reason, you might think that exploiting the __constant__ qualifier and its associated cudaXXXSymbol() routines is a good way of reducing the number of kernel arguments and generally cleaning up your code.https://dispatch3.com/posts/fast_matrix_transpo_kepler_2/Sun, 07 Apr 2013 00:00:00 -0400https://dispatch3.com/posts/fast_matrix_transpo_kepler_2/Last time I presented a clever way of transposing that exploited the GPU’s support of 32-byte stores as its smallest “100% efficient” transaction size. An open question was whether designing a transpose that performed 64-byte stores would achieve better performance at the cost of more instructions. The answer is yes. Performing 64-byte transactions improves throughput by ~8%. The new 64-byte transpose kernel reaches ~148 GB/sec. on a 1024x1024 matrix of 32-bit elements on a K20c (758MHz).https://dispatch3.com/posts/fast_matrix_transpo_kepler/Tue, 26 Mar 2013 00:00:00 -0400https://dispatch3.com/posts/fast_matrix_transpo_kepler/I needed a fast and minimal routine that could transpose a warp with 32 elements per lane and store the result to device memory as the final step in a complex kernel. However, I had a hypothesis that Kepler SHFL instructions and 16-byte stores by pairs of lanes would be functionally equivalent to the standard matrix transpose approach but not require any shared memory or thread block synchronization yet still achieve 100% memory store efficiency.https://dispatch3.com/posts/experiments_with_shfl/Thu, 14 Mar 2013 00:00:00 -0400https://dispatch3.com/posts/experiments_with_shfl/I wanted to double-check my understanding of Kepler’s shfl operation when using negative indices or negative index offsets. The PTX documentation on this instruction is accurate but a little terse so a micro-test was in order. The summary results are: shfl.idx handles negative indices without any problem which means “shuffle rotations” are feasible. shfl.up has no chance to mask the lane - bval value so it sets the in-range predicate to false when negative indices are produced and the lane’s current value is assigned.https://dispatch3.com/posts/hotsort_3232_64/Fri, 19 Oct 2012 00:00:00 -0400https://dispatch3.com/posts/hotsort_3232_64/HotSort has been updated to support 32:32 key-val and 64-bit keys. The results are very good. 1 When sorting 64-bit keys, Kepler achieves ~49% of the throughput of the 32-bit key benchmarks. The wider comparison sort performs twice the number of SASS comparisons and triple the number of calls to __syncthreads() on types that are twice as wide so getting half the throughput is excellent. Additional optimizations were made in the past few weeks and there is now a general performance improvement across all architectures: approximately 12% on GT200 and almost 5% on Kepler.https://dispatch3.com/posts/hotsort/Tue, 04 Sep 2012 00:00:00 -0400https://dispatch3.com/posts/hotsort/Earlier this year I determined I was going to need a specialized sorting algorithm in order to complete another GPU project. I needed a sorter that was portable, in-place, supported key-vals wider than 32-bits and could sort binned (tiled) independent data sets output by other GPU kernels. But most of all, the sorting algorithm had to be uber-fast on small GPUs. A number of months later HotSort was completed. I’ve achieved most of my design goals and exceeded a few of them.https://dispatch3.com/posts/cuda_ion2_benchmarks/Thu, 30 Sep 2010 00:00:00 -0400https://dispatch3.com/posts/cuda_ion2_benchmarks/I needed to test my CUDA application on a low-end configuration so I plunked down $210 and bought a Jetway Mini-TOP D525+ION2 computer. Below are some basic CUDA benchmark results. First, some relevant specifications on this machine: Dual-core Atom D525 @ 1.8GHz Intel NM10 chipset NVIDIA ION2 GPU (GT218) with 512MB of DDR3 DDR2 667/800 SO-DIMMs on a 64-bit bus RealTek Gigabit PCIe Ethernet port There are a number of netbooks and nettops that have nearly identical specifications.https://dispatch3.com/posts/gpu_to_cpu/Mon, 09 Aug 2010 00:00:00 -0400https://dispatch3.com/posts/gpu_to_cpu/Over on the NVIDIA CUDA Computing forum I saw that Dell is now shipping a 16 GPU PCIe Expansion Chassis. If you dig around a little you’ll find a great video by the chassis architect that starts by describing the impetus for the product. I thought it was really interesting that when an oil and gas customer came to Dell and asked for a chassis solution for GPUs, their “GPU-to-server” ratio requirement went from 2:1 in the beginning all the way up to 4:1 (4 GPUs per server).https://dispatch3.com/posts/gpus_and_wall_street/Tue, 27 Apr 2010 00:00:00 -0400https://dispatch3.com/posts/gpus_and_wall_street/The WSJ just published this story: Trading Firms Turn To Videogame Chips To Get Even Faster. The article primarily focuses on FPGAs. Nvidia is mentioned as well as OpenCL. Another company that uses FPGA’s to process market data is Exegy. I suspect that GPUs are being looked at very closely by Wall Street but one challenge that early adopters need to overcome is architecting a solution that gets away from the simple but latent “kernel launch and wait” approach: