CUDA Matrix Multiplication
CUDA matrix-multiply kernels using shared-memory tiling, register blocking, and vectorised loads, compared against cuBLAS.
Matrix multiplication
I started with a naive matrix-multiply kernel and worked through coalesced memory access, shared-memory tiling, register blocking, and vectorised loads. Each change affects more than one constraint: using more registers can reduce memory traffic while limiting the number of active warps.
I use Nsight Compute to inspect occupancy, memory throughput, warp stalls, and register pressure. I check results against cuBLAS on non-power-of-two shapes before comparing performance. This page does not yet include a benchmark table with hardware and run conditions, so it makes no speedup claim.
Attention
The attention kernel tiles Q, K, and V through GPU memory and uses online softmax rescaling to avoid materialising the full attention matrix. I exposed it as a PyTorch custom operator and checked its output against scaled dot-product attention.