Backward pass CUDA kernels for fused GEMM+Bias+GeLU on SM75 (Turing). Float4 vectorized WMMA kernels validated against PyTorch autograd. 3.1x faster than autograd at M=1024. 24/24 tests passing.
deep-learning cuda pytorch transformer cuda-kernels turing gemm fp16 gpu-programming gelu wmma backward-pass sm75
-
Updated
Jul 27, 2026 - Python