Skip to content

feat(cudnn): add batch normalization forward/backward - #405

Open
Rachit2323 wants to merge 1 commit into
Rust-GPU:mainfrom
Rachit2323:feat/cudnn-batch-norm
Open

feat(cudnn): add batch normalization forward/backward#405
Rachit2323 wants to merge 1 commit into
Rust-GPU:mainfrom
Rachit2323:feat/cudnn-batch-norm

Conversation

@Rachit2323

Copy link
Copy Markdown

Summary

Batch normalization wasn't wrapped in the cudnn crate yet (it's marked as not done in features.md), so I added it. It's a pretty common thing to need for neural nets, so figured it'd be useful to have.

I followed the same pattern as the existing softmax module to keep things consistent.

Changes

  • Added a new batchnorm module with these methods on CudnnContext:
    • derive_batch_norm_descriptor
    • batch_norm_forward_inference
    • batch_norm_forward_training
    • batch_norm_backward
  • Added a BatchNormMode enum (per-activation, spatial, spatial persistent)
  • Added a SupportedBatchNorm trait for f32 and f64
  • Added a small TensorDescriptor::from_raw helper (pub(crate)) so the derived
    descriptor can be wrapped
  • Marked batch norm as done in features.md

Testing

  • cargo clippy -p cudnn passes
  • cargo test -p cudnn passes
  • Wrote a test that runs forward inference, forward training and backward on the GPU and checks the output is correct
  • Ran it on Ubuntu 24.04, RTX 6000 Ada, CUDA 13 / cuDNN 9

@Rachit2323
Rachit2323 requested a review from frjnn as a code owner July 28, 2026 12:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants