🆕 Add SegFormer Architecture - #1085
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## develop #1085 +/- ##
=========================================
Coverage 99.88% 99.89%
=========================================
Files 86 88 +2
Lines 11670 11969 +299
Branches 1533 1550 +17
=========================================
+ Hits 11657 11956 +299
Misses 7 7
Partials 6 6 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
Pull request overview
This PR introduces a new transformer-based semantic segmentation architecture (SegFormer) backed by a Mix Transformer encoder, while refactoring existing segmentation architectures to reuse shared utility layers (e.g., Conv2dReLU, SegmentationHead, encoder utilities).
Changes:
- Add new SegFormer model implementation and a Mix Transformer (MiT) encoder backbone.
- Centralize shared encoder and convolution utilities in
architecture.utilsand remove duplicated implementations. - Add/adjust unit tests for the new architecture and for the moved utility functions.
Reviewed changes
Copilot reviewed 10 out of 10 changed files in this pull request and generated 13 comments.
Show a summary per file
| File | Description |
|---|---|
tiatoolbox/models/architecture/utils.py |
Adds shared building blocks (Conv2dReLU, EncoderMixin, dilation/patching utilities) used across architectures. |
tiatoolbox/models/architecture/timm_efficientnet.py |
Refactors EfficientNet encoder to import shared encoder mixin utilities instead of local copies. |
tiatoolbox/models/architecture/segformer.py |
Adds SegFormer model + decoder and inference/pre/post-processing helpers. |
tiatoolbox/models/architecture/mix_transformer.py |
Adds Mix Transformer backbone/encoder implementation used by SegFormer. |
tiatoolbox/models/architecture/grandqc.py |
Replaces local Conv2dReLU with the shared implementation from architecture.utils. |
tiatoolbox/models/architecture/efficientunet_tissue_mask_model.py |
Refactors to use shared Conv2dReLU and SegmentationHead from architecture.utils. |
tests/models/test_arch_utils.py |
Adds tests for moved utilities (patch_first_conv, replace_strides_with_dilation). |
tests/models/test_arch_timm_efficientnet.py |
Removes tests that moved to test_arch_utils.py. |
tests/models/test_arch_segformer.py |
Adds unit tests covering SegFormer model, decoder behavior, and preprocessing/postprocessing/inference. |
tests/models/test_arch_mix_transformer.py |
Adds unit tests for Mix Transformer backbone + encoder behaviors. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| # Licensed under the NVIDIA Source Code License. For full license | ||
| # terms, please refer to the LICENSE file provided with this code | ||
| # or visit NVIDIA's official repository at | ||
| # https://github.com/NVlabs/SegFormer/tree/master. |
There was a problem hiding this comment.
The license is not compatible with TIAToolbox. Can we only include code which is BSD-3 compatible? Or clone the repo when it's requested with appropriate license displayed?
shaneahmed
left a comment
There was a problem hiding this comment.
Thanks @Jiaqi-Lv I am only concerned about the licence terms. These are not compatible with TIAToolbox licence. Otherwise the PR looks good.
|
I'll look into using |
This pull request adds SegFormer segmentation model, and refactors the
EfficientUNetTissueMaskModelto use shared utility modules for common layers.New architectures for semantic segmentation
SegFormer
Mix Transformer (Encoder)
Refactoring for code reuse
EfficientUNetTissueMaskModelto use the sharedConv2dReLUandSegmentationHeadimplementations fromarchitecture.utils, replacing local definitions and reducing code duplication. [1] [2] [3] [4] [5]