Compression-Aware Image Forensics (RGB-N Dual-Stream Fusion Network)

Detecting whether an image is authentic or tampered (e.g., splicing, copy-move) is a critical challenge. However, images shared online are often subjected to heavy JPEG compression, which destroys the subtle high-frequency artifacts (such as boundary inconsistencies or sensor noise) that forensic models typically rely on.

This project implements a robust binary image classifier (Authentic vs. Tampered) that maintains high detection metrics even under heavy JPEG compression. It utilizes a state-of-the-art RGB-N dual-stream fusion network (combining spatial RGB and high-pass SRM noise residuals) trained under a leak-proof group partition.

---

Key Architecture & Concept

1. Spatial Rich Model (SRM) Noise Residual Stream

Standard convolutional backbones pre-trained on ImageNet are biased towards semantic objects (shapes, colors). To force the network to look at forensic anomalies, we prepend a frozen SRM Layer applying three high-pass filter kernels:

• Horizontal first-order difference
• Vertical first-order difference
• Laplacian second-order difference

This isolates noise patterns (e.g., sensor pattern noise, JPEG compression grids) and filters out image semantic content.

2. RGB-N Channel Fusion

The network runs the input through the SRM layer and concatenates the output with the raw RGB input. This yields a 6-channel representation ($3 \text{ RGB} + 3 \text{ SRM residuals}$). The ResNet18 first conv layer (backbone.conv1) is modified to accept 6 channels.

Input Image (RGB) ────> RGB Stream ────────┐
                 ────> SRM Filter Layer ──> SRM Stream ──> [Concatenation] ──> 6-Channel conv1 ──> ResNet18

3. Leak-Proof Group Splitting

To prevent data leakage (which happens when derivatives of the same source image appear in both train and validation splits), we parse CASIA source image IDs from filenames using regex and find connected components. Images sharing source IDs are grouped together, ensuring 0% source ID overlap between train and validation splits.

4. Dynamic JPEG Compression Augmentation

During dataset loading, images are dynamically compressed in memory using a PIL byte stream under a random JPEG quality factor (Q=20 to 50) to teach the network features that survive quantization.

---

Benchmarking Results

Evaluated on the 600-image dataset (extracted from CASIA 2.0) partitioned via group-splits (Acc = Accuracy, AUC = Area under ROC curve):

JPEG Quality Level	Baseline Acc (%)	Baseline AUC	Robust Acc (%)	Robust AUC	AUC Difference
Clean (No Compression)	89.17%	0.8943	78.33%	0.8719	-0.0224
JPEG 90	89.17%	0.8875	79.17%	0.8685	-0.0190
JPEG 70	88.33%	0.9022	83.33%	0.8753	-0.0269
JPEG 50	88.33%	0.9102	80.83%	0.8822	-0.0280
JPEG 30	85.83%	0.8984	84.17%	0.8795	-0.0189
JPEG 10 (Heavy)	80.83%	0.8636	80.83%	0.8837	+0.0201

Key Findings

• Performance Boost: The combination of SRM filtering and dataset scale-up increased validation accuracy from ~50% (random guess) to 89.17% and AUC-ROC to 0.91.
• Robustness trend: Under heavy compression (JPEG 10), the Baseline model's performance decays to 0.8636 AUC, while the Robust model maintains 0.8837 AUC—successfully outperforming the baseline.

---

Project Structure

image-forensics/
│
├── archive/              # (Gitignored) Raw full CASIA 2.0 dataset
├── data/                 # (Gitignored) Extracted data subset
│   └── raw/
│       ├── images/
│       └── masks/
│
├── src/                  # Source code
│   ├── dataset.py        # PyTorch ForensicsDataset with dynamic JPEG augmentation
│   ├── train.py          # SRM Layer, RGB-N Classifier, Group Splitting & Benchmark Loop
│   └── gradcam.py        # Grad-CAM explainability pipeline
│
├── scripts/              # Helper scripts
│   ├── create_subset.py  # Path-aware subset extractor
│   └── debug_dataset.py  # Test dataset shapes and loader
│
├── models/               # (Gitignored) Saved model checkpoints
├── outputs/              # (Gitignored) Grad-CAM outputs and metrics comparison plots
├── requirements.txt      # Python dependencies
└── .gitignore            # Git ignore file

---

Installation & Usage

1. Setup Environment

pip install -r requirements.txt

2. Extract Dataset Subset

To extract 300 authentic and 300 tampered images from the raw CASIA archive:

python scripts/create_subset.py

3. Run Training and Benchmark Suite

To train both baseline and robust models and generate comparative metrics:

python src/train.py

Note: The script saves weights to models/baseline_model.pth and models/robust_model.pth, and outputs the comparison plot to outputs/evaluation_comparison.png.

4. Run Grad-CAM Inference

To generate a localized activation heatmap for a specific image:

python src/gradcam.py --image path/to/your/image.jpg --model models/robust_model.pth

By default, this saves the resulting heatmap overlay to outputs/gradcam/gradcam_output.png.