ITDF - Implicit Topological De-Filtering

Makeup removal via topology-aware implicit neural representations

---

---

What is ITDF?

Makeup is not noise - it is a structured, aesthetically optimized corruption of the face manifold. Foundation merges skin micro-texture components, eyeliner redraws orbital boundary geometry, contouring introduces false gradient ridges, and lipstick obliterates the natural color gradient of the vermilion border.

Standard restoration networks fail here because they optimize:

$$\mathcal{L}_1 = |\hat{I} - I_{\text{bare}}|_1$$

This penalizes average pixel distance - encouraging blurred, topologically wrong outputs that look plausible but are structurally incorrect. A face can score high on PSNR yet have an eye whose boundary cycle is broken, a lip contour that is disconnected, or a pupil rendered as a filled disc instead of an annulus.

ITDF fixes this with topological supervision.

---

How it works

Step 1 - Encode globally with ViT

A Vision Transformer (ViT-B/16) encodes the makeup image into a spatial feature grid $Z \in \mathbb{R}^{h \times w \times 768}$. The global receptive field captures coordinated makeup regions - lips, eyes, and cheeks are modified together and must be understood together.

Step 2 - Decode implicitly with SIREN

For any continuous coordinate $\mathbf{x} = (x,y) \in [0,1]^2$, the bare-face color is predicted by a 5-layer SIREN MLP:

$$h^{(i+1)} = \sin!\big(\omega_0 (W^{(i)} h^{(i)} + b^{(i)})\big), \quad \omega_0 = 30$$

with Fourier positional encoding $\gamma(\mathbf{x})$ using $L=10$ frequency bands. The decoder queries any continuous coordinate - it does not operate on a fixed pixel grid, giving arbitrary-resolution output and natural handling of compression artifacts.

Step 3 - Enforce topology with DECT

For direction $\nu \in S^1$ and threshold $t$, the Euler Characteristic Curve is:

$$\chi_\nu(t) = \chi!\big({p \in K : \langle p,, \nu \rangle \leq t}\big)$$

The Euler Characteristic Transform collects all curves: $\text{ECT}(K) = {\chi_\nu}_{\nu \in S^1}$.

Key theorem (Turner et al.): ECT is injective - two shapes with identical ECTs are identical. Minimizing ECC differences achieves topological matching, not merely approximates it.

Standard ECCs are piecewise-constant and block gradients. We introduce a smooth sigmoid relaxation:

$$\tilde{\chi}_\nu(t) = \sum_{\sigma \in K} (-1)^{\dim(\sigma)} \cdot \sigma_\tau!\big(t - h_\nu(\sigma)\big), \quad \sigma_\tau(x) = \frac{1}{1+e^{-x/\tau}}$$

The DECT loss over $D=8$ directions and $T=64$ thresholds:

$$\mathcal{L}_{\text{DECT}} = \frac{1}{|D|\cdot T} \sum_{\nu \in D}\sum_{t=1}^{T} \Big(\tilde{\chi}_\nu^{(\hat{I})}(t) - \tilde{\chi}_\nu^{(I_{\text{bare}})}(t)\Big)^2$$

Complexity: $O(N \log N)$ - vs $O(N^3)$ for standard persistent homology. Temperature $\tau$ anneals from $1.0 \to 0.1$ over training so gradients sharpen progressively.

Step 4 - Disentangle artifacts with Multiparameter Persistence

Foundation (smooths gradients, uniform intensity shift) and lipstick (sharp gradient, saturated color) look similar under a single intensity filtration. We bifiltrate over intensity AND gradient magnitude simultaneously:

$$\mathcal{F}(\sigma) = \big(f_{\text{int}}(\sigma),; |\nabla f|(\sigma)\big)$$

Stochastic line slicing gives a differentiable Wasserstein loss via Sinkhorn optimal transport ($M=16$ slices per batch):

$$\mathcal{L}_{\text{multi}} = \frac{1}{M}\sum_{i=1}^{M} W_2!\big(\mathcal{B}_{\ell_i}^{(\hat{I})},, \mathcal{B}_{\ell_i}^{(I_{\text{bare}})}\big)$$

Total loss

$$\boxed{\mathcal{L}_{\text{total}} = 10,\mathcal{L}_1 + 2,\mathcal{L}_{\text{LPIPS}} + 0.1,\mathcal{L}_{\text{adv}} + 0.002,\mathcal{L}_{\text{DECT}} + 0.5,\mathcal{L}_{\text{multi}}}$$

---

Results

Qualitative: Makeup removal across 5 styles

Each block shows one identity: left - ground-truth bare face; top row - five distinct makeup inputs; bottom row - ITDF outputs. Across all five styles (natural, heavy eyeshadow, bold lip, contouring, full glam), the model consistently recovers the same bare-face structure - confirming that the many-to-one 1:5 training design successfully forces the network to learn identity topology, not cosmetic style.

---

ECC Topology: What the DECT loss achieves

Each panel shows Euler Characteristic Curves across one of the 9 filtration directions ($\nu_1 \ldots \nu_8$ spatial + intensity). Blue = makeup input. Orange = ITDF output. Purple fill = topology gap closed by training.

The orange curves consistently shift away from the blue (makeup) distribution and toward the bare-face pattern across all 9 directions. This is direct visual evidence that the DECT loss is active - it is not a passive regularizer but is genuinely reshaping the output topology. The gap is largest in the intensity direction (panel 9), where lipstick and eyeshadow create the strongest topological deviation.

---

ECC Gap: Per-direction analysis

This figure shows the mean integrated |ECC(input) − ECC(output)| across the validation set, separately for each of the 9 DECT directions.

The gap is uniform across all 8 spatial directions - there is no privileged orientation where the model performs better or worse. This confirms that the DECT loss provides balanced topological supervision across all filtration angles, not just along easy directions like horizontal/vertical. The intensity-only direction (Dir 9) shows a slightly larger gap, consistent with the observation that intensity-based artifacts (color shift from lipstick, foundation) are harder to fully resolve than spatial-gradient artifacts.

---

Makeup Intensity vs. Identity Similarity

Each point is one validation sample. X-axis: makeup intensity (mean absolute color distance between makeup and bare face). Y-axis: ArcFace identity similarity between ITDF output and ground-truth bare face.

Two key observations:

1. Light makeup (low intensity, left side) → consistently high identity similarity. The model has little to correct and does so accurately.
2. Heavy makeup (high intensity, right side) → identity similarity spreads wider and drops for extreme cases. This is expected - theatrical or full-face coverage removes the texture cues the model relies on.

The relationship is approximately linear in log-intensity, suggesting:

$$\text{ID Sim} \approx a - b \cdot \log(\text{makeup intensity})$$

with the model maintaining $> 0.6$ similarity for the large majority of the validation distribution.

---

Training Dynamics

Training loss curves over 40,000 iterations. The total loss (sum of all weighted components) decreases smoothly without instability. Key observations:

• L1 and LPIPS dominate the loss signal (58.7% and 40.6% of weighted total) and drive the initial rapid descent in the first 5,000 iterations.
• DECT raw loss decreases from 0.44 to 0.07 - an 84% reduction - confirming that the topological loss is not stuck and is genuinely being optimized throughout training.
• No loss spike or divergence occurs, validating the $\tau$-annealing schedule: starting at $\tau=1.0$ (smooth, stable gradients) and ending at $\tau=0.1$ (sharper, more precise topology matching).

---

Limitations

ITDF encounters difficulty in three scenarios:

1. Theatrical makeup (> 40% face coverage) - insufficient texture survives for reconstruction, outputs become blurred.
2. Non-studio lighting - strong side-lighting or dark backgrounds cause domain shift relative to FFHQ-Makeup training distribution.
3. Dataset annotation errors - several low-PSNR validation cases are caused by mismatched ground-truth pairs in FFHQ-Makeup (a makeup image from one identity paired with a bare face from a different individual), not by model failure. PSNR-based benchmarks underestimate true performance when ground-truth pair quality is imperfect.

---

Validation Metrics (FFHQ-Makeup, 14,696 pairs)

Metric	Train	Val
PSNR	25.88 dB	25.20 dB
SSIM	0.794	0.790
LPIPS	0.260	0.273
Identity Accuracy	98.2%	96.8%
Identity Similarity	0.682	0.660

Train/val gap is small - PSNR drops only 0.68 dB, identity accuracy drops 1.4 pp - confirming the model generalizes well across the identity split.

---

Live Demo

Upload a makeup face → get bare face + identity score + ECC topology plot. Runs on free CPU (~35–70 seconds).

---

File Structure

ITDF/
├── train.py                          # Training loop (AdamW, τ annealing, 40K iters)
├── evaluate.py                       # All 6 metrics (PSNR, SSIM, LPIPS, Betti, ArcFace)
├── requirements.txt
│
├── models/
│   ├── itdf.py                       # Full model (encoder + decoder)
│   ├── encoder.py                    # ViT-B/16 contextual encoder
│   ├── decoder.py                    # 5-layer SIREN implicit decoder
│   └── discriminator.py              # PatchGAN discriminator
│
├── losses/
│   ├── dect_loss.py                  # ← KEY: Differentiable ECT loss  O(NlogN)
│   ├── multiparameter_persistence.py # Bifiltration + Sinkhorn Wasserstein
│   └── combined_loss.py              # Weighted total loss
│
├── data/
│   ├── ffhq_makeup_dataset.py        # FFHQ-Makeup 1:5 pair construction
│   └── combined_dataset.py           # Dataloader
│
└── configs/
    └── default.yaml                  # All hyperparameters

---

Requirements

pip install -r requirements.txt

Hardware: Single NVIDIA GPU ≥12 GB VRAM (tested: RTX 4090, 24 GB).

---

Setup & Usage

1. Clone & install

git clone https://github.com/priyadip/itdf.git
cd itdf
pip install -r requirements.txt

2. Prepare data

# configs/default.yaml
data:
  ffhq_dir: "data/FFHQ-Makeup"
  img_size: 512

data/FFHQ-Makeup/
├── 000001/
│   ├── bare.jpg
│   ├── makeup_01.jpg  ...  makeup_05.jpg

3. Train

python train.py --config configs/default.yaml

4. Evaluate

python evaluate.py --config configs/default.yaml \
                   --checkpoint outputs/checkpoints/ckpt_final.pt

5. Single image inference

import torch, yaml, numpy as np
from PIL import Image
from models.itdf import ITDF

cfg   = yaml.safe_load(open("configs/default.yaml"))
model = ITDF(cfg)
ckpt  = torch.load("outputs/checkpoints/ckpt_final.pt", map_location="cpu")
model.load_state_dict(ckpt.get("model", ckpt), strict=False)
model.eval()

img = Image.open("makeup.jpg").convert("RGB").resize((512, 512))
x   = torch.from_numpy(np.array(img, dtype=np.float32) / 127.5 - 1.0).permute(2,0,1).unsqueeze(0)
with torch.no_grad():
    out = model(x)
arr = ((out.squeeze(0).permute(1,2,0).numpy() + 1.0) * 127.5).clip(0,255).astype("uint8")
Image.fromarray(arr).save("bare_face.png")

---

License & Security

Released under CC BY-NC-ND 4.0 - academic use only, no commercial use, no derivatives. See LICENSE and SECURITY.md for full terms and responsible use policy.

---

Model weights: private - request access on 🤗 Hugging Face