By Fabian Sinzinger¹, Youssef Karout¹, Josselin Manceau¹, Marie Arrivat¹², Rémy Peyret¹
(1 Primaa, Paris, 2 LTCI, Télécom Paris, Institut Polytechnique de Paris)

Context

Problem: Multi-stage pathology pipelines often optimize patch segmentation and slide classification separately. Segmentation models see local information, while classifiers rely on coarse slide-level context.

Proposed: We combine both tasks in a differentiable framework. Patch segmentation outputs are aggregated into a slide-level heatmap, allowing classification loss to propagate back into the segmentation model.

Method

Fig. 1: Overview of the proposed training pipeline.

• Inner loop: patch mini-batches are processed by the segmentation model.
• Patch outputs are accumulated and down-sampled into a slide-level map.
• Binary Classifier predicts carcinoma status from the aggregated representation.
• Multi-class Segmentation predicts heat-maps of different carcinoma sub-types (Invasive/in-situ)
• Segmentation and classification objectives can be trained separately or jointly.
• Checkpointing recomputes patch forwards during backpropagation.

Dataset

• Training: MST26 train, 3 centers n=392 (337 carc. / 55 healthy)
• Validation: MST26 val, 3 centers n=86 (72 carc. / 14 healthy)
• Testing: Held-out single-center set n=128 (64 carc. / 64 healthy)

Preliminary Results

A) Ablation: combined loss weight

Fig. 2: fixed number (32) of finetuning epochs with varying classification loss weight.

B) Performance metrics

Tab. 1: Best validation checkpoints selected by AUROC after convergence.

Tab. 2: External full-slide evaluation.

C) Qualitative demonstration: segmentation heatmaps

Conclusion

• We propose a differentiable online aggregation framework linking patch segmentation and whole-slide carcinoma classification.
• Joint training with low classification-loss weight improved online validation AUROC while largely preserving segmentation quality.
• External full-slide evaluation showed higher sensitivity but lower AUROC for joint models; training on patches from complete slides is expected to reduce this validation – test gap.

Limitations

• Classification performance is affected by class imbalance.
• Current aggregation does not explicitly model overlap between neighboring patches.
• Proof of concept models were trained on partially annotated slides.

REMARK: The original abstract demonstrated the framework on skin cancer data. In this poster, we apply the same methodology to a breast carcinoma cohort.

Future Work

• Train on larger public datasets and benchmark tasks from different histopathology problems.
• Compare against MIL and attention-based slide classifiers.
• Improve aggregation with overlap-aware or attention-based pooling.
• Train models from scratch within the same framework.
• Multi class classification