Adaptive self-supervised knowledge transfer for computationally efficient thoracic disease screening from chest X-ray images

Yongjun Ma, Minli Tang, Shi Dong, Haitao Zhang, Wengang Zhou

The development of accurate yet efficient deep learning models for Chest X-ray interpretation is critical for scalable screening and point-of-care diagnostics. While self-supervised learning reduces dependency on costly annotations and knowledge distillation compresses models, effectively integrating them to achieve a compact yet high-performing model remains a challenge. This paper introduces Ada-SKT v2, short for Adaptive Self-Supervised Knowledge Transfer version 2, a novel framework designed to address this gap by achieving three core objectives. First, effective utilization of unlabeled data through a tailored distillation paradigm. Second, the introduction of a hybrid multi-scale distillation loss that guides the student model to learn transferable, clinically salient anatomical representations by prioritizing feature alignment within critical regions identified via a gradient-based attention mechanism. Third, the production of a lightweight model viable for deployment under strict resource constraints. Evaluated on the public ChestX-ray2017 and ChestX-ray14 datasets under a constraint of 5.7 M parameters comprising 4.4 M backbone, 1.3 M projection, and 0.01 M classifier, and with only 20% labeled data, our distilled MobileViT-v2-S model achieves 90.06% accuracy on ChestX-ray2017 and 0.7612 mean AUC on ChestX-ray14. It outperforms MobileViT-v2-S trained from scratch on 20% labels at 0.7003 mean AUC and standard Global KD at 0.7524 mean AUC, while offering exclusive anatomical interpretability validated by saliency maps achieving a Coverage Ratio of 0.15 against expert-annotated pathological bounding boxes, 1.83 times higher than random baseline. Cross-dataset evaluation on CheXpert further confirms generalizability at 0.7843 mean AUC for 5 classes. Ablation studies confirm the significant contribution of each component in the hybrid distillation objective. This work provides a practical and efficient pathway towards deploying accurate, interpretable, AI-assisted screening tools in resource-constrained healthcare settings.