Efficient Gastointestinal Disease Classification using Explainable Knowledge Distillation from ECA-NFNet-L0 to MobileNetV3

Categorizing gastrointestinal diseases is crucial, as it challenges in the global world. Traditional inspection-based methods are tedious and error-prone, as visual inspections of these diseases often closely assimilate one another. With a view to this, Deep Learning (DL) models may provide an effective solution. In this work, we primarily selected the pretrained ECA-NFNet-L0, a state-of-the-art (SOTA) Convolutional Neural Network (CNN), and fine-tuned it on the gastrovision dataset, achieving 83.62\%,76.92\%,81.62\% MCC in lower GI, higher GI, and both GI disease classification. The fine-tuned ECA-NFNet-L0 was employed as the teacher model in a Knowledge Distillation (KD) setup to train the lightweight MobileNetV3Small model with $\approx 2.9M$ parameters.To interpret the model’s decision, we employed several model-specific and model diagnostic Explainable AI (XAI) approaches. Therefore, through KD, we achieved better performance from the lightweight MobileNetV3Small, which may be deployed in edge devices.