Thesis title: Development of drift-adaptive machine learning methods to support health decision making in dynamic data contexts

Abstract: The deployment of conventional artificial intelligence (AI) systems in healthcare is often challenged by the phenomenon of data shift, referring to the deviation between the data distributions used for training and those encountered during real-world deployment. Data shifts may arise from temporal dynamics or from changes in data acquisition conditions, leading to reduced accuracy, generalization, and ultimately reliability of AI models. This issue is of particular importance in healthcare, where data are inherently heterogeneous and continuously evolving. The need to address data shifts has given rise to the research field of resilient AI, which focuses on developing systems capable of detecting and adapting to distributional changes while maintaining high performance and robustness. Within this context, the present thesis investigates two fundamental forms of data shift frequently observed in healthcare: temporal shifts in data streams and acquisition shifts in multi-cohort data.

Temporal shift occurs in environments where data are collected continuously under conditions that may vary over time. Dynamic external factors, such as changes in recording devices, demographic variations among user subgroups, or even the natural evolution of the phenomena being monitored (e.g., the evolution of a virus during a pandemic), lead to statistical distributional changes and, consequently, model performance degradation. This thesis investigates this form of shift through the detection of COVID-19 using crowdsourced cough audio recordings, employing the COVID-19 Sounds and COSWARA datasets. A comprehensive, drift monitoring and adaptive framework for data streams is developed, utilizing the Maximum Mean Discrepancy (MMD) metric to detect distributional changes and trigger model adaptation mechanisms when a shift is identified.

The datasets were temporally divided into development and deployment periods. A baseline Convolutional Neural Network (CNN) was trained on development data, achieving AUC values of 69.13% and 66.8%, and balanced accuracy of 63.38% and 61.64% for the COVID-19 Sounds and COSWARA datasets, respectively. Evaluation on deployment data revealed a significant performance decline, confirming the presence of temporal drift. Two model adaptation strategies were employed, unsupervised domain adaptation (UDA) and active learning (AL). Applying UDA improved balanced accuracy by up to 22% and 24% for the two datasets, while AL yielded even greater improvements of up to 30% and 60%, respectively. The obtained results demonstrate that integrating drift detection with adaptive learning significantly enhances the robustness of COVID-19 detection models, enabling them to maintain high performance even in the presence of severe distribution shifts.

In multi-cohort data, data shifts often arise due to variations in equipment, acquisition protocols, and population demographics. Such acquisition-based variations affect model generalization, as training and deployment occur across data from different sources, requiring robust domain adaptation techniques to ensure consistent performance across diverse clinical settings. This dissertation addresses the problem through the risk stratification of cardiovascular disease using carotid ultrasound imaging. First, an interpretable ensemble of three deep learning models was developed to classify symptomatic and asymptomatic atherosclerotic cases using data from the General University Hospital “Attikon”, consisting of 96 ultrasound images. The model is consisted of three independently trained CNNs. To handle class imbalance, subsampling of training sets, a two-phase training strategy, and cost-sensitive weighting were employed. The final ensemble achieved an AUC of 73%, sensitivity of 75%, and specificity of 70%, demonstrating adequate discriminative performance. Interpretability methods were applied to the model’s predictions, highlighting anatomical regions features associated with increased cardiovascular risk and providing novel insights into the clinical understanding of the phenomenon.

Subsequently, the quantification of domain shift across different imaging centers was investigated. A CNN model trained on data from “Attikon” was evaluated on the Carotid Ultrasound Boundary Study (CUBS) dataset comprising 689 samples. Model performance declined from AUC 90.1% and accuracy 89.7% on the source dataset to AUC 45.6% and accuracy 34.3% on CUBS, confirming the significant effect of data shift. Distance metrics such as Hellinger distance, Kullback–Leibler (KL) divergence, and MMD were computed between the feature distributions of the two datasets across multiple representation depths. The analysis revealed strong negative correlations between the quantified degree of shift and model accuracy (e.g., ρ = –0.75 for Hellinger distance), providing practical tools for assessing and predicting model generalization under domain shift conditions.

To holistically address these challenges, an interpretable active learning framework is proposed, integrating uncertainty estimation via Monte Carlo Dropout, a representativeness criterion based on MMD, and pseudo-labeling for efficient model retraining and adaptation. The framework achieved 94.2% of the optimal performance using only 28.2% of the data, while interpretability techniques such as Grad-CAM and LIME provided transparency in both model predictions and uncertainty estimates, reinforcing AI reliability in real clinical settings.

Overall, this dissertation makes a significant contribution to the understanding and mitigation of data shift in AI-based healthcare applications. By developing frameworks for the detection, adaptation, and interpretation of models under dynamic data conditions, it advances the creation of robust, efficient, and interpretable AI systems capable of maintaining reliable performance in real-world, evolving clinical environments.

Supervisor: Konstantina S. Nikita

PhD Student: Theofanis Ganitidis