Uncertainty Modelling for Open Domain Generalization

Open domain generalization requires a model to perform on domains unseen during training, where both the input distribution and the label space may differ. This thesis studies how predictive uncertainty should be modeled and used in that setting — so a model can express calibrated doubt about the inputs and classes it was never trained on, and defer rather than fail silently.

Problem

Conventional domain generalization assumes a shared label space and related source and target domains. Open domain generalization removes both assumptions: the model may meet new domains and new classes at test time, where confident but wrong predictions are the core failure mode.

Approach

Characterized the behavior of entropy-, dropout-, and ensemble-based uncertainty across diverse natural datasets, modeling each as a function of noise, out-of-distribution samples, and unknown labels. Benchmarked across augmentation-based, generative, meta-learning, and ensemble domain-generalization models, and designed an uncertainty-based decision framework for domain-shift-aware open-set prediction, with digital pathology as a motivating application.

Results

Revealed distinct uncertainty signatures separating in-distribution samples, out-of-distribution samples, and unknown classes, and established a systematic framework for evaluating uncertainty under diverse distribution shifts across models and datasets.