ID Validation vs OOD Validation in Training

[Pen721]

Hi! Just wondering if there's a difference between how in domain validation and out of domain validation are used in training. I'm currently trying to train a model on one subdomain A in Camelyon17 (using 80% of A for training and 20% for validating) and test on another subdomain B, but running into a lot of errors in training when either ID validation or OOD validations are empty.

Thanks so much!

[ssagawa]

In the script in the examples folder, we evaluate on both ID validation and OOD validation in each epoch. But by default, we use OOD validation for hyperparameter tuning and early stopping and don't use ID validation.

[Pen721]

thanks so much!