5  Model validation and comparison 🚧

Learning objectives

After reading this chapter, you should be able to:

• Apply graphical diagnostic methods to assess model fit quality
• Use information criteria (AIC, DIC, BIC) for quantitative model comparison
• Understand the relationship between model selection and predictive accuracy
• Recognize the subjective nature of model selection and make transparent choices

5.1 Essential model validation concepts

1. Graphical diagnostic methods and model checking
2. Information criteria for model comparison
3. Predictive accuracy and cross-validation
4. Model selection philosophy and transparency
5. Ensemble methods and model averaging

Further reading

• Piironen and Vehtari (2017): Technical treatment of predictive accuracy

Piironen, Juho, and Aki Vehtari. 2017. “Comparison of Bayesian Predictive Methods for Model Selection.” Statistics and Computing 27 (3): 711–35. https://doi.org/10.1007/s11222-016-9649-y.