# Intro to model criticism

## Outline

### Topics

- What is a model mis-specification?
- What are the consequences?
- What to do about it.

### Rationale

Model mis-specification (the model being “too wrong”) can lead to serious problems in all types of statistical models, but Bayesian models are often more seriously affected. As a result it is important to detect serious cases of mis-specification and to address them. This is just an introduction, we will go in more depth once we have introduced Bayesian regression models.

## Definitions

**Model mis-specification:** when the model is “too wrong” (in the context of the famous quote by George Box).

**Model criticism:** the task of trying to find mis-specification.

- It can be done by reasoning and discussion with experts.
- It can be done using data (
**goodness-of-fit**)

### Example

**Recall our notation:** \(\rho\) prior PMF, \(\pi\) posterior PMF.

- Suppose we put zero prior mass to having a fair dice. What happens on the posterior?

- If we put prior mass of zero to some realization \(x\), say \(\rho(x) = 0\) then the posterior probability on \(x\) will always be zero, \(\pi(x) = 0\),
*no matter how many observations we get*- This could be disastrous! (e.g. ignoring extreme scenarios can have extreme consequences)

- Principle to avoid this is known as
**Cromwell’s rule**(Oliver Cromwell, 1650): “I beseech you, in the bowels of Christ, think it possible that you may be mistaken.”

**Discussion:** can you identify other issues with the model from Exercise 2?

## How to correct mis-specification

- Improve the model!
- Then iterate with more criticism and improve again if needed.
- Iterative model improvement is part of the current “best practice” in Bayesian analysis.