Challenge questions

Not for grades!

These are not essential for learning the material and can be skipped without affecting your grade. If you successfully solve one set of problem, a week of participation activity will be waived (it does not have to be the same week you submit the challenge question). Submit your answer at any time. I will not post solutions for the challenge questions.

Consider the following regression model:

suppressPackageStartupMessages(require(rstan))

data {
  matrix[3,2] design_matrix; // number of successes
  vector[3] observations;
}

parameters {
  vector[2] coefficients;
}

model {
  coefficients[1] ~ cauchy(0, 1);
  coefficients[2] ~ cauchy(0, 1);
  
  for (i in 1:3) {
    observations[i] ~ normal(design_matrix[i] * coefficients, 1);
  }
}

We now run it on a simple synthetic dataset (note the zeros in the design matrix):

matrix = rbind(
  c(0.1, 0),
  c(-0.3, 0),
  c(0, 0)
)
obs = c(0.2, 0.1, -0.4)
fit = sampling(
  cauchy,
  seed = 1,
  refresh = 0,
  data = list(design_matrix = matrix, observations = obs),        
  iter = 10000                   
)

Warning: There were 71 transitions after warmup that exceeded the maximum treedepth. Increase max_treedepth above 10. See
https://mc-stan.org/misc/warnings.html#maximum-treedepth-exceeded

Warning: There were 1 chains where the estimated Bayesian Fraction of Missing Information was low. See
https://mc-stan.org/misc/warnings.html#bfmi-low

Warning: Examine the pairs() plot to diagnose sampling problems

…and look at the trace plots…

suppressPackageStartupMessages(require(bayesplot))
suppressPackageStartupMessages(require(ggplot2))
mcmc_trace(fit, pars = c("coefficients[1]", "coefficients[2]")) + theme_minimal()

What is going on with the second coefficient? Would you get a different behaviour if you used i.i.d. samples?