Challenge questions
These are not essential for learning the material and can be skipped without affecting your grade. If you successfully solve one set of problem, a week of participation activity will be waived (it does not have to be the same week you submit the challenge question). Submit your answer at any time. I will not post solutions for the challenge questions.
In this question we frame the Metropolis–Hastings random-walk algorithm as an involution on an expanded space. Recall that the standard random-walk MH algorithm for sampling from \(\pi(x) \propto \gamma(x)\), if we are currently at state \(X^{(m-1)}\), proceeds as follows:
- Sample \(\tilde X^{(m)} \sim \mathcal{N}(X^{(m-1)}, \sigma^2)\).
- Draw \(U \sim {\mathrm{Unif}}(0,1)\) and compute \[\alpha = \min\left\{1, \frac{\gamma(\tilde X^{(m)})}{\gamma(X^{(m-1)})}\right\}.\]
- If \(U \leq \alpha\), set \(X^{(m)} = \tilde X^{(m)}\). Otherwise, set \(X^{(m)} = X^{(m-1)}\).
To frame it as an involution, consider the expanded target \(\bar\pi(x,z)\), with an additional auxiliary variable \(z\), where \[ \bar \gamma(x, z) \propto \gamma(x) \cdot \frac{1}{\sqrt{2\pi}} \exp\left(-\frac{1}{2} z^2\right).\] That is, the \(x\)-marginal of \(\gamma(x,z)\) still corresponds to \(\pi(x)\), and the \(z\)-marginal corresponds to a standard normal distribution. Consider the following algorithm:
- Sample \(Z^{(m)} \sim \mathcal{N}(0, 1)\).
- Set \((\tilde X^{(m)}, \tilde Z^{(m)}) = T_\sigma(X^{(m-1)}, Z^{(m)})\)
- Draw \(U \sim {\mathrm{Unif}}(0,1)\) and compute \[\alpha = \min\left\{1, \frac{\gamma(\tilde X^{(m)}, \tilde Z^{(m)})}{\gamma(X^{(m-1)}, \tilde Z^{(m)})}\right\}.\]
- If \(U \leq \alpha\), set \(X^{(m)} = \tilde X^{(m)}\). Otherwise, set \(X^{(m)} = X^{(m-1)}\).
Specify \(T_\sigma: \mathbb{R}\times \mathbb{R}\to \mathbb{R}\times \mathbb{R}\) and prove that it is an involution. It should be chosen such that the random-walk behaviour on the \(x\)-space is preserved.
What is the determinant of the Jacobian of the transformation \(T_\sigma(x,z)\)?