Challenge questions
These are not essential for learning the material and can be skipped without affecting your grade. If you successfully solve one set of problem, a week of participation activity will be waived (it does not have to be the same week you submit the challenge question). Submit your answer at any time. I will not post solutions for the challenge questions.
In this challenge question, we obtain an even more general result than the one presented in Q1 of Homework 10. For this question, recall that a family of distributions \(\{q_\phi : \phi \in \Phi\}\) is an exponential family if the density can be written in the form \[ q_\phi(x) = h(x) \exp(\eta(\phi)^\top T(x) - A(\eta(\phi))), \] where \(h\) is the base measure, \(\eta\) is the natural parameter, and \(T\) is the sufficient statistic. (Note that this is the fully general form, where \(x\) and \(\phi\) may be vectors.)
Show that \[ \nabla_\phi \, \operatorname{KL}(\pi \| q_\phi) = \nabla_\phi \eta(\phi) \cdot \left[\mathbb{E}_{q_\phi}[T(X)] - \mathbb{E}_\pi[T(X)]\right].\]
Hints: Note that \(A(\eta(\phi))\) acts as a normalizing constant for a given \(\phi\). Use the fact that \(\int q_\phi(x) \, dx = 1\) and \(\nabla \log \int f = \frac{\int \nabla f}{\int f}\) to show that \[ \nabla_\phi \, A(\eta(\phi)) = \nabla_\phi \eta(\phi) \cdot \mathbb{E}_{q_\phi}[T(X)].\]