MCMC Hacking
Exercises
Bayes1@UBC
Syllabus
Schedule
Final project
Introduction
Why?
What?
High-level picture
Examples
A bit of history
Probability essentials
Sample space, outcomes, events
Axioms of probability
Random variables
Probability mass functions
Decision trees
Forward sampling
Conditioning
Chain rule
Bayes rule
Expectations
Simple Monte Carlo
Exercises
Bayes on a discrete model
The Bayesian recipe
Bayesian models
Posterior distributions
Point estimates
Credible sets
Decision theory
Prediction
Intro to model criticism
Exercises
Challenge
A first look at PPLs
What is a PPL?
Importance sampling
Continuous models
SNIS consistency
Monte Carlo convergence rate
SNIS Effective Sample Size
Exercises
Challenge
The joy of probabilistic modelling
simPPLe
Bivariate posteriors
Normal distributions
Bernoulli regression
(Normal) regression
Bayesian GLMs
Exercises
Challenge
Some theory
Asymptotics
Optimality
Decision theoretic point estimation
Decision theoretic set estimation
Calibration
Bayesian calibration: well-specified case
Bayesian calibration: mis-specified case
Exercises
Challenge
Hierarchical models
Prior choice
Graphical models
Intro to hierarchical models
Exercises
Challenge
Quiz 1
Terminology review
Practice questions
Logistics
Intro to MCMC
Installing and running Stan
Stan basics
Stan: going further
Metropolis-Hastings
Consistency of MCMC
Stan: hands on
Exercises
Bayesian workflow
Overview
Goodness of fit
Checking correctness
MCMC diagnostics
Effective sample size
Grouped data
Exercises
Modelling techniques
Bayesian Lego
More Bayesian bricks
Custom bricks
Censoring
Data collection mechanisms
Rao-Blackwellization
Mixtures
Quiz 2
Terminology review continued
Practice questions (Quiz 2)
Logistics
MCMC Hacking
Overview
MH as a Markov chain
Law of large numbers for Markov chain
Invariance: intuition
Marginals of a Markov chain
Balance equations
MH is invariant
Kernel mixtures
Kernel alternations
MCMC debugging
Why HMC?
HMC basics
Deterministic proposals
HMC as involution
Autodiff
Exercises
Advanced inference
Overview
Variational methods
The KL divergence
Optimizing the KL
Stochastic gradient descent
Stochastic gradient estimation
Exercises
MCMC Hacking
Exercises
Exercise 9: MCMC Hacking
Caution
Page under construction: information on this page may change.