| Date | Topic | Assigned Reading | Presenter | Additional Resources |
|---|---|---|---|---|
| 8/26 | Introduction + Course Overview |
Thirteen Rules for Giving a Really Bad Talk Efron, B. 2013 |
Jason Pacheco (slides) |
|
| 8/28 | Bayesian Inference |
Jason Pacheco (slides) |
Why Isn't Everyone a Bayesian? Efron, B. 1986 Objections to Bayesian Statistics Gelman, A. 2008 |
|
| 9/2 | Labor Day | No Class | |||
| 9/4 | Graphical Models | WJ - Chapter 2 |
Jason Pacheco (slides) |
JP - Sec. 2.1 ES - Sec. 2.2 |
| 9/9 | Exponential Families | WJ - Chapter 3 |
Jason Pacheco (slides) |
JP - Sec. 2.2 ES - Sec. 2.1 PRML - Sec. 2.4 |
| 9/11 | Variational Inference (VI) | PRML - Sec. 10.1-10.4 |
Jason Pacheco (notes) |
JP - Sec. 2.3 ES - Sec. 2.3 DB - Sec. 28.1-28.4 Variational Inference: A Review for Statisticians Blei, D., et al., J. Am. Stat. Assoc. 2017 |
| 9/16 | VI: Mean Field Variational Methods |
Variational Message Passing Winn, J. and Bishop, C. JMLR, 2005 |
Simon Swenson (notes) |
ES - Sec. 2.3.1 PRML - Sec. 10.1-10.6 WJ - Ch. 5 An Introduction to Variational Methods for Graphical Models Jordan, M. I., et al. Machine Learning, 1999 Tutorial Variational Bayes and Beyond: "Bayesian inference for big data" Broderick, T. 2019 |
| 9/18 | VI: Mean Field Example |
Latent Dirichlet Allocation Blei, D. M., et al. JMLR, 2003 |
Marium Yousuf (slides) |
|
| 9/23 | Research Roundtable (Part 1) | Marina, Kairong | ||
| 9/25 | VI: Stochastic Variational | Stochastic Variational Inference Hoffman, M. D. et al. JMLR, 2013 |
Kairong Jiang (notes) |
|
| 9/30 | Research Roundtable (Part 2) | Marium, Loren, Simon | ||
| 10/2 | VI: Loopy Belief Propagation & Bethe Variational Methods |
Understanding Belief Propagation and its
Generalizations Yedidia, J. S., et al., IJCAI, 2001 |
Marina Kisley (notes) |
WJ - Sec. 4.1 ES - Sec. 2.3.2 JP - Sec. 2.3.1-2.3.2 Constructing Free Energy Approximations and Generalized Belief Propagation Algorithms Yedidia, J. et al., IEEE Info. Theory, 2005 |
| 10/7 | Research Roundtable (Part 2.5) |
Open Topics Discussion: Plan to discuss any changes to scheduled topics
|
Simon | |
| 10/9 | VI: Expectation Propagation |
PRML - Sec. 10.7 |
Jason Pacheco (slides) |
Expectation Propagation Summary Sudderth, E., MIT 6.975 Course Notes, 2002 Expectation Propagation for Approximate Bayesian Inference Minka, T. P. UAI, 2001 |
| 10/14 | Markov chain Monte Carlo (MCMC): Introduction |
Introduction to Monte Carlo Methods MacKay, D. J. C . Learning in Graphical Models. Springer, 1998 |
Lecture: Jason Pacheco (slides) |
|
| 10/16 | MCMC: Hamiltonian Monte Carlo |
Term Project Proposals: Due Fri 10/18
MCMC Using Hamiltonian DynamicsDetails on the Resources page. Neal, R. M., From: "The Handbook of MCMC.", Chapman & Hall / CRC Press, 2011 Read Sec. 1-4 (inclusive) |
Kairong Jiang (slides) |
|
| 10/16 | MCMC: Hamiltonian Monte Carlo (Cont'd) | Typical Sets / Volume Preservation |
Jason Pacheco (slides) |
A Conceptual Introduction to Hamiltonian Monte Carlo Betancourt, M. arXiv, 2017 |
| 10/23 | MCMC: No U-turn Sampler |
The No-U-Turn sampler: Adaptively Setting Path Lengths in HMC Hoffman, M. D. and Gelman, A. JMLR, 2014 |
Simon Swenson (notes) |
|
| 10/28 | MCMC: Reversible Jump MCMC |
RJMCMC Computation and Bayesian Model Determination Green, P. J. Biometrika, 1995 |
Loren Champlin (notes) |
On Bayesian Analysis of Mixtures with an Unknown Number of Components Richardson, S. and Green, P. J. J. R. Stat. Soc., 1997 Minus attached discussion Reversible Jump MCMC Green, P. J. and Hastie, D. I. Genetics, 2009 Trans-dimensional markov chain monte carlo Green, P. J. Oxford Stat. Sci. Ser., 2003 |
| 10/30 | Dynamical Systems: Introduction |
Lecture: Jason Pacheco (slides) |
||
| 11/4 | Dynamical Systems: Introduction (continued) | Lecture: Jason Pacheco | ||
| 11/6 | DS: Particle Filter |
An Overview of Existing Methods and Recent Advances in Sequential Monte Carlo Cappé, O. et al. Proc. IEEE. 2007 Read Sec. 1 & 2 (Inclusive) |
Lecture: Jason Pacheco (slides) |
A Tutorial on Particle Filtering and Smoothing: Fifteen years later Doucet, A. and Johansen, A. M. Handbook of Nonlinear Filtering. 2009 |
| 11/11 | Veteran's Day | No Class | |||
| 11/13 | DS: Particle Filters (Cont'd) | No Assigned Readig | Lecture: Jason Pacheco | |
| 11/18 | DS: Infinite HMM |
The Infinite Hidden Markov Model Beal, M. et al. NeurIPS (2002) |
Jason Pacheco (notes) |
Stochastic Variational Inference for Bayesian Time Series Models Johnson, M. J. and Willsky, A. S. ICML (2014) |
| 11/20 | DS: HMM Topic Models |
Integrating Topics and Syntax Griffiths, T. L. et al. NeurIPS (2005) |
Marium Yousuf (slides) |
|
| 11/25 | Bayesian Deep Learning: Variational Autoencoders | Auto-encoding Variational Bayes Kingma, D. P. and Welling, M. ICLR, 2014 |
Marina Kisley (slides) |
Tutorial on Variational Autoencoders Doersch, C. ArXiv, 2016 Stochastic Backpropagation and Approximate Inference in Deep Generative Models Rezende, et al. ICML, 2014 |
| 11/27 | BDL: Structured VAEs |
Composing graphical models with neural networks for structured representations and fast inference Johnson, M. J., et al. NIPS, 2016 |
Jason Pacheco (notes) |
|
| 12/02 | BDL: Dropout Sampling |
Dropout as a Bayesian Approximation:Representing Model Uncertainty in Deep Learning Gal, Y. and Ghahramani, Z. ICML, 2016 |
Loren Champlin |
Dropout: A Simple Way to Prevent Neural Networks from Overfitting Srivastava, et al. JMLR, 2014 Variational Dropout and the Local Reparameterization Trick Kingma, et al. NIPS, 2015 Variational Dropout is not Bayesian Hron, J. et al. ArXiv, 2017 |
| 12/04 | Course Wrap-Up / Future Directions | Lecture: Jason Pacheco | Last Regular Class! | |
| 12/9 | Term Project Presentations (Group 1) | |||
| 12/11 | Term Project Presentations (Group 2) |
Final Project Reports Due
|
||