- Class: Tuesdays and Thursdays 1:05-2:25 PM, ENGMC 320
- Office hours: Thursdays 2:30-3:30 PM, ENGMC 325
- TA: Harsh Satija
- TA office hours: Fridays 4-5 PM, ENGMC 106
Graphical models are powerful probabilistic modeling tools. They can model the complex behavior of a large system of interacting variables through local relations specified using a graph. These probabilistic models represent the conditional dependencies between a subsets of variables in a compressed and elegant form. The framework of graphical models has achieved a remarkable success across a variety of domains, from near-optimal codes for communication to the state-of-the-art in combinatorial optimization; these models are widely used in bioinformatics, robotics, vision, natural language processing and machine learning. In this course, we study both directed and undirected models, exact and approximate inference procedures including deterministic optimization-based and stochastic sampling-based methods. We also discuss various approaches to learning model from the data and time permitting discuss deep generative models as instances of graphical models.
Basic familiarity with probability theory and algorithm design is required. Assigments need familiarity with Python and Numpy. Background in AI and in particular (applied) machine learning is highly recommended.
Koller, Daphne, and Nir Friedman. Probabilistic graphical models: principles and techniques. MIT press, 2009.
- Wainwright, Martin J., and Michael I. Jordan. Graphical models, exponential families, and variational inference. Foundations and Trends in Machine Learning, 2008
- Murphy, Kevin P. Machine learning: a probabilistic perspective. MIT press, 2012.
- Barber, David. Bayesian reasoning and machine learning. Cambridge University Press, 2012.
Assignments, announcements, slides, project descriptions and other course materials are posted on myCourses.
- Syllabus, review of the probability theory (slides, chapters 2)
- Bayesian Networks (slides, chapter 3)
- Markov Networks (slides) and
- Converting directed to/from undirected models (slides, chapter 4)
- Local and Conditional Probability Models (slides, chapter 5)
- Gaussian Network Models (slides, chapter 6)
- Complexity of Inference and Variable Elimination (slides, chapter 7)
- Junction Trees and Belief Propagation (slides, chapter 10)
- Variational Inference (chapters 8, 11, 13)
- Maximum a Posteriori Inference (slides, chapter 13)
- Sampling Based Inference (chapter 12)
- Overview: Objectives in Learning (slides, chapter 16)
- Maximum likelihood and Bayesian Estimation in Directed Models (slides, chapter 17)
- Structure learning in Directed Models (slides, chapter 18)
- Parameter-Learning in Undirected Models (slides, chapter 20)
- Learning with Partial Observations (slides, chapters 19)
“McGill University values academic integrity. Therefore, all students must understand the meaning and consequences of cheating, plagiarism and other academic offences under the Code of Student Conduct and Disciplinary Procedures” (see www.mcgill.ca/students/srr/honest/ for more information). (Approved by Senate on 29 January 2003)