IE598: Course Information

Sewoong Oh, University of Illinois Urbana-Champaign

Lecture

Credit: 4 undergraduate hours. 4 graduate hours. Meeting schedule: Two 75 minute lectures per week. Lecture is on Tuesdays and Thursdays, 2:00pm-3:15pm, TB 206.

Textbook and optional references

There is no existing text that perfectly matches the content of IE598 SO. However, the following references contain useful additional details and insights.

Steffen L. Lauritzen, Graphical Models, Oxford University Press,1996
Marc Mézard and Andrea Montanari, Information, Physics, and Computation, Oxford University Press, 2009
M. Wainwright and M. Jordan, Graphical models, exponential families, and variational inference, Foundations and Trends in Machine Learning, 2008
D. Koller and N. Friedman, Probabilistic Graphical Models: Principles and Techniques, MIT Press, 2009
Ian Goodfellow, Yoshua Bengio and Aaron Courville, Deep Learning, MIT press, 2016 You really won't need these books; we list them just in case you want to consult some other references.

Course requirements and grading

Requirements:

Attendance.
Homework assignments. Working through (and, yes, often struggling with at length!) the homework is a crucial part of the learning process and will invariably have a major impact on your understanding of the material. In understanding the problems sets, collaboration with your peers in small groups are allowed, even encouraged, but you must write up your own homework to hand in. Homework solutions must be typeset in LaTeX (no handwritten or Microsoft Word submissions will be accepted!) and converted to PDF.
Electronic submission only through gradescope (self-enrollment code 956ZEM)

One midterm quiz and a final project.

Grading

Homework 30%, midterm quiz 30%, final project 40%. These weights are approximate; we reserve the right to change them later.

Prerequisites

Knowledge of basic probability and basic linear algebra is required. Exposure to programming language (e.g. Matlab) is helpful but not required.

Catalog description

Introduction to statistical inference with probabilistic graphical models and low-complexity inference algorithms. In particular, we will treat the following methods: message-passing algorithms, belief propagation, loopy-belief propagation, variational methods, Markov chain Monte Carlo methods, learning structure. Applications and examples will include: Gaussian graphical models; linear dynamical systems and hidden Markov models (forward-backard algorithm, Kalman filtering, Viterbi algorithm); computer vision; machine learning (clustering, classification);

Topical Outline

topic	lectures
Chapter 1. Overview	1
Chapter 2. Graphical Models	1
Chapter 3. Markov Property	2
Chapter 4. Belief Propagation	3
Chapter 5. Density evolution	1
Chapter 6. Max-product algorithm	1
Chapter 7. Gaussian graphical models	3
Chapter 8. Restricted BOltzmann Machines	3
Chapter 9. Markov chain Monte Carlo	3
Chapter 10. Variational inference	3
Chapter 11. Learning	3
Chapter 12. Applications	2
in-class quiz	1
project presentation	3
total	30

Project

Students will propose and read recent research papers on graphical models and present it to the class.

Other courses in graphical models

CS598. Graphical Models Sanmi Koyejo
ECE417. Multimedia signal processing by Thomas Huang
CS440/ECE448. Introduction to artificial intelligence by Eyal Amir
CS598RAR. Probabilistic graphical models for AI by Rob Rutenbar