732A96 Advanced Machine Learning
Course descriptionThe course covers some advanced models in machine learning.
The models are analyzed mainly from a Bayesian perspective.
The course is organized into four topics:
- Graphical Models
- Hidden Markov Models
- Gaussian Process Regression and Classification
- State-space models
Each topic includes lectures, a computer lab and a follow-up seminar.
Course literature- Chapters in the book Pattern recognition and machine learning (PRML) by C.M. Bishop, ISBN 9780387310732.
- The article A Review of Bayesian Networks and Structure Learning in Mathematica Applicanda by Timo Koski and John Noble.
- The article Local Computations with Probabilities on Graphical Structures and Their Applications to Expert Systems in Journal of the Royal Statistical Society B by Steffen L. Lauritzen and David J. Spiegelhalter.
- The article An Introduction to Hidden Markov Models and Bayesian Networks in International Journal of Pattern Recognition and Artificial Intelligence by Zoubin Ghahramani.
- The article Explicit-Duration Markov Switching Models in Foundations and Trends in Machine Learning by Silvia Chiappa.
- Gaussian Processes for Machine Learning by Rasmussen and Willams.
- Probabilistic Robotics by Thrun et al.
- Other material distributed during the course.
Topic 1: Graphical ModelsTeacher: Jose Peña
Read: PRML Chapter 8, Koski and Noble, and Lauritzen and Spiegelhalter.
Lecture 1: Bayesian and Markov networks Slides
Lecture 2: Probabilistic reasoning Slides
Lecture 3: Parameter learning Slides
Lecture 4: Structure learning Slides
Lab 1: Lab
Topic 2: Hidden Markov ModelsTeacher: Jose Peña
Read: PRML Chapter 13.1-13.2, Ghahramani, and Chiappa Sections 3.1-3.3.
Lecture 5: Dynamic Bayesian Networks and Hidden Markov Models Slides
Lecture 6: Autoregressive and Explicit-Duration Hidden Markov Models Slides
Lab 2: Lab
Topic 3: Gaussian Process Regression and ClassificationTeacher: Mattias Villani
Read: PRML 6.4.1-6.4.6 and/or Gaussian Processes for Machine Learning Ch. 2.1-2.5, 3.1-3.4.1, 3.7.
Lecture 7: Gaussian Process Regression - Basics Slides
Lecture 8: Kernels, Hyperparameter learning and Computations Slides
Lecture 9: Gaussian Process Classification and Gaussian Process Optimization Slides
Lab 3: Lab
Extras: Web applet for playing around with a simple GP | Chapter on kernels from David Duvenaud's PhD thesis.
Packages: kernlab with Vignette
Code:R code for simulating from a GP | Quick demo of the R package kernlab | An even quicker demo of GPs in Matlab.
Data: Lidar | JapanTemp | CanadianWages
Topic 4: State-Space ModelsTeacher: Mattias Villani
Read: PRML 13.3 and pages from Probabilistic Robotics
Lecture 10: Linear Gaussian state space models and the Kalman filter Slides
Lecture 11: State smoothing, Bayesian inference and R packages Slides
Lecture 12: Non-linear/non-Gaussian state space models - extended Kalman and particle filters Slides
Lab 4: Lab | Simulated data | Financial data
Packages: dlm for ML and Bayesian analysis with Vignette and book | kfas for GLM models with Vignette | rucm for unobserved components models with Vignette
Code: UC model demo on Nile flow data | DLM package demo on Nile flow data.
Other Material:- The course has an open GitHub repository where much of the course material can be downloaded.
- The course also has LISAM page.
Page responsible: Oleg Sysoev
Last updated: 2016-10-19