Hide menu

732A96 Advanced Machine Learning

Course information

Course description

The course covers some advanced models in machine learning.
The models are analyzed mainly from a Bayesian perspective.

The course is organized into four topics:
- Graphical Models
- Hidden Markov Models
- Gaussian Process Regression and Classification
- State-space models

Each topic includes lectures, a computer lab and a follow-up seminar.

Course literature

- Chapters in the book Pattern recognition and machine learning (PRML) by C.M. Bishop, ISBN 9780387310732.
- The article A Review of Bayesian Networks and Structure Learning in Mathematica Applicanda by Timo Koski and John Noble.
- The article Local Computations with Probabilities on Graphical Structures and Their Applications to Expert Systems in Journal of the Royal Statistical Society B by Steffen L. Lauritzen and David J. Spiegelhalter.
- The article An Introduction to Hidden Markov Models and Bayesian Networks in International Journal of Pattern Recognition and Artificial Intelligence by Zoubin Ghahramani.
- The article Explicit-Duration Markov Switching Models in Foundations and Trends in Machine Learning by Silvia Chiappa.
- Gaussian Processes for Machine Learning by Rasmussen and Willams.
- Probabilistic Robotics by Thrun et al.
- Slides
- Other material distributed during the course.

Topic 1: Graphical Models

Teacher: Jose Peña
Read: PRML Chapter 8, Koski and Noble, and Lauritzen and Spiegelhalter.

Lecture 1: Bayesian and Markov networks Slides
Lecture 2: Probabilistic reasoning Slides
Lecture 3: Parameter learning Slides
Lecture 4: Structure learning Slides

Lab 1: Lab
Seminar 1

Topic 2: Hidden Markov Models

Teacher: Jose Peña
Read: PRML Chapter 13.1-13.2, Ghahramani, and Chiappa Sections 3.1-3.3.

Lecture 5: Dynamic Bayesian Networks and Hidden Markov Models Slides
Lecture 6: Autoregressive and Explicit-Duration Hidden Markov Models Slides

Lab 2: Lab
Seminar 2

Topic 3: Gaussian Process Regression and Classification

Teacher: Mattias Villani
Read: PRML 6.4.1-6.4.6 and/or Gaussian Processes for Machine Learning Ch. 2.1-2.5, 3.1-3.4.1, 3.7.

Lecture 7: Gaussian Process Regression - Basics Slides
Lecture 8: Kernels, Hyperparameter learning and Computations Slides
Lecture 9: Gaussian Process Classification and Gaussian Process Optimization Slides

Lab 3: Lab
Seminar 3

Extras: Web applet for playing around with a simple GP | Chapter on kernels from David Duvenaud's PhD thesis.
Packages: kernlab with Vignette
Code:R code for simulating from a GP | Quick demo of the R package kernlab | An even quicker demo of GPs in Matlab.
Data: Lidar | JapanTemp | CanadianWages

Topic 4: State-Space Models

Teacher: Mattias Villani
Read: PRML 13.3 and pages from Probabilistic Robotics

Lecture 10: Linear Gaussian state space models and the Kalman filter Slides
Lecture 11: State smoothing, Bayesian inference and R packages Slides
Lecture 12: Non-linear/non-Gaussian state space models - extended Kalman and particle filters Slides

Lab 4: Lab | Simulated data | Financial data
Seminar 4

Packages: dlm for ML and Bayesian analysis with Vignette and book | kfas for GLM models with Vignette | rucm for unobserved components models with Vignette
Code: UC model demo on Nile flow data | DLM package demo on Nile flow data.

Other Material:

- The course has an open GitHub repository where much of the course material can be downloaded.
- The course also has LISAM page.

Page responsible: Oleg Sysoev
Last updated: 2016-10-19