Hide menu

732A96 Advanced Machine Learning

Course information

Course description

The course covers some advanced models in machine learning.
The models are analyzed mainly from a Bayesian perspective.

The course is organized into four topics:
- Graphical Models
- Hidden Markov Models
- State-space models
- Gaussian Process Regression and Classification

Each topic includes lectures, a computer lab and a follow-up seminar.

Course literature

- Chapters in the book Pattern recognition and machine learning (PRML) by C.M. Bishop, ISBN 9780387310732.
- The article A Review of Bayesian Networks and Structure Learning in Mathematica Applicanda by Timo Koski and John Noble.
- The article Local Computations with Probabilities on Graphical Structures and Their Applications to Expert Systems in Journal of the Royal Statistical Society B by Steffen L. Lauritzen and David J. Spiegelhalter.
- The article An Introduction to Hidden Markov Models and Bayesian Networks in International Journal of Pattern Recognition and Artificial Intelligence by Zoubin Ghahramani.
- The article Explicit-Duration Markov Switching Models in Foundations and Trends in Machine Learning by Silvia Chiappa.
- Gaussian Processes for Machine Learning by Rasmussen and Willams.
- Probabilistic Robotics by Thrun et al.
- Slides.
- Other material distributed during the course.

Topic 1: Graphical Models

Teacher: Jose Peña
Read: PRML Chapter 8, Koski and Noble, and Lauritzen and Spiegelhalter.

Lecture 1: Bayesian and Markov Networks Slides
Lecture 2: Probabilistic Reasoning Slides
Lecture 3: Parameter Learning Slides
Lecture 4: Structure Learning Slides

Lab 1: Lab
Seminar 1

Topic 2: Hidden Markov Models

Teacher: Jose Peña
Read: PRML Chapter 13.1-13.2, Ghahramani, and Chiappa Sections 3.1-3.3.

Lecture 5: Dynamic Bayesian Networks and Hidden Markov Models Slides
Lecture 6: Autoregressive and Explicit-Duration Hidden Markov Models Slides

Lab 2: Lab
Seminar 2

Topic 3: State-Space Models

Teacher: Jose Peña
Read: PRML Chapter 13.3 and pages from Probabilistic Robotics

Lecture 7: Linear Gaussian State Space Models and the Kalman Filter Slides
Lecture 8: Extended, Unscented, and Particle Kalmar Filters Slides
Lecture 9: Learning Linear Gaussian State Space Models Slides

Lab 3: Lab
Seminar 3

Topic 4: Gaussian Process Regression and Classification

Teacher: Jose Peña
Read: PRML 6.4.1-6.4.6 and/or Gaussian Processes for Machine Learning Ch. 2.1-2.5, 3.1-3.4.1, 3.7.

Lecture 10: Gaussian Process Regression - Basics Slides
Lecture 11: Kernels, Hyperparameter learning and Computations Slides
Lecture 12: Gaussian Process Classification and Gaussian Process Optimization Slides

Lab 4: Lab
Seminar 4

Extras: Web applet for playing around with a simple GP | Chapter on kernels from David Duvenaud's PhD thesis.
Packages: kernlab with Vignette
Code:R code for simulating from a GP | Quick demo of the R package kernlab | An even quicker demo of GPs in Matlab.
Data: Lidar | JapanTemp | CanadianWages

Other Material:

- The course has an open GitHub repository where much of the course material can be downloaded.
- The course also has LISAM page.

Page responsible: Jose M. Pe˝a
Last updated: 2018-08-16