Hide menu

732A91 Bayesian Learning

Course information


The course aims to give a solid introduction to the Bayesian approach to statistical inference, with a view towards applications in data mining and machine learning. After an introduction to the subjective probability concept that underlies Bayesian inference, the course moves on to the mathematics of the prior-to-posterior updating in basic statistical models, such as the Bernoulli, normal and multinomial models. Linear regression and spline regression are also analyzed using a Bayesian approach. The course subsequently shows how complex models can be analyzed with simulation methods like Markov Chain Monte Carlo (MCMC). Bayesian prediction and marginalization of nuisance parameters is explained, and introductions to Bayesian model selection and Bayesian decision theory are also given.


  • Introduction to subjective probability and the basic ideas behind Bayesian inference
  • Prior-to-posterior updating in basic statistical models, such as the Bernoulli, normal and multinomial models.
  • Bayesian analysis of linear and nonlinear regression models
  • Shrinkage, variable selection and other regularization priors
  • Bayesian analysis of more complex models with simulation methods, e.g. Markov Chain Monte Carlo (MCMC).
  • Bayesian prediction and marginalization of nuisance parameters
  • Introduction to Bayesian model selection
  • Introduction to Bayesian decision theory.

Intended audience and admission requirements

This course is given primarily for students on the Master's programme Statistics and Machine Learning. It is also offered to Master students in other subjects and to interested Ph.D. students (with a more advanced examination).

Students admitted to the Master's programme in Statistics and Machine Learning fulfill the admission requirements for the course.
Students not admitted to the Master's programme in Statistics and Machine Learning should have passed:
  • an intermediate course in probability and statistical inference
  • a basic course in mathematical analysis
  • a basic course in linear algebra
  • a basic course in programming
It also required to have a basic knowledge of linear regression, either as a part of a statistics course, or as a separate course.

Course plan

The TimeEdit schedule for the course is available here.
The labs should be done in pairs of students whereas the project is individual work.
All labs and the project should be submitted as PDFs through LISAM.

Module 1 - The Bayesics

Lecture 1: Basics concepts. Likelihood. Bayesian inference. The Bernoulli model.
Read: BDA Ch. 1, 2.1-2.5 | Slides
Code: Beta density | Bernoulli model | One-parameter Gaussian model

Lecture 2: The Normal model. The Poisson model. Conjugate priors. Prior elicitation.
Read: BDA Ch. 2.6-2.9 | Slides

Lecture 3: Multi-parameter models. Marginalization. Multinomial model. Multivariate normal model.
Read: BDA Ch. 3. | Slides
Code: Two-parameter Gaussian model | Prediction with two-parameter Gaussian model | Multinomial model

Math exercises 1: One-parameter models.
Problem set 1 | Solution Problem 2 and 4 | Solution Problem 1 and 3 (Problem 3 is marked as Problem 2 in this solution)

Lab 1: Exploring posterior distributions in one-parameter models by simulation and direct numerical evaluation.
Lab 1 | Computer room instructions | LISAM Submission

Module 2 - Bayesian Regression and Classification

Lecture 4: Prediction. Making Decisions.
Read: BDA Ch. 9.1-9.2. | Slides

Lecture 5: Linear Regression. Nonlinear regression. Regularization priors.
Read: BDA Ch. 14 and Ch. 20.1-20.2 | Slides

Lecture 6: Classification. Posterior approximation. Logistic regression. Naive Bayes.
Read: BDA Ch. 16.1-16.3 | Slides
Code: Logistic and Probit Regression

Math exercises 2: Predictive distributions and decisions.
Problem set 2 | Solutions

Lab 2: Polynomial regression and classification with logistic regression
Lab 2 | Linköping temperature data | Women work data | LISAM Submission

Module 3 - More Advanced Models, MCMC and Variational Bayes

Lecture 7: Bayesian computations. Monte Carlo simulation. Gibbs sampling. Data augmentation.
Read: BDA Ch. 10-11 | Slides
Code: Gibbs sampling for a bivariate normal | Gibbs sampling for a mixture of normals

Lecture 8: MCMC and Metropolis-Hastings
Read: BDA Ch. 11 | Slides
Code: Simulating Markov Chains | Effective Sample Size

Lecture 9: HMC, Variational Bayes and Stan.
Read: BDA Ch. 12.4 and Ch. 13.7 and RStan vignette| Slides
Code: RStan - Three Plants | RStan - Bernoulli model | RStan - Logistic regression | RStan - Logistic regression with random effects | RStan - Poisson model

Math exercises 3: Comparing Bayes and Frequentist. Posterior approximation. Naive Bayes.
Problem set 3 | Solutions | R-code for exercise 3

Lab 3: MCMC using Gibbs sampling and Metropolis-Hastings
Lab 3 | HowToCodeRWM | Rainfall data | eBay data | LISAM Submission

Module 4 - Model Inference and Variable Selection

Lecture 10: Bayesian model comparison
Read: BDA Ch. 7 | Slides
Code: Comparing models for count data

Lecture 11: Computing the marginal likelihood, Bayesian variable selection, model averaging.
Read: Article on variable selection for additional reading | Slides

Lecture 12: Model evaluation and course summary.
Read: BDA 6.1-6.4 | Slides

Math exercises 4: Model comparison.
Problem set 4 | Solutions to problem set 4

Lab 4: Hamiltonian Monte Carlo with Stan
Lab 4 | Campylobacter data | LISAM Submission


  • Bayesian Data Analysis by Gelman, Carlin, Stern, och Rubin, Chapman & Hall, Third edition. The book's web site can be found here.
  • My slides.


The examination for the course Bayesian Learning, 6 credits, consists of
  • Written reports on the four computer labs (3 credits)
  • Computer exam (3 credits)
The computer exam will consist of four problems, all of which are to be solved in a computer exam room. One part of the exam (approximately 25%) will be mathematical problems to be solved with pen and paper. The other part (approximately 75%) will consist of problems to be solved using a computer, using R (Rstudio). You submit the solutions in PDF. LibreOffice (similar to Microsoft Word) will be available in the exam system and you can make a PDF from that program. You can also use Markdown from RStudio if you prefer that.

Computer lab instructions:
Load the course module and start Rstudio in the computer lab rooms by executing the following lines in a terminal window:
$ module add courses/TDDE07/2019-03-25.1
$ module add courses/732A91/2019-03-25.1
$ rstudio

Computer exam instructions:
The following material will be made available in the computer exam system in the folder given_files: Your lab solutions should be submitted in a PDF and/or an R file with code through a LISAM submission, and your own personal lab solutions will appear in the exam system. You can only submit a maximum of two files, one PDF and one R file with code. You can edit your lab solutions, but it is not allowed to add other information that is not a solution to the lab. You will get feedback on the last lab after the exam, so that lab you will bring to the exam without knowing if your solution was correct. Your submissions will be checked before the exam so that no extra material is added in the submitted files.
Here is the LISAM submission for the exam 2019-10-31. Remember to submit before the deadline 2019-10-28 at 23.30 AM.

ID-checks are performed before entering the exam computer room. No students are allowed to leave the room during the first hour of the exam, so for example restroom visits should be done before entering. Wait for the instructions. Do not log in or touch the computers. The technical support will make sure that everyone gets started after the log in instructions have been given.

Here is information about how to use the computer exam system: Step by step tutorial for the computer exam system
The exam system includes a so called communication client. The communication client serves three basic purposes:
  1. To ask questions to the teacher, and for the teacher to inform students about things during the exam. I will also visit the exam room regularly, but do use the communication client to ask questions. To submit a question, just click on the button marked by a dotted blue rectangle in this picture
  2. To get the Client ID (Klient ID in Swedish). This is the area marked by a dashed red rectangle in this picture. This is your personal identication number that you should write on every page of the papers that you submit for the math questions. Do not write your name on the papers, as that will reveal your identity to the grading teacher.
  3. To submit your (electronic) solutions to the exam. You will only have a single submission for the whole exam. The exam is submitted by clicking the button marked by a solid green rectangle in this picture and following the instructions. Note that the system will let you know that the exam has been submitted, but will not tell you that it was received. This is ok and your solution has actually been received.
Here are some additional useful info about the computer exam system:
  • There are two basic folders for the students located in the desktop (the desktop folder):
    • /given_files (the folder where the distributed files to everyone are placed. For example, the exam itself or slides)
    • /my_given_files (the folder where the distributed files to individual students are placed. For example, the student's lab solutions)
  • The student may save work files directly in the desktop but the solutions should be submitted using the communication client.
  • Most students will save the plots directly from RStudio using point-and-click from the menus. But you can also save a plot to a png file using these commands:
    plot(rnorm(4)) # Change this to the plot commands you want
The course 732A91 will be graded on the A-F scale.
The course TDDE07 will be graded on the (U,3,4,5) scale.
Here is picture that shows the percentage of the maximum score for each grade (732A91 to the left, TDDE07 to the right).

Old exams with solutions

R stuff

Other material

Page responsible: Mattias Villani
Last updated: 2019-11-04