Introduction to Markov decision processes FDA171, 2004HT Status Archive School National Graduate School in Computer Science (CUGS) Division TECH-OU Owner Lars Karlsson

Log in

---

Course plan

Lectures

Two lectures/exercises; approx 8 h ex each. Takes place in Örebro.

Recommended for

Potentially interesting for all CUGS students, but in particular relevant for those working with systems that need to interact with physical environments or other agents.

The course was last given

New course.

Goals

To obtain a basic understanding of what a Markov decision process is and how it can be used.

Prerequisites

Basic knowledge in probability theory and optimization.

Contents

The course gives an introduction to Markov decision processes, which is a tool for optimal decision making in situations where the outcomes of the decision-making system's actions have a stochastic element. MDPs are used in computer science, control, biology, economics and a number of other fields. More specifically, the course adresses:

* Markov Chains

* Markov Decision Processes

* Solving MDPs: value iteration and policy iteration

* Partially Observable Markov Decision Processes (POMDPs)

* Applications: CS (planning, learning etc), control, biology, economy, and so on.

Organization

Intensive course

Literature

Martin L. Puterman: Markov Decision Processes: Discrete Stochastic Dynamic Programming, Wiley-Interscience, 1994. ISBN: 0471619779.

Lecture notes and articles.

Lecturers

Lars Karlsson

Examiner

Lars Karlsson

Examination

Home assignment.

Credit

2.5 credits

Organized by

AASS, Örebro university

Comments

Schedule: prel October-November, 2004. The course is given in Örebro.

Page responsible: Anne Moe