Hide menu

Neural Networks and Deep Learning

2020HT
Full

Status Active. Full course - only reserve registrations
School IDA-gemensam (IDA)
Division NLPLAB
Owner Marco Kuhlmann
Homepage http://www.ida.liu.se/divisions/aiics/nlp/courses/nn/

  Log in  




Course plan

Lectures

Basics of Machine Learning. Deep Feed-Forward Networks. Convolutional Networks. Recurrent and Recursive Networks. Adversarial Training. Applications: Classification and Regression on Spatial Data and Sequential Data

Recommended for

Doctoral students with an interest in neural networks and deep learning and how to apply techniques from these areas to their own research

The course was last given

Autumn 2019

Goals

On successful completion of the course the student should be able to:

* explain different network architectures and how these are used in current applications,
* implement, train, and evaluate neural networks using existing software libraries,
* present and critically assess current research on neural networks and their applications,
* relate the concepts and techniques introduced in the course to the student's own research,
* plan and carry out a research project on neural networks within given time limits.

Prerequisites

* basic calculus (derivatives)
* basic linear algebra (matrices, vectors)
* basic probability and statistics
* programming experience in Python or MATLAB

Gaps in these prerequisites, in particular the programming prerequisite, may be filled by teacher-assisted self-study before the start of the course; contact the examiner for details.

Contents

Basics of machine learning (regression, classification, numerical optimisation). Feed-forward networks. Loss functions. Back-propagation training. Regularisation. Adversarial training. Convolutional networks. Recurrent and recursive networks. Processing sequences, images, and hierarchical structures. Applications of neural networks in natural language processing and computer vision. Current areas of research.

Organization

The course is organised in two parts. The first part consists of lectures presenting basic concepts and methods in deep learning, as well as applications from two areas where deep learning has been particularly successful. This part also includes a number of lab sessions that will give students practical experience in implementing, training, and evaluating deep learning architectures using existing software libraries. The second (optional) part of the course is an individual project that the students formulate together with their PhD supervisors.

Literature

The main book for the course is: Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep Learning. MIT Press, 2016.

Additional reading consists of excerpts from the following books:
* Christopher M. Bishop. Neural Networks for Pattern Recognition. Oxford University Press, 1996.
* Simon O. Haykin. Neural Networks and Learning Machines. Third edition. Prentice Hall, 2008.
* Yoav Goldberg. Neural Network Methods in Natural Language Processing. Morgan & Claypool, 2017.

Lecturers

* Marco Kuhlmann (IDA)
* Michael Felsberg (ISY)

Examiner

Marco Kuhlmann (IDA)
Michael Felsberg (ISY)

Examination

* lab assignments (3 credits)
* optional individual project (3 credits)

Credit

3 + 3 credits

Organized by

Natural Language Processing Laboratory (IDA), Computer Vision Laboratory (ISY)

Comments

This is a faculty-level PhD course at the Faculty for Science and Engineering.


Page responsible: Director of Graduate Studies
Last updated: 2012-05-03