TDDE19 Advanced Project Course - AI and Machine Learning

Projects

Projects

Students will be divided in a group of around six students, each group is assigned to a project (according to the preference and skills of each student).

At the end of the project, the students are expected to provide:

• Source code of library/program in a gitlab repository
• Documentation of how to use and install the software (API, command line...)
• A group report describing the work that has been accomplished (which algorithms are used, what kind of results were obtained...)
• An individual list of contributions to the project.

Important: the deadline for the selection of projects is Wednesday 5th of September at 12:00, you need to send me an email (cyrille.berger at liu.se) with the following information:

• Your LiU login.
• A ranked list of prefered projects (details on the projects below)
• If you want to work as a group of several people together, send me a single email with all your LiU logins

Ground Robots Control and Obstacle Avoidance

Customer: Cyrille Berger

Context: a ground robot is driving in rough terrain, to reach its location it needs to be able to avoid obstacle. In this project the students would be working with reactive control and obstacle avoidance technique.

Task suggestions:

• Study literature
• Select several possible techinques for control obstacle avoidance in the literature, implement them and evaluate them in simulator
  • Terrain classification
  • Reinforcement-learning
  • Dynamic window approach
  • ...
• If time permit, test on an actual platform

References:

• The dynamic window approach to collision avoidance, Fox, D.; Burgard, W.; Thrun, S, Robotics & Automation Magazine, IEEE 1997
• High speed obstacle avoidance using monocular vision and reinforcement learning, Jeff Michels, Ashutosh Saxena, Andrew Y. Ng, ICML '05 Proceedings of the 22nd international conference on Machine learning
• A Survey of Deep Learning Techniques for Mobile Robot Applications, Jahanzaib Shabbir, and Tarique Anwe, ARXIV

Advanced Dialog System for Robotic System

Customer: Cyrille Berger

Context: having the ability to control a robotic system through voice commands would allow the operator to get assistance from the robot while his hands are busy. However this is a complex task to accomplish from the software perspective, it requires to recognize speech, understand the command, understand the current state of the system and have the ability to answer to the human. Those tasks are usually implemented in a dialog system.

Many of the currently deployed voice control system (siri, ok google, Mycroft...) have very simple dialog, they react to specific templates (ie setup a meeting at 13:00 with Peter) and have simple answer. Setting up a mission with a robotic system is a complex task which might require the dialog system to ask many different questions to the user and do many check on whether the user request is acceptable.

The goal of this project is to implement a dialog system for a robotic system, that can be used for simple query (where is robot X?) or setup complex mission.

This project is a continuation of last year work, and this year the work should focus on:

• Speech recognition
• Improving the dialog manangement

References:

• Social Robots that Interact with People, Cynthia Breazeal, Atsuo Takanishi and Tetsunori Kobayashi, Springer Handbook of Robotics. Ed. Bruno Siciliano and Oussama Khatib. Berlin: Springer, 2008 (GVRL)
• Spacy
• Spoken dialogue systems: the new frontier in human-computer interaction, Pierre Lison and Raveesh Meena (ACM)
• OpenDial Toolkit: opendial-toolkit
• DeepSpeech: DeepSpeech
• List of neural net dialog papers: NeuralDialogPapers

Deep Learning for Vehicle Maintenance

Customer: Olov Andersson (supervisor, LiU), industrial parters Toyota Material Handling and maybe SAAB.

Context: Inefficient maintenance of vehicles lead to considerable financial and environmental costs. By better predicting the need for maintenance in advance, or aiding troubleshooting after a failure, deep learning may be able to decrease costs. In this project we want to test deep learning on maintenance data. In particular, we are interested in using deep learning to build predictive models of failures in vehicles. Either as an on-site decision support agent for maintenance personell, or to predict and analyze failure rates over time.

The aim is to use real data from our industrial partners such as Toyota. Toyota Material Handling is the world's number one supplier of forklifts and warehouse equipment. Their R&D center in Mjölby works with product knowledge and future technology, e.g. fully connected solutions, IoT for material handling and real time analysis with big data. NOTE: Business data is sensitive, students may have to sign some agreement not to share it outside the project.

Examples of vehicles from the industrial partners:

Task suggestions (finalized in discussion with students and project stakeholders):

• Neural Network Classifier for Decision Support:
  • What the likely cause of a failure is, based on on e.g. age, maintenance history, model for Toyota data.
• Time to failure prediction, using either Recurrent Neural Networks on series of events in time, or some simpler regression. For example:
  • Predict time to failure of Toyota forklifts/parts based on e.g. age, maintenance history, model.
  • Predict time to failure of e.g. jet engine from on-board diagnostic data, system parameters, maintenance history, etc.

Prerequisites:

• Some machine learning or statistics experience is highly recommended, deep learning can then be picked up as you go.
• Some degree of self/group-study is expected at the start of the project. Suitable reading and references will be provided by Olov. Standard deep learning frameworks such as Tensorflow/Keras are fairly easy to pick up, it is more important that you are comfortable working with data and numerical algorithms.

References:

• Tensorflow/Keras tutorials, including standard classification models
• Deep learning reference book
• Learning models for sequential events in time (like failures) requires more advanced models such as recurrent neural networks. There are several open source implementations:
  • WTTE-RNN looks promising, and is the result of a Masters' thesis at Chalmers, although the summarizing blog post might be easier to read (see jet engine example).
  • Deep-TTF is a fork of this specalized on failure prediction.

Arrival time prediction of local buses in Linköping

Customer: Mattias Tiger (LiU), Östgötatrafiken AB.

Context: Östgötatrafiken AB is owned by Östergötland County and is responsible for the public transportation in the county. Accurate arrival/departure time prediction is important for their customers and it is also important for Östgötatrafiken when developing and optimizing the bus service. We have in collaboration with Östgötatrafiken been given access to 3 months of vehicle events for all their vehicles. Most of these events are position events which specify the coordinate of the bus at a rate of about 1Hz. Using this data we would like you to evaluate different techniques for how to do accurate arrival time prediction.

A Master Thesis is about to be concluded on a first analysis of the data set which provides detailed information on properties of the data, the format, suitable pre-processing to be done and so forth.

Task suggestions (finalized in discussion with students and project stakeholders):

• Literature study about techniques for arrival time prediction (forecasting). Categorize different techniques, problems (e.g. type of vehicle, environment, feature types..) and evaluation methods used in the literature.
• Compile a suitable dataset from the given data by pre-processing it (clean, re-organize etc.).
• Evaluate two suitable techniques on the data set.

References:

• Coming soon

GPS spatio-temporal uncertainty estimation

Customer: Mattias Tiger (LiU), Östgötatrafiken AB.

Context:Although the uncertainty and bias is fairly low in open places there is a lot to wish for in places covered by trees and on streets with tall buildings and plenty of signal echos. The externally available uncertainty estimate is fairly limited in current GPS modules, so the GPS accuracy and bias provided is often very crude or simply way off in these more problematic areas. In addition, due to the changing positions of the satellites, the uncertainty and bias will change over time for the same position.

Östgötatrafiken AB is owned by Östergötland County and is responsible for the public transportation in the county. We have in collaboration with Östgötatrafiken been given access to 3 months of vehicle events for all their vehicles. Most of these events are position events which specify the coordinate of the bus at a rate of about 1Hz. Using this data we would like you to evaluate different techniques for how to do accurate arrival time prediction.

When GPS is used for navigation in vehicles these problems are commonly circumvented by projecting the GPS position onto the closest road segment in a geographical information database (GIS) such as Google maps or open street map. This is however not sufficient for advanced functions such as detecting when a bus has arrived or departed a bus stop, or for detecting when a bus is close enough to an intersection such that it can be given a green light. Both these functions are used live today by Östgötatrafiken, but they have problems with the varying quality of the GPS positioning across time and space.

If we would have a map of the GPS uncertainty we could make better decisions and detections, for example weight the uncertainty of the position against how likely it is that the bus has arrived at the bus stop. If this map captures both spatial uncertainty differences and how it varies over time the resulting map would be even more useful. If information about the positions of the satellites is used when the problem of estimating periodic changes in uncertainty might become easier.

A Master Thesis is about to be concluded on a first analysis of the data set which provides detailed information on properties of the data, the format, suitable pre-processing to be done and so forth.

Suitable techniques are Gaussian processes (or other random fields) with for example periodic kernels for capturing the temporal variations.

Source: GPS World

Task suggestions (finalized in discussion with students and project stakeholders):

• Develop and evaluate techniques for estimating a spatial (and spatio-temporal) map of the GPS uncertainty in Linköping using extended trajectories of buses.

References:

• Coming soon

Motion pattern prediction of vehicles in intersections

Customer: Mattias Tiger (LiU), Östgötatrafiken AB.

Context: In intelligent traffic surveillance it is important to detect abnormal or dangerous vehicle behaviors in for example intersections. For autonomous cars it is important to be aware of how surrounding cars are likely to drive in the near future, this in order to avoid accidents and efficiently be able to plan its own path towards its goal. Motion pattern prediction can be used to solve both these problems, where the task is to make predictions about the future state of a vehicle given a single observation or a trace of observations of that vehicle. For this task we will use trajectories of buses driving in Linköping over an extended period of time.

Östgötatrafiken AB is owned by Östergötland County and is responsible for the public transportation in the county. We have in collaboration with Östgötatrafiken been given access to 3 months of vehicle events for all their vehicles. Most of these events are position events which specify the coordinate of the bus at a rate of about 1Hz. Using this data we would like you to evaluate different techniques for how to do motion pattern prediction of vehicles in intersections. A Master Thesis is about to be concluded on a first analysis of the data set which provides detailed information on properties of the data, the format, suitable pre-processing to be done and so forth.

Source: Seattle's bike blog

Task suggestions (finalized in discussion with students and project stakeholders):

• Literature study about techniques for motion pattern prediction of vehicles. Categorize different techniques, problems (e.g. type of vehicle, environment, feature types..) and evaluation methods used in the literature.
• Compile a suitable dataset from the given data by preprocessing it (clean, re-organize etc.).
• Evaluate several suitable techniques on the data set (decided which in a dialog with the customer), including our own method (first reference).

References:

• Mattias Tiger and Fredrik Heintz. "Gaussian Process Based Motion Pattern Recognition with Sequential Local Models". In: 2018 IEEE Intelligent Vehicles Symposium (IV). 2018
• ... Master Thesis
• K. Kim, D. Lee, and I. Essa, "Gaussian process regression flow for analysis of motion trajectories", in Proc. ICCV, 2011.
• Q. Tran and J. Firl, "Online maneuver recognition and multimodal trajectory prediction for intersection assistance using non-parametric regression" in IEEE Intelligent Vehicles Symposium, June 2014.
• S. Ferguson, B. Luders, R. C. Grande, and J. P. How, Algorithmic Foundations of Robotics XI: Selected Contributions of the Eleventh International Workshop on the Algorithmic Foundations of Robotics. Springer International Publishing, 2015, ch. Real-Time Predictive Modeling and Robust Avoidance of Pedestrians with Uncertain, Changing Intentions.
• M. Smith, S. Reece, I. Rezek, I. Psorakis, and S. Roberts, "Maritime abnormality detection using gaussian processes", Knowledge and Information Systems, pp. 1–26, 2013.
• B. T. Morris and M. M. Trivedi, "Understanding vehicular traffic behavior from video: a survey of unsupervised approaches", Journal of Electronic Imaging, vol. 22, no. 4, pp. 041–113, 2013.

Page responsible: infomaster
Last updated: 2022-08-26