TDDE19 Advanced Project Course - AI and Machine Learning
Projects
Projects
Students will be divided into a group of around 4-5 students. Each group is assigned to a project (according to the preference and skills of each groups).
At the end of the project, the students are expected to provide:
- Source code of library/program in a gitlab repository
- Documentation of how to use and install the software (API, command line...)
- A group report describing the work that has been accomplished (which algorithms are used, what kind of results were obtained...)
- An individual list of contributions to the project.
Important: the deadline for the selection of projects is Wednesday 30th of September 2023 at 12:00, you need to send me an email (cyrille.berger at liu.se) with the following information:
- The group number from webreg.
- A ranked list of all projects (details on the projects below)
Natural Language to Query
Customer: Cyrille Berger
Context: the goal of this project is to transform a question in English into a query executed on a database. We are particularly interested in queries expressed in the SPARQL language for the Semantic Web. We are also interested in applying this technique to a custom query language (called scQL). We do not have a dataset for that language yet. An important question for us would be how much training data we need to use transfer learning for our custom query language.
Task suggestions:
- Study literature.
- Extend the work from last year.
- Use of an answer model for scQL.
References:
- [Text-to-SQL] Learning to query tables with natural language
- WikiSQL
- SPBERT: A Pre-trained Model for SPARQL Query Language
- Last year project: Source code and Report
Automatic Quest Generation
Customer: Annonymous (Represented by Cyrille Berger)
Context: in video games, most quests are hand written by the developers (or some time modders). The idea of this project is to use the advance in deep language models to automatically generate quests based on the current state of the game and interest of the players.
Task suggestions:
- Study literature
- Implementation of "Personalized Quest and Dialogue Generation in Role-Playing Games".
References:
- Personalized Quest and Dialogue Generation in Role-Playing Games: A Knowledge Graph- and Language Model-based Approach
- Fine-tuning GPT-2 on annotated RPG quests for NPC dialogue generation
- Generating Role-Playing Game Quests With GPT Language Models
- Scheherazade’s Tavern: A Prototype For Deeper NPC Interactions
Chatbot as teaching assistant
Customer: Cyrille Berger
Context: the goal is to develop a chatbot that is capable of answering questions from students. The chatbot should give accurate answers. Because large language models (LLM) are prone to hallucination and may give wrong answers, they are not suitable to use directly. Instead the project should focus on developing a chatbot that guide the students toward an answer that was written by a teacher.
The idea is that the student describe to the chatbot his problem, the chatbot then select the answer that is the most likely. The chatbot may also ask questions to help refine its answer.
Example of use:
- "S: I have a problem, my robot is not moving."
- "C: What exercise are you doing?"
- "S: The exercise where we sent a velocity to the robot."
- "C: Have you checked this: [[ answer 1 from the teacher ]]"
- "S: It did not help."
- "C: Have you checked this: [[ answer 2 from the teacher ]]"
Operational constraints: real time, ease of installation and adding questions.
References:
- Ontology-based Integration of Knowledge Base for Building an Intelligent Searching Chatbot
- QUEST: A Keyword Search System for Relational Data based on Semantic and Machine Learning Techniques
- Chatbot for university related FAQs
- Chatbot for College Website
- Context Sensitive Conversational Agent Using DNN
- Knowledge Graph and Deep Learning-based Text-to-GraphQL Model for Intelligent Medical Consultation Chatbot
Continual learning onboard of a robot
Customer: Cyrille Berger
Context: the goal of this project is to allow to train a robot to detect an object during the execution of missions by the robot. Traditionnaly, machine learning algorithms are trained offline on a predefined dataset. This is inconvenient for deploying robot in the real world, the goal of this project is to allow the update of a detection algorithm. The idea is that if the robot fails to find an object, the operator can select a bounding box for this object in the image and then the model get updated.
Task suggestions:
- Study literature
- A framework for continual learning in ROS
- A simple approach with retraining a state of the art image detection algorithm
Operational constraints: integration with real platforms, and relatively slow hardware for training.
References:
- Incremental online learning of objects for robots operating in real environments
- Continual learning for robotics: Definition, framework, learning strategies, opportunities and challenges
- Online continual learning in image classification: An empirical survey
Bee traffic monitoring
Customer: Cyrille Berger
Context: bee traffic is an important health indicator for beehive. Traditional approach involves manual counting or expensive sensor solutions or complicated installation. The goal of this project is to develop a bee counter using a camera connected to a microcontroller.
Operational constraints: integration on a low cost, low computational embedded platform (maix bit or Raspberry PI).
References:
- Approaches, challenges and recent advances in automated bee counting devices: A review
- Machine Learning and Computer Vision Techniques in Continuous Beehive Monitoring Applications: A survey
- counting bees and source code
- dataset 1 and dataset 2
Previous years projects:
- 2022 (with source code and reports)
- 2021 (with source code and reports)
- 2020 (with source code and reports)
- 2019 (with source code and reports)
- 2018
- 2017
Page responsible: infomaster
Last updated: 2023-08-28