729G17 Language Technology
The main purpose of the project is to put you in a position to make judgements about the difficulty and feasibility of language technology applications (learning objective 4). A secondary purpose of the project is to give you an opportunity to deepen and apply the knowledge that you have acquired in the course.
The project should be carried out in groups of approximately 5 students. The minimal project looks as follows:
- Choose a language technology application (example: question answering)
- Choose a concrete system that implements the chosen application (example: mini_qa)
- Evaluate the chosen system with respect to relevant properties (example: correctness, efficiency)
- Draw conclusions about the difficulty and feasibility of the chosen application
More advanced projects will typically involve more elaborate evaluations (such as system comparisons or user studies) and/or own system development. However, the goal with the project should always be to help you assess the difficulty and feasibility of the chosen language technology application in general, not merely a specific system.
Choice of topic
You can freely choose both the language technology application and the specific system that you want to investigate. To get some inspiration for the choice of your project topic, you can have a look at the abstracts of previous projects (mostly in Swedish).
The project runs W4–W10, but most of the work is concentrated during the project week in W9. When you plan your time for the project, you should calculate approximately 48 hrs per group member, that is, 240 hrs for a group with 5 members.
When you plan your time for the project week (W9), you should calculate approximately 16 hrs per group member, that is, 80 hrs in total for a group with 5 members.
While the choice of project topic is completely up to your group, the form of the project implementation is rather rigid. In particular, throughout the project you will have to make seven deliverables (D1–D7), which are designed to keep you on track and give you feedback on your progress. The rest of this page contains detailed information about these deliverables.
D1: Group contract
During W4–W5 you will form your project group. We encourage you to form groups that include students with different backgrounds, skills, and interests, as this can improve the quality of the project. In particular, consider inviting IP students to your group.
After formation, your group is required to make a group contract that will govern your collaboration. The contract should spell out those behaviours that you expect of all group members, as well as procedures for resolving impasses in the group. Specific questions to think about include:
- How will we communicate with each other? At what times?
- How often and where will we meet?
- How will we make sure that our meetings are productive?
- What will we do if somebody does not show up at a meeting?
- What will we do if somebody breaks any rule set out in this contract?
Instructions: Make a group contract and have it signed by all members of the group. Submit the signed contract as a PDF document. (Please re-read the instructions for the submission of hand-in assignments before doing so.) Due date: 2017-02-03
Format of the subject line: LT-2018 D1 marku61
Upon receiving your group contract, we will assign your group a group ID that you should use in future submissions (see below).
D2: Project plan
During the first four weeks of the project (W4–W7), your group should meet at least once a week to plan the project. At the end of this phase, your group is required to hand in a project plan that should answer the following questions:
- Which language technology application do we want to investigate?
- Which specific system do we want to evaluate?
- How will we interact with the system? (For example, is there a web interface, or do we have to run a program?)
- How will we evaluate the system? (What are our evaluation criteria and our evaluation method?)
- How much system development will we need to do?
- How much annotation work (production of data) will we need to do?
- How will our project serve its purpose? (Make a connection to the learning objective.)
- Who will be responsible for what in the project?
Instructions: Write a project plan answering the above questions and submit it as a PDF document. Due date: 2017-02-17
Format of the subject line: LT-2018 D2 your group ID marku61
Example: LT-2018 D2 G1 marku61
Feedback: You will get feedback on your project plan from the examiner. You can use this feedback to understand to what degree your planned project meets the project-related assessment criteria in the Project Rubric.
D3: Pre-project paper
Apart from the project plan, you are required to submit an individual paper with your reflections ahead of the project. In this paper you should address the following questions:
- What is your group’s project about? (Summarise the project plan in your own words.)
- How will your group’s project help you to accomplish the relevant learning objective?
- What knowledge and what abilities will you personally be able to contribute to the project?
Instructions: Write a paper addressing the above questions. The length of your paper should be around 500 words (approximately 1 page). Submit your paper as a PDF document. Due date: 2017-02-17
Format of the subject line: LT-2018 D3 your LiU-ID marku61
Example: LT-2018 D3 marjo999 marku61
Feedback and examination: You will get feedback on your paper from the examiner, who will assess it according to the criteria spelled out in the Project Rubric. This assessment will contribute to your grade for the examination component. At the same time, the feedback will be useful to you when preparing your post-project paper (see below).
D4: Project abstract
At the end of the project week (W9), your group is required to submit a short abstract of your project. The abstract should summarise what you have done in the project, as well as your main results. The purpose of the abstract is to announce your presentation ahead of the ‘mini-conference’. To get some inspiration, you can have a look at the abstracts of previous projects (in Swedish).
Instructions: Prepare a short abstract of your project (max. 200 words) and submit it as plain text. Due date: 2017-03-03
Format of the subject line: LT-2018 D4 your group ID marku61
Example: LT-2018 D4 G1 marku61
D5: Project presentation
In the week following the project week (W10), your group will present your project at the course’s ‘mini-conference’. You are allotted a 15 minute time slot for this presentation. You are free to choose the presentation’s content and structure, but you should bear in mind that the presentation needs to be understandable to everybody in the course.
In preparing the presentation, you may want to consider the following questions:
- What have you done in this project? Which application, which system did you choose?
- Why have you chosen this particular project?
- How does the system that you studied work?
- How have you evaluated the system?
- What are your conclusions regarding the difficulty and the feasibility of the application?
Instructions: Present your project, following the instructions above. Date of the presentation: 2017-03-08, 13–15. Please reserve the full four hours. The exact schedule for the mini-conference will be announced at the beginning of W10.
Feedback and examination: The examiner will assess your presentation according to the criteria spelled out in the Project Rubric. This assessment will contribute to your grade for the examination component. At the same time, the feedback will be useful to you when preparing your post-project paper.
What if your group cannot present at the mini-conference? The group presentation can be replaced by a written report in which your group presents your project. Please contact the examiner for details about this assignment.
D6: Peer feedback
In addition to giving a presentation yourself, your group will give peer feedback on another group’s presentation. Your feedback should make it clear which aspects of the presentation you found most interesting, and which parts you believe can be improved upon. For the feedback to be holistic, each member of your group should be in charge of a specific task:
- Summarise the presented project in at most three sentences.
- Presentation of the application and the system: Which parts were clear? Which parts could be made even more clear?
- Presentation of the evaluation: Which results were most interesting? Which other aspects could be evaluated?
- Presentation technique: What worked well? What could be improved?
- Based on the presentation, draw your own conclusion regarding the difficulty and feasibility of the chosen application.
Instructions: Distribute the tasks among the members of your group and give feedback according to the task descriptions at the mini-conference. All feedback should be constructive.
What if you cannot be present at the mini-conference? The peer review can be replaced by an individual assignment in which you interview the members of another group about their project. Please contact the examiner for details about this assignment.
D7: Post-project paper
The final project-related assignment is an individual paper in which you reflect on the project and draw conclusions from it. The form of this paper should be a coherent essay that assesses the usefulness of the method that you have investigated in your project, taking into account the potential and the limitations of the project itself. When writing this paper, you can think about the following questions:
- What was your group’s project about? (If applicable, you may recycle text from your pre-project paper.)
- To what extent did your group’s project help you to accomplish the relevant learning objective?
- What knowledge and what abilities were you personally able to contribute to the project?
- Based on your group’s project, how would you judge the difficulty and feasibility of the application that you investigated?
Note that forward-looking versions of the first three of these questions also appear in the pre-project paper. The purpose of the post-project paper is to look back and reflect on whether everything worked out as anticipated. The paper should not assume the reader to have read your pre-project paper but should be independent.
Instructions: Write a paper according to the above specification. Make sure to take into account both the feedback that you got on your pre-project paper and your group’s presentation. The length of your paper should be around 1,500 words (approximately 3 pages). Submit your report as a PDF document. Due date: 2017-03-17
Format of the subject line: LT-2018 D7 your LiU-ID marku61
Example: LT-2018 D7 marjo999 marku61
Examination: The examiner will assess your paper according to the criteria spelled out in the Project Rubric. This assessment will contribute to your grade for the examination component.
Page responsible: Marco Kuhlmann
Last updated: 2017-09-22