Hide menu

729G17 Language Technology


The examination for this course is centred around the learning objectives as specified in the syllabus: For each objective, there are one or more examination components that test how well you meet that objective. This page presents these components and their specific knowledge requirements, forms of examination, and grade requirements in detail.

Examination components

The examination for this course consists of the following components:

  • TEN3 Written examination, 2 hp (U, G, VG)
  • LAB3 Practical assignments, 2 hp (U, G, VG)
  • UPG2 Project assignments, 2 hp (U, G, VG)

In order to pass the course, you must pass (grade G or higher) all three components. In order to pass with distinction (grade VG), you need distinctions in at least two of the components.

When will the grades be determined? For each attempted component, the examiner will make a decision about your grade at most 10 working days after the component’s formal examination date. To see the examination date for the written exam, you can go to the Student Portal. The examination date for the other components is the Friday of the week of the written exam. The decision about your final grade for the course will be made as soon as the grades for all components have been determined.

When does an examination component count as attempted? For the written exam, an examination attempt is when you sit the exam. For components with hand-in assignments, an examination attempt is when you submit at least one of the assignments.

Failing a component and re-examination. If you attempt but do not pass an examination, your grade for the respective component will be Fail (U). This does not only apply to the written examination, but also to the components with hand-in assignments. For each attempted component, there are two re-examination dates during the year following the course; see below for component-specific information.

Written examination

The written exam tests how well you are able to explain basic methods for the analysis and interpretation of words, sentences, and texts (learning objective 1). This includes relevant validation methods.

Knowledge requirements

Grade G. You demonstrate a thorough understanding of the basic methods that are covered in the course. You assess the applicability, requirements, and limitations of methods with simple judgements. You use relevant terminology and notation with some certainty.

Grade VG. You demonstrate a thorough understanding of the basic methods that are covered in the course, as well as several of the more advanced methods. In addition, you demonstrate good understanding of some method described in the scientific literature. You assess the applicability, requirements, and limitations of methods with well-developed judgements. You use relevant terminology and notation with certainty.

Assessment criteria for critical judgements

Form of the examination

The written exam consists of three parts:

Part A consists of 5 items, each worth 3 points. These items test your understanding of the basic methods that are covered in the course. They require only compact answers, such as a short text, calculation, or diagram.

Part B consists of 3 items, each worth 6 points. These items test your understanding of the more advanced methods that are covered in the course. They require detailed and coherent answers with correct terminology.

Part C consists of 1 item worth 12 points. This item is an essay question that tests your understanding of a method described in the scientific literature. To answer this question you will have to read a scientific article before sitting the exam.

The relevant article for the 2017 run of the course is:

David Ferrucci, Eric Brown, Jennifer Chu-Carroll, James Fan, David Gondek, Aditya A. Kalyanpur, Adam Lally, J. William Murdock, Eric Nyberg, John Prager, Nico Schlaefer, Chris Welty. Building Watson: An Overview of the DeepQA Project. AI Magazine 31(3):59–79, 2010.

Exams from previous runs of the course

Grade requirements

For grade G, you need at least 12 points in Part A. For grade VG, you additionally need at least 12 points in Part B, and at least 6 points in Part C. Note that you cannot use surplus points in one part to raise your score in another part.

Feedback

To get feedback about how well you meet the knowledge requirements for this component, you can answer the diagnostic questions in the study materials, attend the teaching sessions, and take the optional tests. You can also get feedback during the examiner’s office hours; see the timetable.

Optional tests. During the course you can take optional tests on individual topics. When you pass the test for a topic, you earn a ‘wildcard’ for the respective item in Part A of the written exam, which automatically gives you full credit for the item. You can use at most three (3) wildcards in total, and they are only valid for the first attempt at the written exam (not the re-exams).

Questions from the tests 2017 with sample solutions

Fast grading. In the interest of fast turnaround, we stop grading your exam when it becomes clear that you can no longer get a higher grade. If you want feedback on items that we have not graded, you are welcome to contact the examiner. Detailed solutions for items in Part A and Part B will be made available after the exam.

Re-examination

Please see the Student Portal for the dates of the written re-exams.

Hand-in assignments

The following general rules apply to the examination of hand-in assignments:

When you have completed an assignment, mail it to nlp-handin@ida.liu.se in the form specified in the instructions. You will receive an automatic confirmation and a ticket ID that you should use in all correspondence regarding your submission.

Format of the subject line. When you submit an assignment, we ask you to format the message’s subject line in a specific way such that the submission can be automatically routed to the right person. You will find details about this in the assignment’s instructions.

If you are asked to re-submit an assignment, you should send the new version as a reply to the confirmation that you received for the original submission, such that all correspondence about the submission is filed under the same ticket.

Each assignment has two due dates: The first date is stated in the instructions for the respective assignment. The second date is the examination date of the component that the assignment belongs to. This date is a hard deadline in the sense that you will not be able to submit or revise assignments after that date.

Why should you try to meet the first due date? If you meet the first due date, you will get formative feedback, which will give you a chance to revise your submission before the deadline (when your grade will be determined).

To meet a due date, it suffices to submit the assignment before 08:00 the first working day after that date. For example, if the due date is on a Friday, it suffices to submit the assignment before 08:00 the following Monday (assuming that this is a working day).

Practical assignments

The practical assignments test how well you are able to practically apply language technology methods and systems to texts and text collections (learning objective 2) and evaluate language technology components and systems using standard validation methods (learning objective 3).

Knowledge requirements

Grade G. You correctly apply the basic methods that are covered in the course according to given instructions. You make minor modifications to existing systems and implement simple functions. You interpret results and assess the applicability, requirements, and limitations of methods with simple judgements.

Grade VG. You correctly apply the basic methods that are covered in the course, as well as several of the more advanced methods, and use them to independently solve practical problems. You make extensive modifications to existing systems and implement complex functions. You interpret results and assess the applicability, requirements, and limitations of methods with well-developed judgements.

Assessment criteria for critical judgements

Form of the examination

The assignments of this component consist of lab assignments or ‘labs’, which should be done in pairs, and a reflection paper, which is individual. The labs come at three levels:

Level A consists of 5 labs that test your command of the basic methods that are covered in the course. These labs come with step-by-step instructions and ready-to-run systems. The instructions ask you to make minor modifications to these systems, evaluate the systems on text data, and interpret your findings.

Level B consists of 3 labs that test your command of some of the more advanced methods that are covered in the course. These labs are formulated as problem specifications and come with ready-to-run systems. To solve the specified problems, you will have to make significant modifications to the these systems, evaluate them on text data, and interpret your findings.

Level C consists of 2 labs that test your command of some of the more advanced methods that are covered in the course. These labs are formulated as problem specifications and come with little or no code. To solve the specified problems, you will have to implement your own limited-size language technology systems, evaluate them on text data, and interpret your findings.

Grade requirements

For grade G, you need a pass grade on all the labs at level A as well as on the reflection paper. For grade VG, you additionally need a pass grade on at least three labs at levels B or C.

Feedback

To get feedback about how well you meet the knowledge requirements for this component, you can attend the tutored lab sessions, and make sure that you submit your assignments in time for the first due date. You can also get feedback during the examiner’s office hours; see the timetable.

Re-examination

Re-examination of this component is offered in connection with the written re-exams (see above); the formal examination dates are the Fridays of the weeks of the written re-exams. The following procedure applies:

  • You register for the re-examination by sending an email to the examiner at least 10 working days before the relevant date.
  • At the same time, you submit all labs as if they were new assignments (rather than re-submissions).
  • You do not need to submit your reflection paper; you will present this paper orally during the examination.
  • The examiner confirms your registration and books a time with you in the week of the written re-exam.
  • At the examination, you give an oral presentation of your reflection paper, and answer to questions about specific labs.

Project assignments

The project assignments test how well you are able to judge the difficulty and the feasibility of language technology applications (learning objective 4). The assignments are centered around a group project on a language technology application of your choice. The main purpose of this project is to put you in a position to judge the difficulty and feasibility of the chosen application.

Knowledge requirements (group)

Grade G. In collaboration with the other members of your group, you identify, plan, and carry out a simple project that serves its main purpose to some extent. You present the project in a way that makes it reasonably clear what your conclusions are and how you arrived at these conclusions.

Grade VG. In collaboration with the other members of your group, you identify, plan, and carry out a varied project that serves its main purpose well. You present the project in a way that makes it clear what your conclusions are and how you arrived at these conclusions.

Knowledge requirements (individual)

Grade G. You assess the potential and the limitations of your project with simple judgements. You assess the difficulty and feasibility of the language technology application that you have investigated with simple judgements, making it reasonably clear how you arrived at these judgements based on your project.

Grade VG. You assess the potential and the limitations of your project with well-developed judgements and make suggestions for how the project could be improved to better serve its main purpose. You assess the difficulty and feasibility of the language technology application that you have investigated with well-developed judgements, making it clear how you arrived at these judgements based on your project.

Assessment criteria for critical judgements

Form of the examination

The project should be done in groups of approximately 5 students, and is examined by means of hand-in assignments and a project presentation. Some of these deliverables are group work, some of them are individual.

Your grade for this component is based on a weighted assessment of the following parts:

  • your pre-project reflection paper (10%)
  • your group’s project (30%)
  • your group’s presentation (20%)
  • your post-project reflection paper (40%)

Each of these parts is assessed according to criteria spelled out in the Project Rubric, and assigned a specified point score. Your grade is based on the sum of these scores.

Grade requirements

For grade G, you need at least 30 points in the Project Rubric. For grade VG, you need at least 44 points.

Feedback

Detailed information on how you get feedback about how well you meet the knowledge requirements for this component is available on the Project page. In addition to what is described there, you can also get feedback during the examiner’s office hours; see the timetable.

Re-examination

Re-examination of this component is offered in connection with the written re-exams (see above); the formal examination dates are the Fridays of the weeks of the written re-exams. The following procedure applies:

  • You register for the re-examination by sending an email to the examiner at least 10 working days before the relevant date.
  • At the same time, you submit all written group deliverables as if they were new assignments (rather than re-submissions).
  • You do not need to submit your reflection papers; you will present these papers orally during the examination.
  • The examiner confirms your registration and books a time with you in the week of the written re-exam.
  • At the examination, you give an oral presentation of your reflection papers, and answer to questions about your project.

Page responsible: Marco Kuhlmann
Last updated: 2017-01-11