Hide menu

729A27 Natural Language Processing


The examination for this course is centred around the learning objectives as specified in the syllabus: For each objective, there are one or more examination components that test how well you meet that objective. This page presents these components and their specific knowledge requirements, forms of examination, and grade requirements in detail.

Examination components

The examination for this course consists of the following components:

  • TEN1 Written examination, 2 credits (U, G, VG)
  • LAB1 Practical assignments, 2 credits (U, G, VG)
  • UPG1 Project assignments, 2 credits (U, G, VG)

In order to pass the course, you must pass (grade G or higher) all three components. In order to pass with distinction (grade VG), you need distinctions in at least two of the components.

When will the grades be determined? For each attempted component, the examiner will make a decision about your grade within 10 working days after the course’s formal examination date. The examination date for the 2018 run of the course is 2018-03-16. The decision about your final grade for the course will be made as soon as the grades for all components have been determined.

When does an examination component count as attempted? An examination attempt is when you take an exam or submit at least one of the assignments that belong to that component.

Failing a component and re-examination. If you attempt but do not pass an examination, your grade for the respective component will be Fail (U). In this case you have the right to be re-examined. For each attempted component, the examiner offers two re-examination attempts during the year following the course; see the section on Re-exams for detailed information about this.

TEN1 Written examination

The written exam tests how well you are able to ‘explain state-of-the-art NLP algorithms and analyse them theoretically’ (learning objective 1).

Knowledge requirements

Grade G. You demonstrate a thorough understanding of the basic algorithms that are covered in the course. You assess the applicability, requirements, efficiency, and limitations of algorithms with simple judgements grounded in theoretical analysis. You use relevant terminology and notation with some certainty.

Grade VG. You demonstrate a thorough understanding of the basic algorithms that are covered in the course, as well as several of the more advanced algorithms. You assess the applicability, requirements, efficiency, and limitations of algorithms with well-developed judgements grounded in theoretical analysis. You use relevant terminology and notation with certainty.

Assessment criteria for critical judgements

Form of the examination

The written exam consists of three parts:

Part A consists of 5 items, each worth 3 points. These items test your understanding of the basic algorithms that are covered in the course. They require only compact answers, such as a short text, calculation, or diagram.

Part B consists of 3 items, each worth 6 points. These items test your understanding of the more advanced algorithms that are covered in the course. They require detailed and coherent answers with correct terminology.

Part C consists of 1 item worth 12 points. This item tests your understanding of an algorithm that has not been explicitly covered in the course. This item requires a detailed and coherent answer with correct terminology.

Exam from the previous run of the course (2017)

Grade requirements

For grade G, you need at least 12 points in Part A. For grade VG, you additionally need at least 12 points in Part B, and at least 6 points in Part C. Note that you cannot use surplus points in one part to raise your score in another part.

Feedback

To get feedback about how well you meet the knowledge requirements for this component, you can answer the diagnostic questions in the study materials, attend the teaching sessions, and take the optional tests. You can also get feedback during the examiner’s office hours (Wednesdays 13-17 in Building E, Room 3G.476).

Optional tests. During the course you can take optional tests on individual topics. When you pass the test for a topic, you earn a ‘wildcard’ for the respective item in Part A of the written exam, which automatically gives you full credit for the item. You can use at most three (3) wildcards in total, and they are only valid for the first attempt at the written exam (not the re-exams).

Questions from the tests 2017 with sample solutions

Fast grading. In the interest of fast turnaround, we stop grading your exam when it becomes clear that you can no longer get a higher grade. If you want feedback on items that we have not graded, you are welcome to book an appointment with the examiner. Detailed solutions for items in Part A and Part B will be made available after the exam.

LAB1 Practical assignments

The practical assignments test how well you are able to ‘implement NLP algorithms and apply them to practical problems’ (learning objective 2) and ‘design and carry out evaluations of NLP components and systems’ (learning objective 3).

Knowledge requirements

Grade G. You implement the basic algorithms that are covered in the course and apply them to practical problems according to given instructions. You make minor modifications to existing systems. You choose appropriate validation methods, use them to evaluate systems, and interpret your results with simple judgements.

Grade VG. You implement the basic algorithms that are covered in the course, as well as several of the more advanced algorithms, and use them to independently solve practical problems. You make extensive modifications to existing systems. You choose appropriate validation methods, use them to evaluate systems, and interpret results with well-developed judgements.

Assessment criteria for critical judgements

Form of the examination

The assignments of this component consist of lab assignments or ‘labs’, which should be done in pairs, and a reflection paper, which is individual. The labs come at three levels:

Level A consists of 6 labs that test your command of the basic algorithms that are covered in the course. These labs come with step-by-step instructions and ready-to-run systems. The instructions ask you to provide your own implementations of individual functions in these systems, evaluate the systems on text data, and interpret your findings.

Level B consists of 3 labs that test your command of some of the more advanced algorithms that are covered in the course. These labs are formulated as problem specifications and come with ready-to-run systems. To solve the specified problems, you will have to make extensive modifications to these systems, evaluate them on text data, and interpret your findings.

Level C consists of 2 labs that test your command of some of the more advanced methods that are covered in the course. These labs are formulated as problem specifications and come with little or no code. To solve the specified problems, you will have to implement your own limited-size systems, evaluate them on text data, and interpret your findings.

Grade requirements

For grade G, you need a pass grade on all the labs at level A as well as on the reflection paper. For grade VG, you additionally need a pass grade on at least three labs at levels B or C.

Feedback

To get feedback about how well you meet the knowledge requirements for this component, you can attend the tutored lab sessions, and make sure that you submit your assignments in time for the first due date. You can also get feedback during the examiner’s office hours (Wednesdays 13-17 in Building E, Room 3G.476).

UPG1 Project assignments

The project assignments test how well you are able to ‘seek, assess, and use scientific information within the area of NLP’ (learning objective 4). The assignments are centered around a group project in which you implement a realistic NLP system. The main purpose of this project is to give you an opportunity to show that you can make use of methods described in the NLP literature.

Knowledge requirements (group)

Grade G. In collaboration with the other members of your group, you plan and carry out a simple development project that serves its main purpose to some extent. You present the project in a way that makes it reasonably clear how you used NLP methods described in the scientific literature, what results you obtained with these methods, and how your interpret these results.

Grade VG. In collaboration with the other members of your group, you plan and carry out a varied development project that serves its main purpose well. You present the project in a way that makes it clear how you used NLP methods described in the scientific literature, what results you obtained with these methods, and how you interpret these results.

Knowledge requirements (individual)

Grade G. You assess the potential and the limitations of your project with simple judgements. You assess the applicability, requirements, and limitations of the NLP methods that you have used with simple judgements, making it reasonably clear how you arrived at these judgements based on your project.

Grade VG. You assess the potential and the limitations of your project with well-developed judgements and make suggestions for how the project could be improved to better serve its main purpose. You assess the applicability, requirements, and limitations of the NLP methods that you have used with well-developed judgements, making it clear how you arrived at these judgements based on your project.

Assessment criteria for critical judgements

Form of the examination

The project should be done in groups of approximately 6 students, and is examined by means of hand-in assignments and a project presentation. Some of these deliverables are group work, some of them are individual.

Your grade for this component is based on a weighted assessment of the following parts:

  • your pre-project reflection paper (10%)
  • your group’s project (30%)
  • your group’s presentation (20%)
  • your post-project reflection paper (40%)

Each of these parts is assessed according to criteria spelled out in the Project Rubric, and assigned a specified point score. Your grade is based on the sum of these scores.

Grade requirements

For grade G, you need at least 30 points in the Project Rubric. For grade VG, you need at least 44 points.

Feedback

Detailed information on how you get feedback about how well you meet the knowledge requirements for this component is available on the Project page. Apart from by the forms described there, you can also get feedback during the examiner’s office hours (Wednesdays 13-17 in Building E, Room 3G.476).

Rules for hand-in assignments

The following general rules apply to the submission and examination of hand-in assignments:

When you have completed an assignment, mail it to nlp-handin@ida.liu.se. You will receive an automatic confirmation and a ticket ID that you should use in all correspondence regarding your submission.

Format of the subject line. When you submit an assignment, we ask you to format the message’s subject line in a specific way such that the submission can be automatically routed to the right person. You will find details about this in the assignment’s instructions.

Each assignment has two due dates: The first date is stated in the instructions for the assignment. The second date is the course’s examination date (2018-03-16). This date is a hard deadline in the sense that you will not be able to submit or revise assignments after that date.

Why should I try to meet the first due date? If you meet the first due date, you will get formative feedback on your assignment and the chance to revise it before the course’s formal examination date.

To meet a due date, it suffices to submit the assignment before 08:00 the first working day after that date. For example, if the due date is on a Friday, it suffices to submit the assignment before 08:00 the following Monday (assuming that this is a working day).

Re-examination

If you fail an examination, you can repeat it on two occasions during the year following the course. The rules for the re-examination differ from component to component.

TEN1 Written examination

Please see the Student Portal for the dates of the written re-exams.

LAB1 Practical assignments

Re-examination of this component is offered in connection with the written re-exams (see above); the formal examination dates are the Fridays of the weeks of the written re-exams. The following procedure applies:

  • Submit or re-submit all labs that you do not yet have a pass grade on, as well as all other labs that you want to get graded.
  • Do not submit your reflection paper; you will be asked to present this paper orally during the examination.
  • Register for the re-examination at least 10 working days before the relevant examination date.
  • Be prepared to give an oral presentation of your reflection paper, and answer questions about specific labs.

UPG1 Project assignments

Re-examination of this component is offered in connection with the written re-exams (see above); the formal examination dates are the Fridays of the weeks of the written re-exams. The following procedure applies:

  • Register for the re-examination at least 10 working days before the relevant examination date.
  • Be prepared to give an oral presentation of your reflection papers, and answer questions about your project.

Page responsible: Marco Kuhlmann
Last updated: 2017-09-22