Hide menu

729A27 Natural Language Processing


The examination for this course center around the learning objectives: For each objective, there are one or more examination components that test how well you meet that objective. This page presents these components, their specific knowledge requirements, and forms of examination in detail.

Examination components

The examination for this course consists of the following components:

  • TEN1 Written examination, 2 credits (U, G, VG)
  • LAB1 Practical assignments, 2 credits (U, G, VG)
  • UPG1 Project assignments, 2 credits (U, G, VG)

In order to pass the course, you must pass (grade G or higher) all three components. In order to pass with distinction (grade VG), you need distinctions in at least two components.

When will the grades be determined? For each attempted component, the examiner will make a decision about your grade within 10 working days after the component’s formal examination date. The decision about your final grade for the course will be made as soon as the grades for all components have been determined. The last ordinary examination date for the course is 2018-03-17.

When does an examination component count as attempted? An examination component counts as attempted when you take an exam or submit an assignment that belongs to that component.

Failing a component and re-examination. If you attempt but do not pass an examination, your grade for the respective component will be Fail (U). In this case you have the right to be re-examined. For each attempted component, the examiner offers two re-examination attempts during the year following the course; see the section on Re-examination for detailed information about this.

TEN1 Written examination

The written exam tests how well you are able to explain state-of-the-art NLP algorithms and analyse them theoretically (learning objective 1).

Knowledge requirements

Grade G. You demonstrate a thorough understanding of the basic algorithms that are covered in the course. You assess the applicability, requirements, and limitations of algorithms with simple judgements grounded in theoretical analysis. You use relevant terminology and notation with some certainty.

Grade VG. You demonstrate a thorough understanding of the basic algorithms that are covered in the course, as well as several of the more advanced algorithms. You also show that your understanding extends to algorithms that have not been explicitly covered in the course. You assess the applicability, requirements, and limitations of algorithms with well-developed judgements grounded in theoretical analysis. You use relevant terminology and notation with certainty.

Assessment criteria for critical judgements

Form of the examination

The written exam consists of three parts:

Part A consists of 5 items, each worth 3 points. These items test your understanding of the basic algorithms that are covered in the course. They require only compact answers, such as a short text, calculation, or diagram.

Part B consists of 3 items, each worth 6 points. These items test your understanding of the more advanced algorithms that are covered in the course. They require detailed and coherent answers with correct terminology.

Part C consists of 1 item worth 9 points. This item tests your understanding of an algorithm that has not been explicitly covered in the course. This item requires a detailed and coherent answer with correct terminology.

Exams from previous runs of the course

Grade requirements

For grade G, you need at least 12 points (80%) in Part A. For grade VG, you additionally need at least 14 points (80%) in Part B, and at least 7 points (80%) in Part C.

Note that you cannot use surplus points in one part to raise your score in another part.

Examination date and registration

The formal examination date for this component is the date of the written exam, which you will find in the Student Portal. In order to take the written exam, you have to register for it. The last day to register is 10 days before the exam.

Feedback

To get feedback about how well you meet the knowledge requirements for this component, you can answer the diagnostic questions in the study materials, attend the lectures, and take the optional tests. In addition, you can always get personalised feedback from the examiner (drop-in office hours during term time – or book an appointment).

Fast grading. In the interest of fast turnaround, we stop grading your exam when it becomes clear that you can no longer get a higher grade. If you want feedback on items that we have not graded, you are welcome to contact the examiner. Detailed solutions for items in Part A and Part B will be made available after the exam.

KTR1 Optional tests

During the course you can take optional tests on individual topics. When you pass the test for a topic, you earn a ‘wildcard’ for the respective item in Part A of the written exam, which automatically gives you the full number of points for the item. We will credit you with at most three (3) wildcards in total, and they are only valid for the first attempt at the written exam (not the re-exams).

You will find the dates, times, and locations for the optional tests in the timetable. You do not need to register for the optional tests.

Optional tests 2018 with sample solutions

LAB1 Practical assignments

The practical assignments test how well you are able to implement NLP algorithms and apply them to practical problems (learning objective 2) and design and carry out evaluations of NLP components and systems (learning objective 3).

Knowledge requirements

Grade G. You implement the basic algorithms that are covered in the course and apply them to practical problems according to given instructions. You make minor modifications to existing systems. You choose appropriate validation methods, use them to evaluate systems, and interpret your findings with simple judgements. You make reflections on your learning.

Grade VG. You implement the basic algorithms that are covered in the course, as well as several of the more advanced algorithms, and use them to independently solve practical problems. You make extensive modifications to existing systems. You choose appropriate validation methods, use them to evaluate systems, and interpret your findings with well-developed judgements. You make reflections on your learning.

Form of the examination

The assignments of this component consist of lab assignments or ‘labs’, which should be done in pairs, and a reflection paper, which is individual. The labs come at three levels:

Level A consists of 6 labs that test your command of the basic algorithms that are covered in the course. These labs come with step-by-step instructions and ready-to-run systems. The instructions ask you to provide your own implementations of individual functions in these systems, evaluate them on text data, and interpret your findings.

Level B consists of 3 labs that test your command of some of the more advanced algorithms that are covered in the course. These labs are formulated as problem specifications and come with ready-to-run systems. To solve the specified problems, you will have to make extensive modifications to these systems, evaluate them on text data, and interpret your findings.

Level C consists of 2 labs that test your command of some of the more advanced algorithms that are covered in the course. These labs are formulated as problem specifications and come with little or no code. To solve the specified problems, you will have to implement your own limited-size systems, evaluate them on text data, and interpret your findings.

The labs are examined based on written lab reports, so-called notebooks.

The reflection paper is an academic essay in which you reflect on your learning during the lab series. The paper should contain a description and an analysis of your work on the labs, and conclusions regarding your learning.

Grade requirements

For grade G, you need a Pass on all the labs at level A as well as on the reflection paper. For grade VG, you additionally need a Pass on at least three labs at levels B or C.

Examination date and registration

The formal examination date for this component is the final examination date for the course, 2018-03-17. Before submitting your first lab, you and your lab partner will have to register in Webreg.

Feedback

To get feedback about how well you meet the knowledge requirements for this component, you can attend the tutored lab sessions, and make sure that you submit your assignments in time for the first due date. In addition, you can always get personalised feedback from the examiner (drop-in office hours during term time – or book an appointment).

UPG1 Project assignments

The project assignments primarily test how well you are able to seek, assess, and use scientific information within the area of NLP (learning objective 4). They also test how well you are able to implement NLP algorithms and apply them to practical problems (learning objective 2) and design and carry out evaluations of NLP components and systems (learning objective 3).

The assignments center around a group project in which you implement and evaluate a method described in the NLP literature.

Knowledge requirements

Grade G. In collaboration with the other members of your group, you plan and carry out a simple project based on scientific information from the area of natural language processing, and present the project in a way that makes it clear what method you implemented, what results you obtained, and how your interpret these results. You assess the usefulness of NLP methods with simple judgements and make simple reflections on your project and your own learning.

Grade VG. In collaboration with the other members of your group, you plan and carry out a complex project based on scientific information from the area of natural language processing, and present the project in a way that makes it very clear what method you implemented, what results you obtained, and how your interpret these results. You assess the usefulness of NLP methods with well-developed judgements and make well-developed reflections on your project and your own learning.

Form of the examination

The project should be done in groups of approximately 4 students and is examined based on hand-in assignments and a project presentation. Some of these deliverables are group work, some are individual. Your grade is based on the following three parts:

  • pre-project paper (individual)
  • project contents and presentation (group)
  • post-project paper (individual)

Each of these parts is assessed according to criteria spelled out in the Project Rubric.

Grade requirements

For a pass grade, you must have a pass grade (grade G or higher) on all three parts. In order to pass with distinction (grade VG), you need distinctions on at least two of the parts.

Examination date and registration

The formal examination date for this component is the final examination date for the course, 2018-03-17. You register for the examination by submitting the first deliverable.

Feedback

Detailed information on how you get feedback about how well you meet the knowledge requirements for this component is available on the Project page. In addition, you can always get personalised feedback from the examiner (drop-in office hours during term time – or book an appointment).

Rules for hand-in assignments

The following general rules apply to the submission and examination of hand-in assignments:

When you have completed an assignment, email it to nlp-handin@ida.liu.se. You will receive an automatic confirmation and a ticket ID that you can use in all correspondence regarding your submission.

Format of the subject line. When you submit an assignment, you need to format the message’s subject line in a specific way such that the submission can be automatically routed to the right person. You will find details about this in the assignment’s instructions.

Each assignment has two due dates: The first date is stated in the instructions for the assignment. The second date is the final examination date for the course, 2018-03-17. This date is a hard deadline in the sense that you will not be able to submit or revise assignments after that date.

Why should I try to meet the first due date? If you meet the first due date, you will get formative feedback on your assignment and the chance to revise it before the final examination date.

To meet a due date, it suffices to submit the assignment before 08:00 the first working day after that date. For example, if the due date is on a Friday, it suffices to submit the assignment before 08:00 the following Monday (assuming that this is a working day).

Re-examination

New rules effective 2018-06-19

There are two occasions for re-examination:

  • during the re-exam period for VT1 (last day: 2018-06-08)
  • during the general re-exam period after the summer break (last day: 2018-09-01)

After this you can still be examined in connection with the next run of the course. Note however that the next run can have different content, different assignments, and different examination requirements.

The following specific rules apply for the different examination components.

TEN1 Written examination

Please see the Student Portal for the dates of the written re-exams. In order to take a written re-exam, you need to register for it. The last day to register is 10 days before the date of the re-exam.

LAB1 Practical assignments

To take a re-examination of this component, you need to do the following:

  • Submit (or re-submit) all hand-in assignments (labs, reflection paper) that you do not already have a Pass on.
  • Register for the re-examination by sending an email to the examiner stating that you want to be re-examined.

The deadline for both the submissions and the registration is the last day of the relevant re-examination period.

When you submit an assignment for a re-examination, you need to follow the same instructions as when submitting it while the course is ongoing. Please read the Rules for hand-in assignments.

UPG1 Project assignments

To take a re-examination of this component, you need a Pass on the project (D5). If you do not already satisfy this requirement, you must consult with the examiner. The last day to contact the examiner is 10 days before the end of the relevant re-exam period.

Assuming that you have a Pass on the project (D5), you need to do the following:

  • Submit (or re-submit) all reflection papers (pre-project paper, post-project paper) that you do not already have a Pass on.
  • Register for the re-examination by sending an email to the examiner stating that you want to be re-examined.

The deadline for both the submissions and the registration is the last day of the relevant re-examination period.

When you submit an assignment for a re-examination, you need to follow the same instructions as when submitting it while the course is ongoing. Please read the Rules for hand-in assignments.


Page responsible: Marco Kuhlmann
Last updated: 2018-01-12