Transformer-based models

Transformer-based models

Level

Basic (22 points)

In this lab, you will implement the encoder–decoder architecture of Sutskever et al., 2014, including the attention-based extension presented of Bahdanau et al., 2015, and evaluate this architecture on a machine translation task.

Link to the basic lab

BERT for Natural Language Inference

Level

Advanced (22 points)

One of the main selling points of pre-trained language models is that they can be applied to a wide spectrum of different tasks in natural language processing. In this lab you will test this by fine-tuning a pre-trained BERT model on a benchmark task in natural language inference.

Link to the advanced lab