Word representations

To process words using neural networks, we need to represent them as vectors of numbers. In this unit you will learn different methods for learning these representations from data. The unit also introduces the idea of subword representations, and in particular character-level representations, which can be learned using convolutional neural networks.

Video lectures

Section	Title
2.01	Introduction to word representations
2.02	Word embeddings via matrix factorisation
2.03	Word embeddings via neural networks
2.04	The skip-gram model
2.05	Subword models
2.06	Contextualised word embeddings

Reading

• Eisenstein (2019), chapter 14
• Research article: Word Embeddings Quantify 100 Years of Gender and Ethnic Stereotypes

In-class session

Notebook on the CBOW classifier (launch on Binder)