Hide menu


Our work ranges from basic research on algorithms and machine learning to applied research in language technology.

STING: Synthesis and analysis with Transducers and Invertible Neural Generators

Human communication is multimodal in nature, and occurs through combinations of speech, language, gesture, facial expression, and similar signals. To enable natural interactions with human beings, artificial agents must be capable of both analysing and producing these rich and interdependent signals, and connect them to their semantic implications. Unfortunately, even the strongest machine learning methods currently fall short of this goal: automated semantic understanding of human behaviour remains superficial, and generated agent behaviours are empty gestures lacking the ability to convey meaning and communicative intent.

The STING NEST intends to change this state of affairs by uniting synthesis and analysis with transducers and invertible neural models. This involves connecting concrete, continuous­valued sensory data such as images, sound, and motion, with high­level, predominantly discrete, representations of meaning, which has the potential to endow synthesis output with human­understandable high­level explanations, while simultaneously improving the ability to attach probabilities to semantic representations. The bi­directionality also allows us to create efficient mechanisms for explainability, and to inspect and enforce fairness in the models.

Recent advances in generative models suggest that our ambitious research agenda is likely to be met with success. Normalising flows and variational autoencoders permit both extracting disentangled representations of observations, and (re­)generating observations from these abstract representations, all within a single model. Their recent extensions to graph­structured data are of particular interest because graphs are commonly­used semantic representations. This opens the door not only to generating structured information, but also to capturing the composition of the generation itself (which is a graph in its own right) by exploiting and transferring techniques from finite­state transducers and graph grammars.

In addition to its scientific value, the project is expected to have a substantial societal imprint. The resulting technologies may, e.g., be used to create virtual patients for medical training, to model non­playable characters in video games, and to derive affective states and underlying health issues from human speech and non­verbal behaviour.

A Practical Theory of Computation for Modern Neural Network Architectures

Neural networks are today the most important computational paradigm for AI and machine learning. While their computational power has been studied for decades, the theoretical development has not been able to keep up with the recent upshot of new neural architectures, such as the Transformer. Also, existing theoretical results are of limited practical value because they often apply to monolithic architectures, whereas applications typically use networks that are composed of re-usable components. Moreover, theoretical results tend to make idealising assumptions such as infinite precision or unlimited training time, rather than acknowledging that such commodities are in short supply and should thus be treated as parameters of expressive power. In this project, we want to address these shortcomings and develop a practical theory of computation for modern neural network architectures. The project combines methods from theoretical computer science – especially the theory of computation and formal languages – with empirical validation in natural language processing (NLP).

Relation Extraction with Deep Neural Language Models

The field of natural language processing (NLP) has seen major progress during the last few years with the development of deep neural language models, which learn tasks such as question answering, machine translation, and text summarization without any explicit supervision. This project will apply these models to the task of extracting semantic relations between named entities from raw text. Our main goal is to design, implement, and evaluate an end-to-end system for relation extraction based on deep neural language models. Because training these models from scratch is extremely resource-intensive, we are specifically interested in developing methods for maximizing the performance that can be obtained by fine-tuning pre-trained models, and in particular models for smaller languages such as Swedish.

Interpreting and Grounding Pre-Trained Representations for Natural Language Processing

Building computers that understand human language is one of the central goals in artificial intelligence. A recent breakthrough on the way towards this goal is the development of neural models that learn deep contextualized representations of language. However, while these models have substantially advanced the state of the art in natural language processing (NLP) for a wide range of tasks, our understanding of the learned representations and our repertoire of techniques for integrating them with other knowledge representations and reasoning facilities remain severely limited. To address these gaps, we will develop new methods for the interpretation, grounding, and integration of deep contextualized representations of language, and to evaluate the usefulness of these methods in the context of threat intelligence applications together with our industrial partner, Recorded Future.

Semantic Parsing for Text Analytics

Much of what we know and think is expressed as written text, and more and more of it is available in digital form: on personal computers, corporate networks, and on the Internet. This project will develop new techniques for transforming textual data into structured information, and eventually, into actionable intelligence. The process, which involves information retrieval, natural language processing, and machine learning, is known as text analytics. One of the key component technologies in text analytics is semantic parsing, the automatic mapping of a sentence into a formal representation of its meaning. This project will initially focus on meaning representations in the form of dependency graphs. Parsing to dependency graphs has been a very active research area in the past few years. In spite of this, several important research questions remain unresolved. This project will specifically address the following: How can we develop semantic parsers that strike a good balance between accuracy and efficiency? How can we learn semantic parsers from large amounts of data without human intervention, and how can we handle the dynamicity of that data? How can we integrate semantic parsers into systems with existing forms of structured data, such as relational databases or models of software systems?

Page responsible: Marco Kuhlmann
Last updated: 2021-11-12