Project financed by:  

Multimodal Interaction for Information Services


Research goals

Research Issues


Project work plan

Scientific production

Project prototypes




Project Work Plan

The first phase of the project is mainly concerned with development of software for various subtasks such as for conducting empirical investigations, and tools for creating information sources. It also includes studies of ontologies and investigations of information needs. The second phase is more focused on building a first integrated dialogue system application and developing knowledge sources. It also includes analyses of a first unimodal corpus and development of a multi-modal corpus. Finally, in the third phase, the multimodal dialogue system will be refined and evaluated. We will also make the resources developed within this project available in an Open Source code repository.

Nov 2001-Oct 2002

  • Development of an experimental environment for investigations on control and cooperation in multimodal interactions.
  • Collection of a unimodal question corpus as a basis for understanding the kind of information that needs to be identified and processed in the document collection.
  • Development of methods for utilising taggers, lemmatisers and dependency analyzers in building knowledge and information sources.
  • Incorporation of results from empirical study at RSV into the prototype RSV Information System.
  • First investigation into creation of ontologies, a small ontology will be developed for a restricted domain.

    Nov 2002-Oct 2003

  • Investigations on control and cooperation in multimodal interactions.
  • Categorised analysis of the unimodal question corpus: question and answer types. Perspectival differences between concepts used in questions and the concepts in the document base.
  • Collection of a multimodal question corpus including analysis.
  • Development of language technology modules for automatic extraction/generation of information and knowledge sources.
  • Development of language technology modules for interpretation of questions.
  • Development of a first version of an integrated dialogue system for one domain.
  • Study of ontological issues, domain concepts and representation, in regards to machine learning.

    Nov 2003-Oct 2004

  • Continued work on multimodal interactions, interpretation of questions, and ontological issues.
  • Creation of a collection of resources, such as wrappers, lexicons, etc.
  • Further development of the integrated dialogue system, including work on dialogue and domain knowledge management in the \malin\ dialogue system framework.
  • Evaluation of the integrated dialogue system.

  • Page responsible: Webmaster
    Last updated: 2012-05-07