A Dialogue Model for Efficient Multi-modal Information Support

The purpose of this research project is to develop theories for human computer interaction combing natural language dialogue with other modalities such as pointing, graphics and hypertext. This will be carried out in the domain of transport information and scheduling. The project will study human computer interaction systems and multi-modal systems utilizing natural language both as input modality and for generating user friendly responses. The proposal will begin with empirical studies of such human computer interaction at a local public transport company Östgötatrafiken but the results will be applied to other applications in the transport domain as well.

To be efficient it is necessary that the interfaces developed allow a user to formulate her information needs in a simple and intuitive manner. This is the main advantage of a natural language system, especially if able to cope with connected dialogue and enhanced with interpretation and generation mechanisms that integrate other modalities such as pointing, graphics and hypertext.

The main issue addressed in this project is: How are communicative modalities selected and combined to provide efficient interaction for different information needs? One aim is to develop a taxonomy for different kinds of information situations combined with guidelines for which different kinds of interaction methods are optimal in particular situations. There are two interrelated issues here; first the issue of which set of parameters are required for describing the different kinds of information situations, and, second, which interaction models are optimal given particular values on these dimensions. Within the project a demonstration system will be developed which combine the interactive flexibility of natural language with the expressive power of a multimedia system.

Arne Jönsson E-mail: arnjo@ida.liu.se

Back to Current Research