The purpose of this research project is to develop theories for human computer interaction combing natural language dialogue with other
modalities such as pointing, graphics and hypertext. This will be carried out in the domain of transport information and scheduling. The project
will study human computer interaction systems and multi-modal systems utilizing natural language both as input modality and for generating user
friendly responses. The proposal will begin with empirical studies of such human computer interaction at a local public transport company
Östgötatrafiken but the results will be applied to other applications in the transport domain as well.
To be efficient it is necessary that the interfaces developed allow a user to formulate her information needs in a simple and intuitive manner.
This is the main advantage of a natural language system, especially if able to cope with connected dialogue and enhanced with interpretation
and generation mechanisms that integrate other modalities such as pointing, graphics and hypertext.
The main issue addressed in this project is: How are communicative modalities selected and combined to provide efficient interaction for
different information needs? One aim is to develop a taxonomy for different kinds of information situations combined with guidelines for which
different kinds of interaction methods are optimal in particular situations. There are two interrelated issues here; first the issue of which set of
parameters are required for describing the different kinds of information situations, and, second, which interaction models are optimal given
particular values on these dimensions. Within the project a demonstration system will be developed which combine the interactive flexibility of
natural language with the expressive power of a multimedia system.
