|
The Wallenberg laboratory
for research on
Information Technology
and Autonomous Systems
|
Demonstrated System - December 1998
The WITAS project started on January 1, 1997, and is planned for
a first phase of three years (1997-1999) and a second phase of four
years (2000-2003).
The project has two goals with equal dignity:
- Construct a particular kind of system -- the onboard IT system
whereby an unmanned aerial vehicle can "think" for itself --
for demonstration by the end of the year 2003.
- Conduct disciplinary research on topics that are of importance in the
international research community, and that are relevant for the
system being built. This research involves both theory and
methodology.
It is this combination of building a complex system and doing
disciplinary research that gives the WITAS project its special
character. The present memorandum summarizes the results towards
the systems goal during WITAS Phase I. Additional details and
descriptions of other aspects of the project are found at the
WITAS website,
http://www.ida.liu.se/ext/witas/
Background and general directions for the systems development
The WITAS developed system must then consist of the following essential
components or subsystems:
- On-board computer system
- Computer vision system (video and IR cameras, camera housing for
stabilization, etc)
- Other on-board sensors
- Communication with an operator on the ground, or in another vehicle.
Such a flying vehicle is usually called a UAV, for Unmanned Aerial
Vehicle. The UAV in our project is intended for a particular
operational environment, namely, for roads carrying automobile traffic.
It is therefore required to "understand" what
happens on those roads - conventional maneuvers of individual cars and
other road vehicles, dangerous or otherwise exceptional maneuvers,
structure of the traffic e.g. congestion. It is also to perform
tasks that are assigned by the operator or triggered by the observations
it makes itself, for example to follow a certain car that flees from the
scene of an apparent crime, or to assist a certain car so that it can make
it through difficult traffic and get to a particular destination as quickly
as possible, or to deliver a particular parcel to a particular point.
The UAV is supposed to perform these functions autonomously, that is,
without the direct intervention of a human operator. It is therefore
not sufficient to design it for remote control of its maneuvers
and other detailed operations: the operator is only supposed to
communicate general commands, often using a combination of a phrase in
natural language, and pointing to a map or video image. The most
important capabilities for such a system are therefore (1) to form
a model of ("understand") scenes and events that it observes on the ground,
and (2) prediction, planning, and autonomous decisions using that
model.
System Behavior Goals
For a project of this kind it is important to have a clear definition
of the system one intends to build. The system can be defined in various
ways (physical characteristics, what are the major subsystems, and so
on), but the most important is the specification of the behavior
that the developed system is required to exhibit and the scenarios
that it is required to master. These definitions were made already
in the original project proposal from 1996, where they were stated as
follows:
- Fly over a congested highway system (typically in a city)
and detect traffic accidents by the pattern of how cars are standing
and/or moving
- Fly over a congested highway system and find the best route
for an emergency vehicle (avoiding traffic jams)
- Fly over roads in a segment of countryside where there have been
hostilities (but fighting has stopped), identify how roads have been
blocked (by obstructing vehicles, destroyed bridges, etc), and
identify a route where trucks can pass
- ``Watch that car'' -- fly over a particular car (for example
a high-value transport) and supervise what happens around it. If
there seems to be any trouble, then fly down and take close-up
photographs; also report back to a communication center.
- ``Follow that car'' -- if there has been an incident in
the activity of the previous item, then follow a particular
fleeing car and check where it goes. If the people in it
switch to yet another car, then watch what happens and follow
the (a) car that moves rather than (the) one that stays.
- Interception analysis: In the context of the car-watch and
car-following activities, do lookahead across the road network to
predict if the watched car or the fleeing car will be forced
to stop because of an obstacle. Also, identify ways of stopping it
using available, cooperating vechicles (police cars, etc).
The original project proposal continued as follows:
"The proposal, then, is to develop an IT system (sensors, computer
hardware and software, radio communication) where all of these
functions are performed automatically: the on-board vision sensors
observe the roads and situations under the aeroplane and analyze
the images, and the decision system specifies where to fly and which
communication to have with the base. (...)"
Essential aspects of the end-1998 demo
Since the project started on January 1, 1997, we have had
development milestones in December 1997, May 1998, and December 1998.
For each milestone we produced a video showing the accomplishments
of the system that had been developed so far.
During calendar year 1999 we put a larger emphasis on theoretical and
other foundational work. With the start of Phase II in year 2000
we have come back to system implementation issues, including the
building of the on-board system for actual (non-simulated) operation.
This is however outside the scope of the present memo.
The following are the essential aspects of the demo completed in
December, 1998:
- A simulated UAV flies in a simulated operational
environment ("virtual reality")
of the kind that is defined in the WITAS challenge, that is,
over a system of roads where cars move around.
- The UAV simulation is based on a quite precise, mathematical model
of a helicopter, which has been obtained from contemporary literature
and under the direction of an expert from Saab Dynamics, civ.ing. Erik
Skarman.
- The control system of the simulated UAV contains the required levels
of control, up to a "deliberative" layer where the system is able to
predict the possible actions of cars that it observes, and to
take "intelligent" action with respect to them.
- The simulated UAV also contains a computer vision system that analyzes
images that are obtained from the virtual reality. The vision system
provides the results of that analysis to the cognitive layers of the
architecture. It makes use of new advances in computer vision technology.
Parts of it have been structured as a set of relatively
independent "skills", thus moving away from the traditional concept of
a monolithic vision system. In this way the higher levels of the WISS
architecture are able, more than before, to control
the allocation of computational resources within the scene understanding
process.
- The computer vision system has been developed on the basis of research
frontline techniques that must be run in laboratory environments and on
very powerful computers when they are in their development stage.
In our system, some parts of the computer vision algorithms
have been transferred to a compact computer system that weighs only
a few kilograms, and that can therefore realistically be brought on
board a small conventional UAV. As the project proceeds, all the vision
algorithms will be transferred to the on-board computers. Further
minimization using various VLSI techniques is also foreseen.
- The UAV simulation and the virtual operational environment are
constructed using a combination of commerically available software tools
and the results of our own research.
The grahical rendering software is a standard, commercial product.
The project develops a modelling system and language, Modelica, for
describing complex, object-oriented systems. In this respect it
particpates in an international, cooperative effort that is intended
to lead to standardization.
- The developed system has a clean, well defined, and well documented
architecture that lends itself to the replacement of modules and
the extension with additional subsystems. In particular, modules
communicate by TCP/IP connections, which means that different
modules can be located on different nodes in the Internet, even at
arbitrary distance from each other. The architecture therefore provides
an excellent platform for the continued reseach.
Relationship to the disciplinary research part of WITAS
The other part of WITAS, besides systems building, is the disciplinary
research, whose primary means of reporting is through published
journal and conference articles. A list of these publications from WITAS
Phase I is published separately and is available on the WITAS webpage.
Some of the publications are primarily descriptions of some aspects
of the WITAS systems development. This applies e.g. for the following
articles (with the numbers used in the publication list):
1998:026, 1998:163, and 1999:051.
However, most of the publications describe disciplinary research
results that have not directly contributed to the system that was
built during 1997 and 1998. In many cases the results will be used
in the next generation system being built in phase II. It is
interesting, however, that there is often a connection in the opposite
direction, where the experience from the systems building in phase I
is part of the background for the research publications during those
years. Schematically, we can see a chain of the following form:
Systems implementation during phase I (1997 - 1998)
leads to
Generalization of principles published during phase I (1998 - 1999)
leads to
Application of these principles for systems implementation
during phase II (2000 and onwards)
We believe that this is a natural and sound way of connecting systems
work and disciplinary research.
In addition, of course, the publications during phase I contain a
number of more theoretical results that are expected to go into the
phase II system.
Additional documentation
An exhaustive description of the WITAS system architecture from the
project's phase I is fond in the following reference:
Patrick Doherty. The WITAS Integrated Software System Architecture.
Published by Linköping University Electronic Press and available
at http://www.ep.liu.se/ea/cis/1999/017/.
Maintenance information:
Latest update 6.11.2000 by WITAS secretariat.
Edit mode aml, position code D.witas.info.demos.
To proceed, click other item in menue |