| Issue 98080 | Editor: Erik Sandewall | 16.11.1998 |
Today |
The workshop on Cognitive Robotics at the AAAI Fall Symposium in October contained several contributions and discussions that are of great interest for the Actions and Change research area. The present Newsletter contains a position statement by Yves Lesperance where he summarizes and comments about one of the discussion topics at the workshop, namely, the format of robot competitions at the AAAI conferences.
Debates |
It has been our experience that the winners of the AAAI and RoboCup robot competitions are those that can engineer the solution to the specific problem and who can program for long hours. We are soliciting ideas on how to design a robot competition that will exploit robot brains as well as robot and programmer brawn.
This was recently discussed at a panel on robot competitions held during the AAAI Fall 1998 Symposium on Cognitive Robotics. Appended below is a summary of that discussion. It is organized as follows:
3 main proposals for competition tasks made:
The 1st panelist was Alan Schultz, who is chair of the AAAI robot competition organizing committee. He gave an overview of the history of AAAI robot competitions. At first, the robots had no manipulators and the tasks assigned (e.g. navigation problems, give a tour, find life on Mars, find the remote control, clean the room) reflected this. Later manipulators became available and the tasks changed to reflect this. Vision systems are still not required.
Then, Alan said that the competition was being split into a more basic student competition, and a challenge competition where tasks require significant advances to the state of the art and stay unchanged for several years. As an e.g. of a challenge task, he mentioned "registering for AAAI, attending talks, summarizing what was learned, and putting the results in a web page". A kind of "scavenger hunt" task was also mentioned. He said the committee was very interested in entertaining proposals from the cognitive robotics community.
During the subsequent discussion, Sheila McIlraith suggested that a "scavenger hunt" task where clues are provided would be a good test for cognitive robotics. Many participants reacted enthusiastically to this proposal. Yves Lesperance pointed out that we should be careful about game-like tasks (e.g. chess), which seem to involve a specific kind of intelligence not typical of commonsense reasoning. Later in the discussion, Stuart Shapiro suggested that the task could involve finding people given clue messages in natural language. Someone also suggested that the clues be given on the web.
The 2nd panelist, Murray Shanahan, listed some criteria for a good competition task. It should not be a task where the robot becomes a mere front end to a reasoning module. It should require reasoning as part of the perception action loop. Elaboration tolerance should also be required. Erik Sandewall added to this that the task could involve performing activities while moving; it should combine planning and obstacle avoidance.
The third panelist, Michael Jenkin, said that we want to design a competition which favors 'neat' solutions over 'scruffy' ones. He suggested looking at existing 'scruffy' competitions:
The fourth panelist, Sebastian Thrun, said that existing AAAI competitions do not provide much of a challenge on the cognitive level, the reason being that they typically focus on a single, pre-determined task that does not require on-line symbolic planning and decision making. He suggested a competition centered around the theme of an intelligent office assistant. This robot would be in charge of delivering mail (which arrives sporadically), coffee (at scheduled times), and occasionally people would request tours or guidance through the building. Such a scenario emphasizes two important things: (1) on-line planning and decision making on the "cognitive" level, and (2) the integration of low-level navigation skills with high-level planning and decision making. He encouraged people in the cognitive robotics community to really do it, i.e. build cognitive robots and compete. In the robotics community, this is the only way to get attention.
Martha Pollack then described a similar idea for a task called "Be My Administrative Assistant". Ahead of the competition, examples of possible office tasks to be assigned would be provided. Then at competition time, each robot would be assigned a randomly generated set of tasks with randomly generated deadlines, some of which may involve forming new plans out of the actions in the announced tasks; e.g.:
In the end, many participants seemed enthusiastic about the more ambitious tasks proposed, but some seemed to want to stick to simpler tasks where there were clear interactions between perception, action, and reasoning. Developing a test suite of tasks with varying degrees of difficulty would seem to be a good idea. But we still need to agree on a competition task.
Gerhard Lakemeyer raised the important point that unless we had teams that were ready to compete this summer, we shouldn't push our agenda for the 1999 robot competition.
Stuart Shapiro suggested that we perhaps start it as a (can't remember the exact term used) "demonstration sport" in the first year (like ballroom dancing at the Olympics).
It was agreed that we would try to:
This summary was reconstructed from my notes and material obtained from Chitta Baral, Michael Jenkin, Sheila McIlraith, and Sebastian Thrun. Apologies to anyone whose ideas may have been distorted or left out. Please post your clarifications!
Yves Lesperance
lesperan@cs.yorku.ca