Hide menu

The Stream-Based Reasoning Project

Both deliberative and reactive parts of a robotic architecture require access to information about the environment in which an autonomous robot operates. For example, generating a plan usually requires a complete or incomplete snapshot of information about the current state of the environment. For plan execution, lower level action implementations often need continuous and timely updates about how the environment evolves over time, and fault detection and diagnosis may build on a history of states. Thus, information gathering is closely connected to both planning and diagnosis.

A partial high-level view of the incremental information and knowledge processing required for a UAV traffic surveillance scenario.

Gathering the required information about the environment is a complex process, often involving the combination of a multitude of information sources and processing techniques at symbolic as well as sub-symbolic levels. For example, information used in an unmanned aerial vehicle (UAV) may originate in color and thermal cameras, inertial measurement units, GPS (global positioning system) sensors, geographic information systems, direct communication with ground operators or other autonomous agents, or a variety of other sources at highly varying levels of abstraction. Such information must be integrated and processed using a number of techniques including image processing, data and information fusion techniques, and higher level processes such as qualitative spatial reasoning.

DyKnow is a stream-based knowledge processing middleware service that helps organize the required processing in a coherent network of processes connected by streams. In addition to providing conceptual support, DyKnow is fully implemented and serves as a central component of the UASTech UAV architecture.

A knowledge processing application in DyKnow consists of a set of knowledge processes connected by streams satisfying policies. A policy is a declarative specification of the desired properties of a stream. Each knowledge process is an instantiated source or computational unit. A source makes external information available in the form of streams while a computational unit refines and processes streams. The knowledge processes provide stream generators which create streams. A formal language called KPL is used to write declarative specifications of DyKnow applications. The DyKnow service, which implements the DyKnow framework, takes a set of KPL declarations and sets up the required processing and communication infrastructure to allow knowledge processes to work according to specification in a distributed system. Due to the use of CORBA as an underlying communication infrastructure, knowledge processes are location-agnostic, providing support for distributed architectures running on multiple networked computers.

DyKnow also provides support for chronicle recognition, detecting complex events that are defined in terms of temporal patterns of primitive events. Object Linkage Structures support object classification, where hypotheses about object types and identities can be formed and continuously validated or rejected.

A formula progression component provided by DyKnow is essential for monitoring the execution of a plan. TALplanner allows operators to be manually or automatically annotated with monitor formulas, conditions that must hold throughout the execution of the given operator. Such conditions are included in any generated plan. During execution, DyKnow continuously gathers information to generate a sequence of states, through which monitor formulas can be progressed. Any violation is signalled to the execution system, which can then abort, replan, or perform any other context-dependent form of recovery.

DyKnow Federations

A conceptual view of DyKnow Federations, with two participating platforms.

DyKnow is mainly intended to support knowledge sharing and processing within the scope of a single platform, such as an unmanned aerial vehicle. The processing taking place on this platform is generally distributed over multiple computers, and DyKnow must take into account the possibility of delays and information loss during communication. However, DyKnow still takes advantage of having a single global specification of the required processing at any point in time, and assumes all information present in the platform should be made available to any knowledge process that requires it.

Many scenarios also require knowledge sharing between multiple agents and platforms. Though such agents would be cooperating for the duration of a particular mission, they may still be partly independent of each other, may not be willing to share all available information with each other, and may even use different base architectures.

DyKnow Federations provides a suitable architecture for controlled knowledge sharing between such platforms, where agents may dynamically join or leave a federation. Specific services are provided for finding agents in possession of a particular piece of information, or agents capable of providing streams of such information. Agents in the DyKnow Federations framework comply with the FIPA ACL standard. Speech acts in the ACL language are used to set up communication channels, after which stream communication is handled through efficient lower level channels.

Publications

[21] Fredrik Heintz, Jonas Kvarnström, and Patrick Doherty. A Stream-Based Hierarchical Anchoring Framework. In IEEE/RSJ International Conference on Intelligent RObots and Systems (IROS), St. Louis, Missouri, October 2009. Accepted for publication. [ Conference ]
[20] Fredrik Heintz, Jonas Kvarnström, and Patrick Doherty. Stream Reasoning in DyKnow: A Knowledge Processing Middleware System. In Stream Reasoning Workshop, Heraklion, Crete, May 2009. Accepted for publication. [ Conference ]
[19] Fredrik Heintz. DyKnow: A Stream-Based Knowledge Processing Middleware Framework. Ph.d. thesis, Linköping University, Department of Computer and Information Science, March 2009. [ .pdf | E-Press ]
[18] Patrick Doherty, Jonas Kvarnström, and Fredrik Heintz. A Temporal Logic-based Planning and Execution Monitoring Framework for Unmanned Aircraft Systems. Journal of Autonomous Agents and Multi-Agent Systems, 2009. Published and available online February 2009; print publication forthcoming. [ Journal ]
[17] Mattias Krysander, Fredrik Heintz, Jacob Roll, and Erik Frisk. Dynamic Test Selection for Reconfigurable Diagnosis. In Proceedings of the 47th IEEE Conference on Decision and Control (CDC), Cancun, Mexico, December 2008. [ Conference | .pdf ]
[16] Fredrik Heintz, Jonas Kvarnström, and Patrick Doherty. Knowledge Processing Middleware. In Stefano Carpin, Itsuki Noda, Enrico Pagello, Monica Reggiani, and Oskar von Stryk, editors, Proceedings of the 1st International Conference on Simulation, Modeling, and Programming for Autonomous Robots (SIMPAR), volume 5325 of Lecture Notes in Artificial Intelligence, Venice, Italy, November 2008. Springer Verlag. [ Conference | .pdf ]
[15] Jonas Kvarnström, Fredrik Heintz, and Patrick Doherty. A Temporal Logic-Based Planning and Execution Monitoring System. In Jussi Rintanen, Bernhard Nebel, J. Christopher Beck, and Eric Hansen, editors, Proceedings of the 18th International Conference on Automated Planning and Scheduling (ICAPS), Sydney, Australia, September 2008. [ Conference | .pdf ]
[14] Fredrik Heintz, Mattias Krysander, Jacob Roll, and Erik Frisk. FlexDx: A Reconfigurable Diagnosis Framework. In Alban Grastien, Wolfgang Mayer, and Markus Stumptner, editors, Proceedings of the 19th International Workshop on Principles of Diagnosis (DX), Blue Mountains, Australia, September 2008. [ Conference | .pdf ]
[13] Fredrik Heintz, Jonas Kvarnström, and Patrick Doherty. Bridging the Sense-Reasoning Gap: DyKnow - A Middleware Component for Knowledge Processing. In Proceedings of the IROS workshop on Current Software Frameworks in Cognitive Robotics Integrating Different Computational Paradigms, Nice, France, September 2008. [ Conference | .pdf ]
[12] Fredrik Heintz and Patrick Doherty. DyKnow Federations: Distributing and Merging Information Among UAVs. In Proceedings of the 11th International Conference on Information Fusion (Fusion'08), Cologne, Germany, July 2008. [ Conference | .pdf ]
[11] Fredrik Heintz, Piotr Rudol, and Patrick Doherty. Bridging the Sense-Reasoning Gap Using DyKnow: A Knowledge Processing Middleware Framework. In Joachim Hertzberg, Michael Beetz, and Roman Englert, editors, KI 2007: Advances in Artificial Intelligence, volume 4667 of Lecture Notes in Artificial Intelligence, pages 460-463. Springer Verlag, Berlin / Heidelberg, September 2007. [ Conference | .pdf ]
[10] Fredrik Heintz, Piotr Rudol, and Patrick Doherty. From Images to Traffic Behavior - A UAV Tracking and Monitoring Application. In Proceedings of the 10th International Conference on Information Fusion (Fusion'07), Quebec, Canada, July 2007. ISIF, IEEE, AES. [ Conference | .pdf ]
[9] Fredrik Heintz and Patrick Doherty. A Knowledge Processing Middleware Framework and its Relation to the JDL Data Fusion Model. Journal of Intelligent and Fuzzy Systems, 17(4):335-351, February 2006. [ Journal | .pdf ]
[8] Fredrik Heintz and Patrick Doherty. A Knowledge Processing Middleware Framework and its Relation to the JDL Data Fusion Model. In Petter Ögren, editor, Proceedings of the 3rd Swedish Workshop on Autonomous Robotics (SWAR'05), pages 50-51, September 2005. [ Conference | .pdf ]
[7] Fredrik Heintz and Patrick Doherty. A Knowledge Processing Middleware Framework and its Relation to the JDL Data Fusion Model. In Erik Blasch, editor, Proceedings of the 8th International Conference on Information Fusion (Fusion'05). ISIF, IEEE, AES, July 2005. [ Conference | .pdf ]
[6] Fredrik Heintz and Patrick Doherty. A Knowledge Processing Middleware Framework and its Relation to the JDL Data Fusion Model. In Peter Funk, Thorsteinn Rögnvaldsson, and Ning Xiong, editors, Proceedings of the 3rd Joint Workshop of the Swedish Artificial Intelligence Society and the Swedish Society for Learning Systems (SAIS-SSLS), pages 68-77, April 2005. [ Conference | .pdf ]
[5] Fredrik Heintz and Patrick Doherty. DyKnow: A Framework for Processing Dynamic Knowledge and Object Structures in Autonomous Systems. In Barbara Dunin-Keplicz, Andrzej Jankowski, Andrzej Skowron, and Marcin Szczuka, editors, Proceedings of the International Workshop on Monitoring, Security, and Rescue Techniques in Multiagent Systems (MSRAS), Advances in Intelligent and Soft Computing, pages 479-492, Heidelberg, March 2005. Springer Verlag.
[4] Fredrik Heintz and Patrick Doherty. DyKnow: An Approach to Middleware for Knowledge Processing. Journal of Intelligent and Fuzzy Systems, 15(1):3-13, November 2004. [ Journal | .pdf ]
[3] Fredrik Heintz and Patrick Doherty. Managing Dynamic Object Structures using Hypothesis Generation and Validation. In Proceedings of the AAAI Workshop on Anchoring Symbols to Sensor Data, San Jose, California, July 2004. [ Conference | .pdf ]
[2] Fredrik Heintz and Patrick Doherty. DyKnow: A Framework for Processing Dynamic Knowledge and Object Structures in Autonomous Systems. In Proceedings of the International Workshop on Monitoring, Security, and Rescue Techniques in Multiagent Systems (MSRAS), June 2004. [ Conference | .pdf ]
[1] Fredrik Heintz. Chronicle Recognition in the WITAS UAV Project - A Preliminary Report. In Proceedings of the National Swedish Artificial Intelligence Workshop (SAIS), Skövde, Sweden, March 2001. [ .pdf ]

Page responsible: Patrick Doherty
Last updated: 2014-04-30