|Moderated by Erik Sandewall.|
Introduction to the Fluent Calculus
mentioned above has been submitted to the Electronic
Transactions on Artificial Intelligence, and the present page
contains the review discussion. Click here for
explanations and for the webpage of theauthor, Michael Thielscher.
Overview of interactions
Q1. The Editor (15.3.1999):
The present introductory remarks refer to the statements by Anonymous Referee number 1, which are listed below.
The standard procedure in the ETAI is that we have the discussion during the open review period, and referees only make recommendations for a yes or no decision on acceptance. However, we also wish to accomodate the case where a referee adds substantially to the discussion about an article, as is the case here. In keeping with the ETAI's idea, we wish to make these comments open to the author (to give him a chance to answer) as well as for the readers of the Newsletter and web page who may also contribute to the discussion.
Both the author's contribution and the referee's comments must of course be interpreted in the context for which the article was submitted, that is, as a reference article. The explanation of the reference article concept, and the refereeing criteria for acceptance, can be found from the general ETAIJ web page by clicking "For Authors" and "Referee". The first criterium was specified as follows:
Since reference articles were not supposed to include a defense of the respective approach, it is appropriate to open the discussion at this point and to obtain the author's point of view. Other contributions to the discussion are also welcome.
The second part of the Referee's comments refer to refereeing criteria number 2 and 5:
The remaining criteria include the question whether the article can effectively be used as a standard reference for the notation and the basic concepts of the approach being presented, for use in other articles using that approach.
Q2. Anonymous Referee 1 (16.3.1999):
Objections to the approach
The idea in this approach is a fairly simple one. 'States' (it would be better to say 'state descriptions') are lists of atomic assertions encoded as terms, and one uses a unification trick to ensure that any things in the list that aren't explicitly mentioned in the 'update axiom' get automatically transferred to the list which describes the post-action state. However, the 'deletion' here depends on an assumption about unification which isn't logically valid: that if two terms don't unify, they have disjoint referents.
Apart from the logical difficulties, this is computationally trivial (sub-STRIPS) and unoriginal. It is indequate for realistically complex domains. For example, take the toy blocks world as given, and add 'Above' to the vocabulary, defined recursively in terms of 'On'. The update axiom now needs recursion. Examples like this have been around for a quarter of a century. In general, the FP is only nontrivial when one has a vocabulary such that some aspects of the state description depend on others in ways that vary with the particular situation: in STRIPS terms, not all the vocabulary is part of the core. If not, then one can just have explicit add and delete lists for each action description, which is essentially what Thielscher is doing here, but in Prolog.
It doesn't have a clear semantics. The account given uses logical terminology, but it doesn't really use first-order logic. The extended unique names assumption is so strong that it renders equality trivial, as the authors take pains to show.
The limitations of the techniques being used ought to be carefully stated. In particular, the authors should refer to the STRIPS planner developed at SRI in the 1970s and explain how these techniques are inferior to those used by it.
A2. The Author (27.3):
These are responses to the objections raised by the anonymous reviewer against the approach:
Q3. Anonymous Referee 1 (16.3.1999):
Objections to the presentation as a reference article.
I confess it was very hard to finish the first page without getting very puzzled. The text assumes the reader shares nonstandard definitions of technical terms from obscure conference papers, and the illustrative example is very hard to understand.
Example 1 uses a (binary!) function symbol
Apparently, $z$ here is supposed to be a fact (or, still more curious, "facts"), yet we have somehow managed to quantify over it (them?). How can one consistently quantify over facts, written using a logic, in that logic itself? (A rhetorical question, of course: one can't. Ref. Gödel).
Thielscher says that asserting
On page 2 we are told that
The text ought to be rewritten and reorganised. First, the English is turgid and sometimes hard to follow. Second, the terminology being used should be carefully explained, since the uses here of "fluent" and "situation" (for example) are not those used by many other authors. Third, the logical semantics of the methods being used should be carefully explained; one cannot casually refer to a domain of quantification as consisting of "facts".
A3. The Author (27.3):
These are responses to the objections raised by the anonymous reviewer against the presentation as a reference article:
My fault. I shouldn't say "fact" at this point and be more precise about the method used here known as "reification."
C3-1. Anonymous Referee 2 (27.3):
Referees should never be refereed (but judged by editors). In this case, since the editor explicitly welcomed contributions to the discussion of the report of Referee 1 in ENRAC of 16 March, I feel that the following remarks are appropriate, or rather necessary. Thereby I will not take up technical issues, since these are the matter of the author, except for generally pointing out that in my understanding the referee mostly failed to do a technically careful job. Eg. his first complaint
Well, referees are not to blamed for making technical mistakes. We all are prone to this human weakness. And authors are able to defend themselves against such mistakes. However, there are issues involved in Referee 1's report which are awkward to be addressed by any author, especially a young one, and which are not in compliance with the behavior expected from fair and responsible referees of international journals. Here I point out three such issues.
1. The referee disqualifies the paper's notation as being taken ``from obscure conference papers''. The author cites five papers as already mentioned above. Two of them appeared in the AIJ, one is conditionally accepted for publication in the AIJ, the remaining two appeared in the journals New Generation Computing and Annals of Mathematics and Artificial Intelligence. To flash five journal papers of this calibre as ``obscure conference papers'' amounts to an extremely unfair behavior which should be banned by the community.
2. The referee maintains that ``the English is turgid and sometimes hard to follow'' without providing even a single example or hint. I challenge Referee 1 to write a paper in a language of his/her choice, and foreign to him/her, as competent as Thielscher did in the present case. The way the referee's complaint is brought forth in this case demonstrates an example of uttermost narrow-mindedness and arrogance inappropriate for a referee of an international journal.
3. The referee disqualifies the entire approach as ``computationally trivial (sub-STRIPS) and unoriginal''. This remark alone would already have disqualified the competence of Referee 1. There have been many attempts to embed STRIPS in a logical framework and it was not until the discovery in the mid-eighties (prior to Lifschitz' STRIPS semantics paper) of the predecessor of the fluent calculus (the linear connection method, now also referred to as transition logic) that this fundamental issue was solved.
Anonymity of referees is an important and sacred principle. As is the avoidance of public discussions of referees' reports, for which reason I feel very uneasy with this type of discussions (yet encouraged by the editor). The rules of ETAI might be reconsidered in this respect.
Referee 2 writes,
Now, I can imagine a researcher responding to this view in the following way: "If I'm open to public criticism myself, why should I referee papers? It's hard work, it's a nuisance, and it's voluntary. So I won't bother any more." The answer is that reviewing is not a favour, but a duty. All of us who contribute papers ourselves and expect them to be well refereed have a duty to referee other papers, and to do so with the utmost professionalism.
Q4. Anonymous Referee 2 (27.3):
The paper is submitted to ETAI as an ETAI Reference Article. As such it does not claim to offer original results (which it does not). Rather I am asked to referee the paper under the published refereeing criteria for this particular ETAI category.
Under these considerations
I recommend acceptance of the paper provided ...
the author follows a few recommendations given below, and adds pointers (with comments) to some further articles which give a full account of the history of the approach and its relation to variants and alternative approaches (and which do exist).
This referee is in a somewhat awkward position since this report was written after having seen the report by Referee 1. Not that that report would have influenced this referee's judgment. But of course it does influence the justification of the judgment which now follows, beginning with responding to the set criteria.
Q5. Area Editor (27.3):
At this point (27.3) two referees disagree quite strongly, and the author has given substantial answers to the objections raised by Referee number 1. Naturally now we invite Referee number 1 to express jirs view before the accept/decline decision is taken.
Anonymous Referee 2 also raises some important policy questions in jirs comment to question 3. In particular, je subscribes to the view that "avoidance of public discussions of referee's reports ... is an important and sacred principle", and "referees should never be refereed (but judged by editors)". This has indeed been the traditional practice, but ETAI is not traditional. One of our fundamental innovations is to open up the review process to the largest extent possible, without compromising the quality control that is obtained through anonymous referees that vote to accept or decline an article after the review period.
If all relevant questions and critique comes out during the open review, then it is sufficient for the referees to vote 'pass' or 'fail', and there is not much to debate. (It is always possible to continue the open review debate, of course). However we have had a few cases, including the present one, where an anonymous referee added jirs own contributions to the review debate, and in order to be consistent we must then make it possible to discuss such opinions by referees as well. In a similar vein there is no reason why the policies and decisions of the area editor should not be open to discussion. (However the final decisions are taken by the area editor and not by popular vote).
The system of open debate has been made practically possible through the advent of Internet, e-mail, and web pages for managing on-line debates. It would have been virtually impossible to apply it in the earlier age of conventional print-and-mail communication. In thinking about whether and how to use the new possibility, one must consider both the advantages and the possible disadvantages of the open reviewing approach.
Its advantages have been described extensively before, when the ETAI was started and in its present web pages. The following are some possible objections to the openness:
These are important questions, both for the ETAI and for the field, and the comments by the readership are cordially invited.
Language remark. The following synthetic pronouns are introduced and will be used e.g. in reference to Anonymous Referees in order to combine concerns for anonymity preservation and for political correctness: je subsuming "he" and "she"; jem subsuming "him" and "her", and jirs subsuming "his", "her", and "hers".
The referees make a recommendation whether the article is to be
accepted or declined, as usual. The article and the discussion
remain on-line regardless of whether the article was accepted or
not. Additional questions and discussion after the acceptance decision
The Review Protocol Page is used as a working structure for the entire
reviewing process. During the first (review) phase it accumulates the
successive debate contributions. If the referees make specific
comments about the article in the refereeing phase, then those comments
are posted on the RPP as well, but without indicating the identity
of the referee. (In many cases the referees may return simply an
" accept" or " decline" recommendation, namely if sufficient feedback
has been obtained already in the review phase).
The referees make a recommendation whether the article is to be accepted or declined, as usual. The article and the discussion remain on-line regardless of whether the article was accepted or not. Additional questions and discussion after the acceptance decision are welcomed.
The Review Protocol Page is used as a working structure for the entire reviewing process. During the first (review) phase it accumulates the successive debate contributions. If the referees make specific comments about the article in the refereeing phase, then those comments are posted on the RPP as well, but without indicating the identity of the referee. (In many cases the referees may return simply an " accept" or " decline" recommendation, namely if sufficient feedback has been obtained already in the review phase).