Journal
 
 
 

Refereeing Criteria for Articles with Crisp Results

The ETAI asks confidential referees a small number of precise questions; the answers to those questions form the basis for the acceptance decision. We try to avoid questions such as "please grade the quality of this article on a scale from 0 to 10", and we favor more concrete questions. The choice of questions depends on the type of article.

The major article type is the one that proposes new, crips results for a well-defined problem. For such articles, we normally suppose that there exists a summary that characterizes the article's results in a succinct way. The summary should be longer and more concrete than the usual abstract: it should specify what are the results and not merely talk about them. Phrases such as "the article proposes a new approach to ..." are not appropriate in a summary; the summary should say what the approach is and what has been achieved with it. New results in the framework of an existing approach are at least as highly valued as a proposal for a new approach.

For this type of articles, referees are asked the following set of questions:

  1. Are the results of the article, as specified in the summary, something that researchers in the same area ought to know about?

  2. Does the body of the article prove the claims that are made in the summary?

  3. Have the same results been presented before in a refereed journal article, either by the same author or by someone else, to the best of your knowledge? (Previous publication in a technical report or conference or workshop proceedings does not disqualify).

  4. Is the article well organized? (Not too long, not too short considering the contents; are concepts and notation defined before they are used, etc. Note that we can accept very long and very short articles if the contents motivate it; length is evaluated relative to what the contents require).

  5. If questions 1, 2, and 4 are answered by Yes, and question 3 is answered by No by the referees, then normally the article will be accepted by the ETAI. Is there some additional circumstance that you feel ought to influence the acceptance decision?

We foresee using other question sets for other types of articles, and in particular for articles that propose a new approach to a particular class of problems, and where the article is largely argumentative rather than proposing crisp results. We also foresee a third set of questions for survey articles. These question sets have not yet been worked out completely.