Refereeing Criteria for Articles with Crisp ResultsThe ETAI asks confidential referees a small number of precise questions; the answers to those questions form the basis for the acceptance decision. We try to avoid questions such as "please grade the quality of this article on a scale from 0 to 10", and we favor more concrete questions. The choice of questions depends on the type of article.
The major article type is the one that proposes new, crips results for a well-defined problem. For such articles, we normally suppose that there exists a summary that characterizes the article's results in a succinct way. The summary should be longer and more concrete than the usual abstract: it should specify what are the results and not merely talk about them. Phrases such as "the article proposes a new approach to ..." are not appropriate in a summary; the summary should say what the approach is and what has been achieved with it. New results in the framework of an existing approach are at least as highly valued as a proposal for a new approach.
For this type of articles, referees are asked the following set of questions:
We foresee using other question sets for other types of articles, and in particular for articles that propose a new approach to a particular class of problems, and where the article is largely argumentative rather than proposing crisp results. We also foresee a third set of questions for survey articles. These question sets have not yet been worked out completely.