E-mail message
From: Jon Doyle
Subject: Neo-classical structure etc.
Date: Fri, 4 Apr 1997 10:15:28 -0500
Dear Erik,
I've read over your papers on publications, and am quite excited by
many of the ideas. I had thought somewhat about these things, but you
have gone far beyond my thinking in depth, detail, and care of
exposition. You have made a very masterful analysis.
I told the AAAI Executive Council about your work, and will write a
brief synopsis for them and send you a copy. We had been thinking
about what to do about the (presently essentially useless and unused)
AAAI preprint server, and I would like for us to explore cooperation
with ECCAI on this.
I append some comments on the papers below. By and large I'm either
in agreement or willing to try out the things you suggest. I have a
number of picky points of minor disagreement. There is just one point
on which I have big worries about unintended adverse consequences of
the policy you suggest, and want to find out what you think about
this. I've interspersed the minor typos along with the major
comments. Most of the comments are on the first paper I read. It has
been a couple weeks since I read them, with a lot of stuff
intervening, so I may have had additional opinions on the papers
overall that I can't recall right now, but I'll let you know if I
think of anything else.
I'm sorry it has taken me so long to respond to your proposals. I
hope I'm not too late to have some effect or to offer some help. I
certainly would like to see some AAAI-ECCAI cooperation on FPAs and
ENJs, or maybe even more.
Sincerely yours,
Jon
----------------------------------------------------------------------
Comments on:
A Neo-Classical Structure for Scientific Publication and Reviewing
----------------------------------------------------------------------
I really like the first and last bullets of the synopsis in Sect. 1.
I'm more dubious about the prospects for success for the switch to
open reviewing after the current period of anonymous reviewing, but am
not adverse to trying it out.
5.2: I don't believe "preprint archive" is a contradiction in terms.
Such an archive serves to make something readable (on a screen) before
it is printed (on paper). It is only a contradiction if you call
displaying something on a screen "printing", which is arguably not a
good current meaning. The term would have been a contradiction in
terms 40 years ago, when any archive of papers would have been paper
copies (or maybe handwritten manuscripts wouldn't count as "printed").
I do agree that the term is not a good one, even if it is not
self-contradictory. Whatever term is used should be invariant to
changes of practice, namely the possible disappearance of printing on
paper (apart from making personal copies). I like the FPA term. The
only thing I can see as a drawback of that name is that some might
read it as providing a guarantee that its contents are, through their
appearance there, made public for the first time. That may not be the
case, since someone may have been making the papers public elsewhere
beforehand, or worse still, may be plagarizing something made public
previously (even in the same archive). But I'm not sure this is
really anything to worry about. I like the term you suggest.
p.19, after the bullets: This paragraph ("chaired professor") sounds
too European; not inappropriate for something done by ECCAI, perhaps,
but it would be nice if the proposal worked in North America as well,
where even assistant professors can be leaders of research groups
directing hired researchers.
Here, and in the next paragraph, I wonder why the selection
people are restricted so narrowly. One might add in "fellows" or
other honorees of professional or scientific societies; or perhaps
even broaden it to allow submission by any member of a scientific or
professional society.
In the AAAI discussions, we had talked about a scheme in which
anything would be included in the preprint server (FPA) when submitted
by a member, with the name of the submitter attached if the
member-submitter was not also the author. I agree you want stronger
filtering for the ENJ's.
Another possibility is to have some ENJ-specific designation
of people admitted to submit items for mention, akin to the BBS
Associate status. Membership in this designation could be granted by
blanket permission to all senior professors and leaders of research
laboratories; others could be added by decision of the journal
editors or by petition or nomination by others.
p.20, top: I'm really bothered about the definition given here of when
a research result counts. If appearance in an FPA doesn't by itself
count as defining research priority, then the term FPA is a lie. If
you look at the practice of historians, they don't say someone didn't
discover something because no one noticed their publications for
years. They give priority (eventually) to the person who published
the idea first, even if everyone else first learned of it from some
later author. So the idea that publication in an FPA doesn't count
for priority unless some ENJ notices it seems really wrong to me.
I would propose that FPA's alone define research priority.
Perhaps what is needed is additional credit for someone helps convince
or enlighten the field of some result, even if they were anticipated
by someone else.
I have more to say about this issue below; I'm bothered as
well by related aspects of your proposal.
5.4: I would think seriously about changing the nomenclature a bit to
remove the term "electronic" from ENJ and ET's, or at least find some
more appropriate adjective. I've had this argument with ACM, and
lost, but I'll make it for you too.
Electrons are not central to the issue, and so shouldn't
figure in the title of a publication medium. We don't call the AIJ a
vegetable-fiber journal, after all. Moreover, many of the storage
and transmission media for "electronic" information involve optics,
either photons in optical fiber, or physical light-deflecting dimples
on CD's, so a term referring to electrons is hardly appropriate.
"Digital" might be better, but I'm not positive, as it is also
tied to technology in ways that might change. "Online" might be
better, as it refers more to function or accessibility than medium,
but perhaps there is some even better term.
5.4: I have worries about switching the world back to an
open-reviewing regime after it has grown so used to anonymous
refereeing, but I am willing to go along with the gamble for all the
good reasons you propose. Mike Wellman is already worried we will be
moving before long to having to pay reviewers for their service; the
open credit scheme may solve that problem without money changing
hands. My worry is that people might be too hesitant to write public
reviews compared to anonymous ones, since the effort required is
greater and the potential risks are larger. Whether the additional
benefits of credit for improvements and analysis will outweigh the
costs to reviewers has to be seen.
Perhaps the strongest argument is that we can't go on as at
present, so public reviewing provides not only better advantages for
the field at large but also the only economically feasible approach.
5.5, last paragraph: I understand the motivation, but have doubts
about the debate article mechanism proposed here. How many people
would choose to read such articles? In my experience, a significant
fraction of crank papers are in essence complaints that prior crank
papers have not been adequately appreciated. (For that matter, this
may be true even if one removes both occurrences of the word "crank"
from the last sentence.) There was a long strech of years where a
fellow in Connecticut published an abstract in almost each issue of
the AMS Notices proving the inconsistency of ZF set theory, with
periodic abstracts wondering why no one paid attention to these
claimed proofs. I'm not sure we want to encourage such behavior.
p.22, top: Again, why the proviso that FPA doesn't count unless
reviewed in an ENJ? This says FPA isn't publication.
p.22, bottom: I'm not clear on the policy here for publishing
rejection decisions. Would such decisions be published only with the
consent of the author of the paper in question, or would they be
considered part of the open review commentary? If the latter, or if
they were to be published with some revised article in and ENJ as
suggested, they would have to be of a publishable form.
p.24, second full paragraph: The proposal of this paragraph is the one
I am most vigorously opposed to. I understand and sympathize with the
motivation, but think that in practice it will have a very bad effect.
As mentioned above, I think that priority should go to publication,
which to me means appearance in an FPA and not to the fact that
someone also noticed the publication. We already have plenty of
things (Stoke's law, Zorn's lemma) named after people who were not the
originators, and it seems pointless, cruel, and unscholarly to say
that popularizers should get the credit, leaving the originators with
nothing. But the idea that something counts only if it is explicitly
mentioned in the long abstract of the article, seems likely to have
even worse consequences.
What you will get with this rule is a regime like that in
biology and medicine, in which all papers represent "least publishable
units", LPUs. People will quickly settle on a form of publication
much like the default result of Latex2Html; each definition, theorem,
proof, explanation, analysis, etc. will be prepared as a separate
publication, unread and unreadable. Each of these micro-publications
will be entered in the FPA. The author will then publish the "full"
paper there as well, as a "survey" of the set of component papers.
No one would want to read the micro-papers, but they will quickly
dominate the FPA contributions in numbers.
Now maybe this wouldn't happen, but I think the incentives are
definitely there. I like to write long papers that say many things.
If I thought I couldn't get credit for something I said in these
unless it was said in its own publication, I'd quickly produce a set
of Latex macros to encapsulate each item in its own publication.
Maybe you don't think this result would be bad. From an
author's point of view, it might even be nicer to publish even small
papers incrementally, publishing each theorem as proven, and then
assembling a full paper out of those published bits that still seem to
make a coherent story. I could conceive of this not being too
offensive to science, as long as one had some reasonable way of
navigating the resulting mass of microresults (e.g., the full papers),
but to bias the whole credit and publication scheme in this way toward
tiny papers and away from larger views seems a bad idea to me. I
don't think there should be any such bias.
To recap, I think that publication is publication, and if
FPA's are to be true to their names, then publication in one should
count for priority purposes, whether or not the result is recognized
in a timely way in an ENJ, and whether or not the author is able to
(or chooses to) include it in a short synopsis of the publication.
Let me point out, as additional support for my contention,
that the abstracts of many mathematical papers simply list the
theorems proven, even when the main contribution of the paper is a new
method, and even when the author intends the method to be the major
contribution.
Sorry to go on so long about this, but I really think this
idea is a bad one. Maybe there is some other way to get the effect
you want, but I don't think this particular form will do the trick.
p.24, next paragraph: Will the "date" of the result be the date of
the original FPA appearance, or the date of the abstract? I say it
should be the FPA apperance. Publication is not the same as
appreciation.
6.1, first bullet: I think that this particular division of the form
of an article is not especially attractive, unless it is done very
carefully. As someone who cares a lot about how his articles read and
appear, it distresses me to have the form dictated, especially if the
dictated form conflicts with what to me seems the clearest or most
beautiful way of presenting the material. The proposal of this
paragraph reminds me all too much of DARPA proposals, which are long
on stupid bureaucratic form and short on making the document readable
(I presume EU/Esprit proposals have similar shortcomings).
Also, I'm not sure the division proposed here is workable.
Often the right place for a comparative note is in the presentation of
a result ("this theorem improves on the result of Foo and Bar by.."),
as the comparison serves to help make the result more intelligible.
Will this be forbidden by the proposed division? Will it be forbidden
to include diagrams and pictures in the contribution part? If not,
what is the point of the "multimedia" portion? It seems wrong to
separate out something into this section just because it is not
economic for some publisher to print it. Recall that some journals do
publish color photographs and diagrams; should the form of the same
article differ if published in AIJ instead of Science?
(I also detest the term "multimedia"; it seems even more
pointless than "electronic", since we are moving to a world in which
all things will be "multimedia". In such a situation, having a
separate "multimedia" section won't serve any useful purpose.)
p.25, sixth bullet: I think you need some temporal qualifications
here. It isn't clear to me that libraries will want to put these
bound volumes on their shelves. Most are already short of space and
funds, and the cost of printing, binding, and storing the exploding
scientific literature is causing many to look toward having the
primary form of retrieval be online, with printing on paper done on a
per-customer basis, paid for by the individual patron. The scheme
described in the paragraph makes for a reasonable transition path, but
I'm not sure libraries will find it attractive in the long run.
6.3, second bullet: I don't think that the archive version of an
article can be changable by the author at all. Changes would have to
constitute additional publication, not replacement of existing
versions. Replacement of archive contents by the author should be
impossible, not just forbidden. In practice, this means the archives
must do their own storage, not rely on pointers to author-maintained
copies.
p.27: "done on a European level" -> "done on a regional level"; also,
strike the "EU" in the next sentence.
----------------------------------------------------------------------
Comments on:
On the concept of online publication
----------------------------------------------------------------------
p.1: I already complained about "electronic", but let me rant a bit
more about "electronic document" as defined here. Why 0's and 1's?
Are dimples in CD's numbers like these? Does an article printed on
paper using a series of bar-codes readable by a scanning device count
as an electronic document? The definitions given here seem either
wrong or hopelessly vague. For example, any current printed article
might be considered an electronic publication when there is a robotic
library that can pull out the volume, open the page, scan the article,
and display it on a screen. Do you mean to include that?
It isn't clear to me why you need to try to define this
concept, any more than that of document.
p.4: I don't understand why online publishers must be complemented
with "national" archives. Why not provincial? International?
Industry or field wide archives? I agree that one probably wants more
than individual journals, but the connection to governments, or levels
of governments, seems unmotivated. If there is some stronger argument
for this, you need to make it.
----------------------------------------------------------------------
Comments on:
The ECCAI System for Specialized Research Publication
----------------------------------------------------------------------
p.5, first full paragraph: This requirement may be hard to determine
or enforce. It would be better to date improvements separately
and let historians or the field judge later which were mere
improvements in presentation and which amounted to improvements in the
results themselves.
A case in point: the mathematical formulas we call Newton's
laws today were never discovered, or even known, to Newton. They were
discovered a hundred years later by Euler. Popular wisdom, to the
extent it realizes that Newton never wrote these equations, believes
they are mere translation of Newton's prose statements into formulas,
but that is not at all true. Is Euler's accomplishment mere
improvement in presentation, or new results?
3.7: It can't be the rule that the decision depends on who has the
last word, or authors may have to spend all their time responding to
endless criticisms. There needs to be some more definite form of at
least temporary acceptance by editors or social agreement that
maintains a result even if criticisms persist, together with means for
reopening the discussion if someone puts together a significant new
critique. (I'm not especially happy saying this, but I think there is
a problem here, and am not sure how to solve it.)
p.9, first bullet: to repeat a point made in the comment on p.5, who
decides what is a clarification and what is an error? I think the
mechanisms ought not to take on this burden, but to leave such
decisions open for determination by the audience and editors.
=========================================================================
Jon Doyle URL: http://www.medg.lcs.mit.edu/doyle
Clinical Decision Making Group e-mail: doyle@mit.edu
Laboratory for Computer Science Voice: 617-253-3512
Massachusetts Institute of Technology Fax: 617-258-8682
=========================================================================