From: Jon Doyle 
Date: Fri, 18 Apr 1997 07:01:03 -0400
Subject: FPA's and conferences

Dear Erik,

I realized that you probably can make sense of the following note I sent to Randy Davis last fall. [...] you can see why I am excited about your proposal, especially how clearly you have captured some of the ideas. Maybe I'm forgetting something of your proposals at the moment, but one thing I think might be an addition to yours is the integration of the FPA with conference submission and proceedings proposed here. In a sense, the conference proceedings would be a specialized ENJ.

I hope this proves useful in some way. ---

From: Jon Doyle 
Date: Mon, 4 Nov 1996 14:59:04 -0500
To: davis@ai.mit.edu
Subject: AAAI Conference Organization, Addendum

Dear Randy,

I have thought more about the issue of AI conference organization since my October 7 report to you. This message is to write some of these down prior to this Friday's executive council meeting.

I think it reasonable to try to think through more systematically the purpose of conferences. I suspect someone has done things like this in the past, and perhaps written it down somewhere, but I don't know of anything written on this. The thought below are not very coherent, but hopefully will raise some issues that someone can debug if it seems worthwhile. I'm not sure I have real answers to some of the obvious questions, but that doesn't seem sufficient reason not to raise the issue.

My new idea is that we might try decoupling paper submission from conferences; letting authors submit papers at any time to an "AI Conference Pool" and letting meetings invite presentations based on these submissions as they choose.

The aim of the scheme is to reduce expenses for conferences, and to make life simpler for authors. To make life simpler, authors are given a means for submitting papers at any time and getting some feedback with some rapidity, and without as many concerns about unsuccessful submission to one meeting precluding submission to a later one. To reduce expenses, the primary publication medium of conference papers would be the web; meetings could choose to print volumes of their own papers if they wished for use at meetings using a standardized and streamlined publication preparation facility. Similarly, the burden on program committees would be reduced by sharing reviews.

The idea is that a number of major conferences (or even just AAAI meetings alone) would agree on some standard submission format or formats, say AAAI-size papers (6 conference pages), or KR-size papers (12 conference pages), and agree on a calendar for the different meetings, spanning a year or two.

Authors could submit papers to the "conference pool" at any time of the year. They could specify preferences about what meetings they might wish to attend along with these submissions.

Each meeting participating in the pool would first select a subset of the pool for review, and contact the authors involved to ensure interest in the meeting if that interest is not explicitly stated in the preferences. Each meeting would then review its selected body of "virtual submissions" using its own program committee.

The cooperating conferences could pool reviews across conferences. This may necessitate some mechanisms for ensuring anonymity of the reviewers. It would also call for a bit more care in how to interpret reviews, and forethought on the part of reviewers, as a paper unsuitable for one meeting might well be viewed as good for another. Sharing reviews would not be trivial, since authors would often revise papers after receiving reviews, and later meetings would be considering these revised versions. To make this work, the pool would have to preserve paper versions, reviews of specific versions, and to aid in reviewing new versions, responses to reviews by the authors (either disputing the review assertions or explaining how the revision addresses the criticisms).

With the reviewing completed, the meeting would select a subset to invite for presentation at that meeting. These might be papers already designated for presentation at earlier meetings if the meeting didn't care about the duplication (as with workshop presentations vs. major conference presentation).

Acceptance of invitations to present at a meeting would carry a commitment, in that the participating meetings might say that backing out of an accepted invitation just because of a later invitation to present at a later meeting would be grounds for (or would necessitate) the later meeting withdrawing its invitation.

This scheme shades off into preprint archives and journal publication. The conference pool resembles some proposals for simply grading or reviewing preprints stored around the web or in preprint archives, with acceptance by a journal one of the higher grades possible (but not the highest, as retrospective reviews would offer evaluations of the cumulative influence of articles). The system, if workable for conferences, might well constitute an electronic journal in addition; or some existing journal (JAIR?) might wish to try out this added role.

Why might we want to do this? Let's look at the cost/benefit situation for scientific meetings.

Scientific meetings have several purposes:

1. getting people together to talk;
2. letting people hear presentations of published work (most commonly published for the first time in the meeting proceedings or working papers);
3. publishing collections of papers on recent research;
4. bestowing credit on authors of accepted papers;
5. ancillary functions (side-meetings, book buying, trade exhibits, sightseeing)

Scientific meetings also have several sources of cost:

1. printing and distribution of proceedings
2. time and effort of program committee
3. time and effort of submitters of papers (accepted or not)
4. time and money of attendees
5. rental of meeting facilities.

Just as electronic media like the web are changing the face of journal publication, it seems to me they might also change the face of meetings as well.

I presume there is benefit to getting people together to talk at a meeting; I can't see any near-term replacement for this with video technology (but wouldn't mind being surprised).

Absent a dramatic improvement in the quality of authors and writing skills, I also don't see any immediate end to the benefit some receive from hearing talks about papers. One might have these on video recordings, but this aspect of meetings is probably around for a while as well.

What has changed already is the visibility of work prior to conferences, and the value of printed proceedings volumes. In many cases, people who want to can see conference papers on the author's web pages prior to the meeting. That hasn't hurt conferences too much yet, and I doubt it will hurt much in the future. We ought to capitalize on this, as some of the best meetings I've been at is where the participants arrive familiar with the basic ideas and are keen to discuss particular questions etc.

As with electronic journals, the credit received from acceptance by a prestigious forum is (or soon will be) independent of paper printing of the forum.

Why pay for expensive conference proceedings? Many people have shelves full of them, or discard them after a few years. Some commerical concerns are now selling services of publishing proceedings on CDs instead of paper to avoid the shelf-space crunch, but at a cost not too much less than that for paper proceedings (plus, you can't look at the paper during the meeting). What if the preparation of papers for conferences was standardized in format and invested with permanence on the web so that meetings could dispense with printed proceedings altogether if they chose to do so? Suppose that upon payment of the registration fee, an attendee received access to a web site containing all the papers at the meeting, and could print out the subset they expected to want to look at during the meeting. Alternatively, attendees could order a printed copy of the volume via one of the emerging rapid-cheap-publication technologies (Xerox Docutech?), either at time of registration or after perusing the papers on the web. How much would suffer in convenience?

Why strain the resources of the field trying to recruit program committees for numerous conferences, often times having the same paper, or slightly different versions of the same, reviewed anew by many people? As a fan of economics and democracy, I'm well aware of the virtues of distributed activity and market niches, and of the dangers of the tyranny of established opinion. So I'm not trying to centralize PC work; just to allow PC's to build on each other's efforts in any way that doesn't unduly prejudice the author's treatment. This may not be possible, but it seems worth thinking about whether there is some way to make it work.

Why force people to choose in advance among acceptable forums for presenting their work? Authors always complain when the notification date for conference X is shortly after the due date for conference Y. Forcing exclusivity in advance increases the risk to authors without any benefit; the aim of these is mainly to avoid having the PC review things that will be retracted for earlier presentation.

Authors today sometimes submit slight variants of the same paper to different meetings to maximize their chances of acceptance somewhere. If all that is desired is the chance to present the work at some high-quality forum, this wastes the author's time (not to mention the PC member's time). Clearly some people will want to have papers accepted at several meetings so they can attend all with justification. If most submissions were in a common pool, the attend-one author could focus on the single paper and its revisions, and the devotee of "least publishable units" would have a harder time of it. This in turn would increase the quality of conferences, and reduce the cost-per-idea-learned to the conference attendees.

Maybe there isn't any way to make this work. Maybe no one would really want it to work (are we all addicted to LPU's?). But the changing face of paper dissemination seems to call for some rethinking here, beyond mere ideas of federating conferences and other small modifications of traditional mechanisms.

Respectfully yours,

Jon