An alternative to peer review

Terry Brooks - our Associate Editor for North America drew my attention to an article in the New York Times about attempts by humanities scholars to replace the traditional peer review process. The NYT reports that the Shakespeare Quarterly:

posted online four essays not yet accepted for publication, and a core group of experts... were [sic]  invited to post their signed comments on the Web site MediaCommons, a scholarly digital network. Others could add their thoughts as well, after registering with their own names. In the end 41 people made more than 350 comments, many of which elicited responses from the authors. The revised essays were then reviewed by the quarterly’s editors, who made the final decision to include them in the printed journal, due out Sept. 17.

The question that Terry raised was whether or not such a process could exist in our field and, more specifically, is this something that Information Research could pioneer?

I have my doubts.  Partly because I do not see a great willingness of those researching the field to contribute to debate.  For example, the various academic discussion lists are virtually devoid of anything but conference announcements, job announcements, requests for hotel room sharing and similar matters. Debate on scholarly issues is lacking.  This is paralleled by the relative paucity of debate at conferences.

The other reason is that some submitted work is so poorly written that, were it to be released before refereeing and copy-editing, it would simply be ignored, no matter whether or not it presented novel results or methods or theory. And I can't imagine spending time on copy-editing a piece (I have spent almost 10 hours this past week working on one paper) only to have it 'rejected' by the community.

However, I'll be interested to have your views :-)