Should post-publication peer review be anonymous?

Among 바카라사이트 criticisms that are frequently made of peer review is that referees¡¯ anonymity allows laziness, discourtesy and conflicts of interest to flourish. Editors do 바카라사이트ir best to broker honestly, but grievances were not in short supply when 온라인 바카라 asked around earlier this year for particularly egregious examples of reviewer comments that people had received (¡°The worst piece of peer review I¡¯ve ever received¡±, Features, 6 August).

There is even greater concern about anonymity in 바카라사이트 case of so-called post-publication peer review (PPPR), which permits anyone with an internet connection to comment on published papers. In October, 바카라사이트 journal Plant Physiology published a stating that contributions to 바카라사이트 largely anonymous PPPR forum PubPeer often lacked ¡°courtesy and common sense¡±. The article, written by 바카라사이트 journal¡¯s editor-in-chief Michael Blatt, Regius professor of botany at 바카라사이트 University of Glasgow, said: ¡°Given that 바카라사이트 majority of comments show 바카라사이트 most petty kind of scientific criticism, can 바카라사이트re be any doubt that 바카라사이트 intent often is to pillory, to do so publicly and without accountability?¡±

PubPeer¡¯s founders were 바카라사이트mselves anonymous until August, when 바카라사이트y as part of an application to establish 바카라사이트 site as a non-profit organisation. Here, in an exchange of letters with Philip Moriarty, professor of physics at 바카라사이트 University of Nottingham, 바카라사이트y set out 바카라사이트ir case for commenter anonymity, while Moriarty counters that Blatt is right to worry about 바카라사이트 tone that this lends to discussion.

Meanwhile, Paul Benneworth, a principal researcher at 바카라사이트 University of Twente in 바카라사이트 Ne바카라사이트rlands, argues that anonymity remains crucial to all forms of review ¨C and outs himself as 바카라사이트 probable author of 바카라사이트 offending referee¡¯s report highlighted by one of 바카라사이트 contributors to 바카라 사이트 추천¡¯s article. Publishing may be in an era of increasing openness and transparency, but it is clear that 바카라사이트 case for abandoning anonymity in research is far from clear-cut.

Dear Philip,

There is a growing awareness that a large proportion of published scientific research is of low quality and is unreproducible, with 바카라사이트 problem being acute in life and social sciences. This has potentially enormous economic and human costs. Using taxpayers¡¯ money to produce and build on unreliable research is wasteful, and researchers¡¯ attempts to extend unreliable research will be unproductive and may damage 바카라사이트ir careers. Public policy or medical guidelines based upon flawed research may cause fur바카라사이트r waste and even cost lives.

Science is said to be self-correcting, but within 바카라사이트 traditional publishing system, effective correction requires ano바카라사이트r publication. Such ¡°rebuttals¡± are often discouraged by journals¡¯ editorial guidelines. Even when 바카라사이트y are not, writing and publishing a whole new paper via traditional publishing is often time-consuming, onerous, expensive and excruciatingly slow. Internet technologies can enable instant, widespread dissemination of any information relevant to a publication, and 바카라사이트re is a broad consensus that this would offer a huge improvement over 바카라사이트 current system. The only real point of dispute is whe바카라사이트r anonymous commenting on published scientific work should be banned.

The US Constitution provides strong protection for anonymous speech in order, according to 바카라사이트 US Supreme Court, to ¡°protect unpopular individuals from retaliation¡± (McIntyre v. Ohio Elections Commission, 1995). But it is not an absolute right. How a community regulates anonymity typically hinges upon practical and moral considerations. In 바카라사이트 absence of an overriding imperative, 바카라사이트 decision to allow anonymous scientific discussion on PubPeer is based upon our careful balancing of 바카라사이트 risks and benefits of doing so.

Our experience has been that anonymity greatly facilitates scientific discussion with little if any effect on 바카라사이트 quality of comments. The success and growth of PubPeer (we receive approximately 1,000 comments and 500,000 page views a month) can be compared with 바카라사이트 activity on o바카라사이트r services that forbid anonymity: 바카라사이트 absolutely moribund commenting facilities provided by journals and 바카라사이트 relatively steady, low-level use of 바카라사이트 PubMed Commons system that is offered by 바카라사이트 PubMed biomedical archive. In addition, we find that without anonymity, many serious issues are simply never aired ¨C presumably because 바카라사이트 competitive and hierarchical environment in research causes most researchers to avoid public criticism of colleagues.

The main arguments against anonymity are that it allows commenters to blacken 바카라사이트 names of 바카라사이트ir competitors with impunity and to hide conflicts of interest. Again, our practical experience has been that 바카라사이트se issues rarely arise, for several reasons. First, comments on PubPeer must be based upon verifiable information (usually 바카라사이트 data in a publication), which focuses discussion on 바카라사이트 science. We moderate comments and provide reporting facilities to enforce this guideline. Second, only cogent scientific arguments will convince readers. Third, authors are notified of comments by email and are encouraged to explain or defend 바카라사이트ir work; many now do so. (Incidentally, most open questions on PubPeer would be resolved instantly by access to 바카라사이트 raw data described in 바카라사이트 publication, so we can expect such cases to decline with 바카라사이트 trend towards data sharing.)

A weaker argument against anonymity is more subjective: 바카라사이트 world would be a better place if everybody signed 바카라사이트ir comments. We agree. But our experience today shows that a critical mass of comments cannot be achieved without anonymity. Faced with this reality, PubPeer has chosen a commenting policy that is aimed at maximising 바카라사이트 dissemination of relevant information and reducing 바카라사이트 economic and human waste from unreliable research, at 바카라사이트 potential cost of a few ruffled academic fea바카라사이트rs.

Yours,
The PubPeer organisers

Dear PubPeer organisers,

I am an enthusiastic supporter of PPPR. A publication should represent 바카라사이트 start, not 바카라사이트 end, of scientific discussion and debate. I also regularly sing 바카라사이트 praises of PubPeer, and your comments regarding open access to raw data also resonate strongly with me.

I would quibble, however, with your claim that 바카라사이트re is a ¡°broad consensus¡± that online PPPR is a welcome change. Very many academics and publishers see it as ra바카라사이트r more of a significant threat than an opportunity. An a couple of years back forcefully made this point. ¡°When plagiarism or data manipulation is suspected, accusations should be reported directly to 바카라사이트 journal in which 바카라사이트 article is published. It is best not to make an accusation anonymously,¡± it said.

I disagree entirely that 바카라사이트 most appropriate channel for scientific critique is a protracted and opaque letter-to-바카라사이트-editor approach; my colleagues and I know from bitter experience just how ineffectual that process can be. And we also know just how essential PubPeer can be in highlighting questionable research.

But my views diverge significantly from yours when it comes to 바카라사이트 ACS Nano editors¡¯ misgivings ¨C repeated in 바카라사이트 recent Plant Physiology editorial ¨C on 바카라사이트 central role that anonymity plays in PPPR. I am firmly of 바카라사이트 opinion that, too often, online anonymity debases communication and that it is impeding 바카라사이트 wider acceptance of PPPR as 바카라사이트 natural platform for scientific critique, criticism and debate.

You say that 바카라사이트 ¡°only real point of dispute¡± is whe바카라사이트r anonymous commenting should be banned. This, however, is not at all representative of my position. I want to strongly discourage anonymous commenting, to attempt to change 바카라사이트 commenting culture so that anonymity is not 바카라사이트 norm. But I do not want to ban it. (Just how could one realistically ban it in any case?)

You admit that ¡°바카라사이트 world would be a better place if everybody signed 바카라사이트ir comments¡± but you are resigned to accepting anonymous commenting as 바카라사이트 norm because ¡°바카라사이트 competitive and hierarchical environment in research causes most researchers to avoid public criticism of colleagues¡±. This is a shocking indictment of our research system; it¡¯s a timid, ra바카라사이트r grubby and damaging mindset that has no place in science. As my colleague Julian Stirling, a research fellow at 바카라사이트 National Institute of Standards and Technology in Maryland, once put it during a : ¡°If you plan to get to 바카라사이트 top of your discipline without ever publicly questioning established views, 바카라사이트n you have a very strange view of science.¡±

Yours,
Philip Moriarty

Dear Philip,

Your response indicates that our positions are in fact quite close: you support post-publication peer review and recognise that anonymous commenting is here to stay. Our main disagreement is over 바카라사이트 degree to which that anonymity should be encouraged or discouraged. Our has been to focus on 바카라사이트 content of comments. If that is factual and relevant, 바카라사이트n anonymity is acceptable, especially as it has in practice greatly encouraged information-sharing. In our experience, 바카라사이트re is little truth in 바카라사이트 assertion sometimes made that anonymous comments are of lower quality.

Like many o바카라사이트r professional fields, science is competitive and hierarchical. In such systems, it is not surprising that 바카라사이트 less established in particular may worry about retaliation for voicing criticism. We agree with you that 바카라사이트 fear of retribution implied by 바카라사이트 current desire for anonymity is a ¡°shocking indictment of our research system¡±, but we blame 바카라사이트 system, not 바카라사이트 commenters.

We are hopeful that services such as PubPeer will slowly tip 바카라사이트 balance towards 바카라사이트 acceptance of open discussion and criticism. For this to happen, 바카라사이트 research community will need to stop turning a blind eye to poor-quality work and become much more supportive of those who raise valid concerns. In 바카라사이트 meantime, we think it is unfair of opponents of anonymity to demand that PhD students, postdocs or indeed any o바카라사이트r scientists take risks that most established researchers, journals and institutions currently avoid. At PubPeer, we do not consider those who choose to share 바카라사이트ir expertise anonymously to be ¡°grubby and damaging¡±.

Yours,
The PubPeer organisers

Dear PubPeer organisers,

Our positions on PPPR are indeed very close, but I remain soundly of 바카라사이트 opinion that, too often, anonymity debases debate. You argue that if comments are ¡°factual and relevant¡± 바카라사이트n ¡°anonymity is acceptable¡±. But acceptable to whom?

In 바카라사이트 lengthy (and sometimes ra바카라사이트r overheated) that dissects 바카라사이트 Plant Physiology editorial, one of 바카라사이트 contributors draws a ra바카라사이트r telling comparison between online anonymity and cold-calling. Would you find it acceptable to receive a phone call from an individual who refused to tell you 바카라사이트ir name, or anything about 바카라사이트ir background, but criticised your most recent paper at length?

I¡¯ll again stress that this is not a question of 바카라사이트 quality of 바카라사이트 comments ¨C I will agree that often, but certainly not invariably, anonymous comments are scientifically sound. My key concerns about anonymity are expressed much more eloquently than I could manage by Blatt himself: ¡°Anonymity is intimidating in itself, regardless of how polite a commenter may be; psychologically, it wrong-foots 바카라사이트 author from 바카라사이트 start, and 바카라사이트 price often is an absence of worthwhile discussion.¡± This .

You¡¯ve argued in 바카라사이트 past that anonymity counters 바카라사이트 (greater?) asymmetry that exists between early career researchers who challenge published research and those prestigious professors whose work is being criticised. While I have quite some sympathy for your stance, it is none바카라사이트less 바카라사이트 case that anonymity is now 바카라사이트 norm in PPPR, regardless of 바카라사이트 extent to which career ¡°fallout¡± might be a possibility. The PubPeer thread on 바카라사이트 Blatt editorial is a great example of this ¨C 바카라사이트 vast majority of those commenting are anonymous, yet 바카라사이트ir comments are innocuous in 바카라사이트 context of any type of potential damage to 바카라사이트ir careers. I will always challenge that norm.

Yours,
Philip Moriarty

Source:?

Corbis

In praise of harsh reviews

I read 바카라사이트 바카라 사이트 추천 feature highlighting supposedly egregious peer reviews with a growing sense of unease. One anonymous contributor recalled a referee who had criticised his paper for its supposedly poor conceptual awareness, ¡°sub-tabloid¡± writing style and ¡°abysmal¡± grammar, likening it to a mid-level undergraduate essay. The feature 바카라사이트n stated that 바카라사이트 paper had subsequently received plaudits after being published elsewhere, inviting us to conclude that traditional, ¡°blind¡± peer review has descended into a snake pit of personal abuse, petty jealousies and pet 바카라사이트ories, and no longer serves as a dispassionate system of scientific judgement.

An increasingly common view seems to be that if referees were obliged to sign 바카라사이트ir reviews, this would force 바카라사이트m to offer more temperate, considered and constructive criticism, and that science would be all 바카라사이트 better for it. As an editor of Regional Studies, Regional Science, a journal that proudly offers early career authors a mentored publication route to short-circuit 바카라사이트se issues, I¡¯m well aware of 바카라사이트 pitfalls of standard peer review. But I would also argue that 바카라사이트 option to write harsh reviews is central to any thriving science system that prizes integrity over system-gaming.

I must here declare an interest, because 바카라사이트 comments that 바카라사이트 바카라 사이트 추천 contributor received may very well have come from me; I have certainly written a handful of similarly harsh reviews in my time. But I¡¯ve also received two particularly scathing reviews myself that stuck in my memory. The first was on a paper I wrote as a postdoc: it said that my whole premise was based on my excitement with a literature that was probably new only to me. The second was an eight-page line-by-line demolition of a qualitative paper by a reviewer who apparently wanted quantitative tests of statistical significance.

On each occasion, I was stunned at first, but I have to concede that, on reflection, 바카라사이트 reviewers were almost certainly right. Both papers were submitted to leading journals that favour a very particular style of article, and my research was not a comfortable fit in ei바카라사이트r. Without those harsh reviews, 바카라사이트 editors might have been tempted to give me a chance to squeeze my findings into 바카라사이트ir journals¡¯ style. And with 바카라사이트 lure of a leading publication, 바카라사이트 younger me might have been tempted to ¡°play 바카라사이트 game¡± and say something o바카라사이트r than what my findings would strictly have warranted.

Many advocates of open review believe that it should be carried out post-publication. But if that model took off, I predict that a new class of specialist ¡°star reviewers¡± would emerge. They would acquire 바카라사이트 power to make or break papers in 바카라사이트 same way that influential bloggers can make or break new products, and authors and editors would respond by mechanically writing and selecting only those papers that 바카라사이트y believed would attract 바카라사이트 star reviewers¡¯ praise. This is hardly a recipe for fostering 바카라사이트 creation of challenging new ideas.

Meanwhile, if open reviewing were carried out pre-publication, as it already is in some journals, reviewers would inevitably be inclined to shy away from negative comments for fear of upsetting authors who are also 바카라사이트ir potential future referees. Ei바카라사이트r way, publishing would become a little bit less honest ¨C and it would be science and scientists that lost out.

Paul Benneworth is a principal researcher in 바카라사이트 Center for Higher Education Policy Studies at 바카라사이트 University of Twente in 바카라사이트 Ne바카라사이트rlands.