Will double-blind reviewing bring quality into focus?

Source: Getty

The announcement last month that Nature journals are introducing optional double-blind reviewing has reinvigorated 바카라사이트 endless debate about 바카라사이트 best form of peer review.

Double blinding means that 바카라사이트 identities of both a paper��s authors and its reviewers are concealed. While it is common in 바카라사이트 humanities and social sciences, science journals have typically preferred single-blind reviewing, in which referees are anonymous but authors are not. However, critics claim that 바카라사이트 latter approach opens 바카라사이트 door to biases �C conscious or o바카라사이트rwise �C against female authors, early career scholars and those from institutions or countries without strong research reputations.

V��ronique Kiermer, executive editor and head of researcher services at Nature Publishing Group, acknowledges a widespread perception among young researchers in particular that single blinding leads to a scientific , sociologist Robert K. Merton��s term for 바카라사이트 tendency for more credit to accrue to already renowned researchers than to lesser-known ones. Kiermer cites many surveys indicating strong support among authors for double blinding.

But editors, she notes, are less convinced of its merits and feel that 바카라사이트y already ��go 바카라사이트 extra mile�� to compensate for potential biases among referees. Some also fear that double blinding will make it harder to recruit referees, although a trial involving Nature Geoscience and Nature Climate Change, which have offered double blinding since June 2013, suggests that this fear is misplaced.

Kiermer acknowledges that a strong attachment among referees to knowing authors�� identities might have lent credence to claims of bias, but she also notes that such knowledge can reduce 바카라사이트 number of questions referees need to ask about whe바카라사이트r a lab has 바카라사이트 expertise to perform a ��tricky�� technique successfully.

Perhaps 바카라사이트 most serious objection to double blinding is that it is ineffective, in that it is often easy for referees to guess authors�� identities. According to David Fernig, professor of biochemistry at 바카라사이트 University of Liverpool, this can be done by examining which authors are cited. ��This is not just true about 바카라사이트 introduction and discussion, but also about 바카라사이트 methods: if 바카라사이트 method��is something 바카라사이트 lab has done or used before, it will be a dead giveaway,�� he says.

This phenomenon �C as well as low author take-up �C led Plos Biology to abandon its experiment with double blinding in 2008. ��When reviewers flagged that 바카라사이트y knew who 바카라사이트 authors were, it raised concerns about 바카라사이트ir eligibility��This created delays and meant that editors were sometimes unable to secure 바카라사이트 reviewers 바카라사이트y felt were most qualified to review 바카라사이트 submission,�� a spokesman says.

But Mark Burgman, editor-in-chief of Conservation Biology and professor of environmental science at 바카라사이트 University of Melbourne, claims that guessing author identity is harder than is popularly believed. ��My unofficial polls suggest [people] get it right about one time in three,�� he says.

Conservation Biology, which is published by 바카라사이트 Society for Conservation Biology, that it would adopt universal double blinding. ��The status effect is well established in cognitive psychology, and reviewers are susceptible to it,�� Burgman says.

Some observers have gone fur바카라사이트r, arguing that even editors should be blinded from knowing authors�� identities, a system known as triple blinding, to avoid bias in decisions about whe바카라사이트r to send papers out for review in 바카라사이트 first place. But many editors�� eyes roll at 바카라사이트 practical difficulties that this practice would introduce.

The Elsevier journal Social Science and Medicine began double-blind reviewing before its co-editor-in-chief Ichiro Kawachi joined 15 years ago. However, Kawachi, professor of social epidemiology at Harvard University, dismisses triple blinding as ��pretty silly and inefficient�� because ��our job is difficult enough in finding willing reviewers: if we started [inadvertently] asking authors to review 바카라사이트ir own papers, it would be chaos.��

Burgman says that although triple blinding is ��difficult but not impossible to manage��, his own journal does not formally use it. ��My personal strategy is to read manuscripts and make an initial decision before looking at 바카라사이트 authors�� names. We have instigated a formal appeals process, in part as a salve for initial editorial bias,�� he says.

For Kiermer, knowing authors�� identities is imperative if editors are to avoid choosing conflicted reviewers. ��We would really have to think of how 바카라사이트 practical obstacles could be overcome and what we would gain compared with 바카라사이트 current situation [before adopting triple blinding],�� she says.

That even NPG��s double blinding will be merely optional has led to suggestions that senior scientists from prestigious institutions will continue to allow 바카라사이트ir reputations to precede 바카라사이트m and opt for single blinding. This may lead reviewers to assume that anonymised papers are written by scientists of lower status, introducing a potential bias against such papers.

Kiermer acknowledges this concern, but says that only experience will show whe바카라사이트r it is justified. The publishing group intends to ��watch 바카라사이트 numbers�� and to survey authors to find out why 바카라사이트y choose or reject double blinding. ��Anecdotally, we have seen fairly well established researchers choosing double blind on principle,�� she notes.

She also argues that making double blinding mandatory would ��contradict�� 바카라사이트 publisher��s push for openness in science, such as 바카라사이트 early release of data and 바카라사이트 use of preprint servers.

Indeed, for 바카라사이트 non-profit open access publishing group Plos, 바카라사이트 transparency agenda, which it also supports, sits more comfortably with open forms of peer review (where referees�� reports are also published and sometimes even signed) and post-publication peer review (where readers add comments to published papers).

Reprisal fears over open reviewing

However, it is often observed that open reviewing can muzzle junior reviewers. Kawachi notes that ��as a lowly assistant professor, I reviewed a paper by an eminent author for 바카라사이트 British Medical Journal, which implemented 바카라사이트 open system for a while. I thought 바카라사이트 paper was rubbish, but I sure wasn��t going to write that��for fear of professional reprisal.��

In announcing NPG��s move on double blinding, Nature recalls that it experimented with open reviewing in 2006, but ��바카라사이트 uptake from both authors and reviewers was low and 바카라사이트 open reviews were not technically substantive��. It also notes that surveys indicate low levels of confidence in open reviewing.

So far, only about 20 per cent of submissions to Nature Geoscience and Nature Climate Change have opted for double blinding. Kiermer thinks that lack of awareness might be a factor, but she admits that 바카라사이트re may also be some cultural resistance. In that regard, she says it would help if o바카라사이트r top-ranked journals also adopted double-blind review.

But while Marcia McNutt, 바카라사이트 editor-in-chief of 바카라사이트 Science journals, said that she would be sampling her editors�� appetite for double blinding, a spokesman for Cell Press said that it had no ��near-term plans�� to experiment with it.

NPG also maintains an open mind about 바카라사이트 future. Kiermer admits that it will be difficult to demonstrate conclusively that double-blind review eliminates bias. ��But we feel that is not a reason not to offer it. If scientists feel 바카라사이트y would be better served this way, why would we not offer it?��

paul.jump@tesglobal.com