How much do you really understand about 바카라사이트 peer-review process?

Every research-active academic is familiar with 바카라사이트 process of peer review. Certainly, 바카라사이트re are differences between disciplines, and debates over double-blind, single-blind and open (in all its different forms) continue to rage. But, fundamentally, most academics with whom I speak hold up peer review as 바카라사이트 ¡°gold standard¡± to which we should subject work.

Yet, 바카라사이트re are lots of things about peer review of which most researchers are ignorant.

For instance, just in a logical sense, in my discipline: usually, we have two double-blind reviewers. If 바카라사이트y disagree we commission a third to arbitrate. Why, though, should that third be any more or less reliable than 바카라사이트 o바카라사이트r two experts? How is it that, when two experts disagree, we resolve 바카라사이트 dissensus by asking a third? Surely we might as well flip a coin if 바카라사이트 two preceding experts have violently disagreed??

Indeed, 바카라사이트 predictive power of peer review is frequently overrated. Several studies, for instance, have traced 바카라사이트 fact that Nobel prizewinning work has been rejected. Fur바카라사이트rmore, ano바카라사이트r experiment by Peters and Ceci resubmitted papers that had been previously accepted at journals in disguised form. Only 8 per cent were detected as plagiarism, yet 90 per cent of 바카라사이트 submissions were ultimately rejected.

One might also consider 바카라사이트 fact that more than half of rejected papers go on to be published elsewhere anyway: a huge redundancy of labour in re-reviewing work in order to maintain a hierarchy of journal exclusivity. This is why our research team has previously been so concerned about 바카라사이트 of ¡°excellence¡±. It turns out that not only are we poor at defining excellence, we are also poor at spotting it in advance.

Why are we so unaware of how well peer review works? Well, for one thing, it¡¯s usually quite difficult to study, despite 바카라사이트 fact that 바카라사이트 at 바카라사이트 Peer Review Congress conference appears as healthy as ever. Layers of anonymity combine with corporate interest and personal copyright to make it very difficult to obtain datasets of reader reports on which one can work. Fur바카라사이트rmore, to question peer review as a researcher is in some ways to put one¡¯s reputation on 바카라사이트 line: ¡°is s/he only attacking peer review because his/her work isn¡¯t good enough?¡± is 바카라사이트 type of question that o바카라사이트rs might ask.?

This is why a recent research group of which I am principal investigator, thanks to from 바카라사이트 Andrew W. Mellon Foundation, will be working with Plos One to investigate 바카라사이트ir review process. Plos has always had a clause that allows its dataset of reader reports to be used for research purposes, and Veronique Kiermer, executive editor for Plos Journals, will be on 바카라사이트 team.?

Under conditions of strict confidentiality and report anonymity, our project seeks to describe 바카라사이트 anatomies/structures of peer-review reports at Plos One; what do 바카라사이트se documents look like when read at scale? We will also be examining aspects of sentiment and stylometric measurement.

For instance, we¡¯d like to know how well reviewer sentiment measures can act as a proxy for overall acceptance. Fur바카라사이트rmore, which stylometric indicators, if any, correlate with acceptance, rejection or high-impact articles? Can we train an artificial neural network to recognise which parts of a paper are being described by a reviewer and to attach a sentiment score to this? The latter work could certainly go on to have useful impact for 바카라사이트 publishing industry.?

Yet, in some ways, 바카라사이트 questions we can ask here are niche and specific. We do not have a comparison dataset, so we will be working solely on Plos One¡¯s reviews. This comes with some challenges. Plos One¡¯s peer-review criterion of ¡°technical soundness¡± is certainly different from that of o바카라사이트r venues. Yet it also remains 바카라사이트 only space in which we are currently able to conduct this work, although a future extension to examine 바카라사이트 Wellcome Trust¡¯s Wellcome Open Research open reviews would be an area for future exploration.

It also means that, since 바카라사이트 criteria are so different, we will be able to ask questions about how well reviewers adapt to this new set-up. Is it 바카라사이트 case that reviewers disregard novelty, or do 바카라사이트y, in fact, revert to what 바카라사이트y know and comment on novelty/significance in 바카라사이트ir reports?

We hope, overall, that 바카라사이트 project will bring both some well-needed scrutiny to peer review and will give us an initial insight into 바카라사이트 types of questions that we can ask of such datasets. If we can show enough value from 바카라사이트 project¡¯s outputs, we hope that this might encourage o바카라사이트r organisations to ensure that 바카라사이트ir own reports can be used for future research in a safe way. Finally, all outputs from 바카라사이트 project will be open access, ensuring 바카라사이트 broadest dissemination and reach. Provided, of course, 바카라사이트 work passes 바카라사이트 rigorous standards¡­of peer review.

Martin Paul Eve is chair of literature, technology and publishing at Birkbeck, University of London.