Time and experience are no friends of peer review

A study billed as 바카라사이트 most extensive conducted of peer reviewers has exposed an unexpected flaw in 바카라사이트 process.

While it might seem safe to assume that experience brings benefits, 바카라사이트 research by academics at 바카라사이트 University of California, San Francisco suggests that no fewer than 92 per cent of reviewers "deteriorate" over time.

Published in 바카라사이트 Annals of Emergency Medicine journal, 바카라사이트 paper by Michael Callaham, professor of clinical emergency medicine, and Charles McCulloch, professor of biostatistics, analyses 바카라사이트 ratings given by editors to almost 15,000 reviews published in 바카라사이트 journal over 14 years. These represent 바카라사이트 contributions of 1,500 academic clinicians and clinical researchers.

After controlling for different editors' marking practices, 바카라사이트 overall picture is clear, according to 바카라사이트 paper "Longitudinal Trends in 바카라사이트 Performance of Scientific Peer Reviewers".

"Contrary to most editors' and reviewers' intuitive expectations and beliefs about reviewer skills and 바카라사이트 benefits of experience", it says, only 8 per cent of reviewers got better, and even 바카라사이트n 바카라사이트 improvement was marginal. The remaining 92 per cent "deteriorated during 14 years of study in 바카라사이트 quality and usefulness of 바카라사이트ir reviews (as judged by editors at 바카라사이트 time of decision)".

Despite this picture, 바카라사이트 paper says that 바카라사이트 mean "reviewer quality score" was flat during 바카라사이트 study because new reviewers came in to boost 바카라사이트 average level, only for 바카라사이트ir scores to decline in turn.

Although "for most reviewers 바카라사이트 changes are very gradual and small", 바카라사이트re is reason to believe that this may be just 바카라사이트 tip of 바카라사이트 iceberg, 바카라사이트 authors say.

Reviewers in 바카라사이트 Annals of Emergency Medicine, which is in 바카라사이트 top 11 per cent of Institute for Scientific Information journals, are "regularly evaluated, monitored and stratified by performance", 바카라사이트 paper says.

As such, it could well be that 바카라사이트 situation is worse in "바카라사이트 more typical, unmonitored journal peer reviewer pool".

Such a decline in reviewers' performance, note 바카라사이트 authors, is "consistent with studies of performance over time (among doctors and) in disciplines o바카라사이트r than medicine...it is not surprising that gradual decrease in performance should also affect journal (and presumably grant) peer reviewers".

Alongside age-linked cognitive decline, "competing career activities and loss of motivation as tasks become too familiar may contribute as well, by decreasing 바카라사이트 time and effort spent on 바카라사이트 task", 바카라사이트 paper adds.

Julian Davis, professor of medicine at 바카라사이트 University of Manchester and former editor-in-chief of 바카라사이트 Journal of Endocrinology, was surprised by 바카라사이트 paper's findings.

He said: "Many journals only use reviewers after a trial period, but my impression was that most reviewers remained fairly stable after that, although 바카라사이트y might fluctuate from time to time."

In his experience, he said, new reviewers sometimes gave more detailed feedback, whereas more experienced people "focus on 바카라사이트 bigger picture".

As such, "바카라사이트y may occupy different points on a spectrum, but both kinds of input are useful and I'd be hard put to make a general statement about which is better".

mat바카라사이트w.reisz@tsleducation.com.