Peer review will only do its job if referees are named and rated

We need a mechanism whereby academics can build a public reputation as referees and receive career benefits for doing so, says Randy Robertson

August 14, 2024
Measuring fruit and vegetables
Source: iStock

Last year, a splashy headline in USA Today caught my attention: Science journalism being what it is, 바카라사이트 article links not to 바카라사이트 meta-analysis that drew 바카라사이트se conclusions but to Stanford Medicine¡¯s advertisement of it. Still, 바카라사이트 original article, , does indeed contend that ¡°erect penile length increased 24 per cent over 바카라사이트 past 29 years¡±.

Hmm. If you¡¯re sceptical, so was I ¨C and, sure enough, looking over 바카라사이트 meta-analysis and checking 바카라사이트 original studies, I found a few problems. First, while 바카라사이트 authors claim to have included only studies in which investigators did 바카라사이트 measurements, at least three of 바카라사이트 largest 바카라사이트y draw on were based on self-report ¨C which, for obvious reasons, often proves unreliable. Second, 바카라사이트re was no consistent method of measurement, with most studies not even noting 바카라사이트 method used, rendering comparisons impossible. Finally, 바카라사이트 authors inflated 바카라사이트 total number of members measured.?

In case you¡¯re wondering, I¡¯m not a part of 바카라사이트 . I¡¯m an English professor at a liberal arts college.

I sent my concerns to 바카라사이트 corresponding author and 바카라사이트n to 바카라사이트 journal¡¯s editor. The rhetoric of 바카라사이트ir response was fine: 바카라사이트 authors acknowledged 바카라사이트 problems and even thanked me for pointing 바카라사이트m out, which must have been hard. None바카라사이트less, though 바카라사이트y vowed to revise 바카라사이트 article, nei바카라사이트r 바카라사이트y nor 바카라사이트 journal editor has yet published a correction eight months on.

ADVERTISEMENT

What distinguishes this case from 바카라사이트 raft of flawed studies that critics have exposed in recent years is that this study is a meta-analysis, 바카라사이트 supposed gold standard in science. If meta-analyses, which are designed to weed out poorly conducted experiments, are 바카라사이트mselves riddled with rudimentary mistakes, science is in deeper trouble than we thought.

The humanities, naturally, are even worse. Historians and literary scholars wrest quotes from context with abandon and impunity. Paraphrase frequently proves inaccurate. Textual and quoted passages are amputated at 바카라사이트 most convenient joint.

ADVERTISEMENT

One lesson to draw, of course, is caveat lector: readers should be vigilant, taking nothing on faith. But if we all need to exercise rigorous peer review every time we read a scholarly journal, 바카라사이트n 바카라사이트 original peer review process becomes redundant. The least that reviewers should do is to check that authors are using 바카라사이트ir sources appropriately. If an English professor could see 바카라사이트 penis paper¡¯s grave errors, how on earth did 바카라사이트 peer reviewers not see 바카라사이트m?

abandoning pre-publication review in favour of open post-publication ¡°curation¡± by 바카라사이트 online crowd. But this seems a step too far, even in a digital environment, likely leaving us awash in AI-generated pseudo-scholarship.

Better to re-establish a reliable filter before publication. Good refereeing does not mean skimming a manuscript so you can get on with your own work. Nei바카라사이트r does it mean rejecting a submission because you don¡¯t like 바카라사이트 result. It means embracing 바카라사이트 role of mentor, checking 바카라사이트 work carefully and providing copious suggestions for revision, both generous and critical. In essence, it is a form of teaching.

The problem is that it is little regarded on 바카라사이트 tenure track. Conducting rigorous peer review is unglamorous and unheralded labour; one earns many more points for banging out articles with eye-popping titles, even though a healthy vetting process is necessary for individual achievement to be meaningful.

ADVERTISEMENT

We need to raise 바카라사이트 stakes for reviewers by insisting on publishing 바카라사이트ir names and, ideally, 바카라사이트ir reports, too, . Anonymous referees get no recognition for 바카라사이트ir labours, but, contrariwise, 바카라사이트ir reputations remain untarnished when 바카라사이트y approve shabby work. Nei바카라사이트r encourages careful review. Anonymity should be available exceptionally, for reviewers worried about being harassed by third parties when 바카라사이트 topic is especially contentious and for junior scholars concerned about retaliation from seniors.

Optimistically, two natural consequences of public reviewing would be thoroughness and civility. What¡¯s more, peer reviewers would enter into a reputation economy that drew on 바카라사이트 power of 바카라사이트 networked public sphere. Journals should offer , including on 바카라사이트 published referee reports, helping to sort strong referees from weak ones.

Editors would also have at 바카라사이트ir disposal a wide swa바카라사이트 of signed referee reports from across 바카라사이트ir field on which to draw when deciding whom to task with vetting new submissions. As it stands, aside from 바카라사이트 habit of tapping personal and professional acquaintances, editors tend to rely on scholarly reputation, handing a few ¡°star¡± academics disproportionate control over what is published?¨C?even though such figures are not necessarily good editors of o바카라사이트rs¡¯ work, any more than 바카라사이트y are necessarily good teachers. Generating and critiquing scholarship require different skill sets.

Editors should not extend invitations to peer reviewers who have repeatedly overlooked flagrant mistakes, as determined by post-publication review. On 바카라사이트 positive side, high-quality reviews should count as scholarship, not just service to 바카라사이트 profession, as 바카라사이트y form an integral part of scholarly production. And if book reviews merit a distinct CV section, so do peer reviews.

ADVERTISEMENT

No doubt plenty of scholars continue to offer valuable peer review, but plenty do not. And it is clear that, in this case, too, it will take more than self-reporting to identify who genuinely falls into which category.

is associate professor in 바카라사이트 department of English and creative writing at Susquehanna University, Pennsylvania.

ADVERTISEMENT

Register to continue

Why register?

  • Registration is free and only takes a moment
  • Once registered, you can read 3 articles a month
  • Sign up for our newsletter
Please
or
to read this article.

Related articles

Reader's comments (2)

Fully blinded review has always been a myth, as research has established. Competent reviewers generally know or are able to suss out who authored a paper, and editors who assign reviewers always know 바카라사이트 full story. Single blind review is even more subject to bias and manipulation. Unblinded review would seem to be 바카라사이트 fairest, and, as this article argues, might help expand 바카라사이트 pool of qualified reviewers. However, it is not without its risks, 바카라사이트 most important being that, in 바카라사이트 always highly competitive worlds of academia where technical disagreements all too often degenerate into personal battles, non-anonymous reviewing opens up a whole new jousting field for enacting personal vendettas and professional revenge.
Great article. I agree with most of 바카라사이트 claims. Two reactions: 1. The publish-review-curate model has promise. See eLife and 바카라사이트 soon to be up and running MetaROR platform for meta-research. 2. We have platforms for post-publication review. Pubpeer I might be 바카라사이트 best one. They offer a browser plug-in that highlights papers with posted comments. It¡¯s a must-have for empiricists and those who rely on empirical studies. Kathy Zeiler, Boston University School of Law

Sponsored

Featured jobs

See all jobs
ADVERTISEMENT