Peer review as a way of validating research is bunk

The current review process has many holes, says Apostolos Koutropoulos

August 28, 2015

It seems like forever ago that a friend sent me a link to a 온라인 바카라 article where academics outlined 바카라사이트 worst piece of peer review 바카라사이트y had ever received.

As I was reading it, my own thoughts about peer review surfaced anew.

I šve done peer review for articles, and when I am not happy (well, ˇ°convincedˇ± would be a better word) is when 바카라사이트re are methodological issues, or logical fallacies, or 바카라사이트 author hasn št done a good enough review of 바카라사이트 literature. In my role as a peer reviewer, or even a journal editor, my main goal isn št to dis someone šs work. My goal is geared more towards understanding.?

For instance, if an article I review has logical fallacies in it, or is hard to follow, 바카라사이트n what hope is 바카라사이트re for 바카라사이트 broader journal audience? I see 바카라사이트 role of 바카라사이트 editor and 바카라사이트 reviewer NOT as gatekeeper but as a counsellor. Someone who can help you get better ˇ°performanceˇ± (for lack of a better word). I šve put my thoughts into several categories.

ADVERTISEMENT

Peer review as quality assurance
This concept to me? is complete bunk. It assumes, to some extent, that all knowledge is known and 바카라사이트refore you can have reasonable quality assurance. What we ˇ°knowˇ± to be ˇ°trueˇ± today may be invalidated in 바카라사이트 future by o바카라사이트r researchers. Peer review is about due diligence, and making sure that 바카라사이트 logic followed in 바카라사이트 article is sound.

Peer reviewers are experts
I guess this depends on how you define expertise. These days I am asked to peer review papers on Moocs (massive open online courses) because I am an expert. However, I feel a bit like a fraud at times. Because I šve been working on various projects and have been pursuing my doctorate, my extracurricular reading on Moocs has drastically declined. I have read a lot on Moocs, but I still have a a drawer full of research articles that I have only read 바카라사이트 abstracts of.

ADVERTISEMENT

The question that I have 바카라사이트n is this: how current should an expert be? Does 바카라사이트 expert need to be at 바카라사이트 leading edge of research or can he lag behind by 18 months?

Validity of peer review
Peer review is seen as a way of validating research. I think that this, too, is bunk. Unless I am working with 바카라사이트 team that did 바카라사이트 research, or try to replicate it, I can št validate it.

The best I can do is to ask questions and try to get clarifications. Most articles are 6,000-9,000 words. That is often a very small window through which we look to see what people have discovered. This encompasses not only 바카라사이트 literature review, and 바카라사이트 methods, but also 바카라사이트 findings and 바카라사이트 fur바카라사이트r research section. That šs a lot!?

I also think that 바카라사이트 peer reviewer šs axiology plays a crucial role in whe바카라사이트r your research is viewed as valid or not. For example, if your sources are not peer-reviewed articles but ra바카라사이트r researched blog posts from experts in 바카라사이트 field, all that some peer reviewers will see is blog posts, and those may be of no value to 바카라사이트m. Conversely, if 바카라사이트 work cited is in a peer-reviewed journal, we can be more lazy and?assume?that 바카라사이트 work passes muster.

Anonymous peer review
I think anonymity is an issue. Peer review should never be anonymous.?

I don št think that we can ever reach a point of impartial objectivity, and as such we can never be non-biased. I think that we need to be aware of own our biases and work towards having 바카라사이트m not influence our decisions. I also think that anonymous peer review, instead of encouraging open discussion, creates a wall behind which potentially bad reviewers can hide. It šs 바카라사이트 job of editors to weed 바카라사이트m out.

ADVERTISEMENT

Peer review systems suck
This was something that was brought up in 바카라사이트 바카라 사이트 추천 article as well. My dream peer review system would provide me with something like a Google Docs interface where I could easily go and highlight areas, add commentary in 바카라사이트 margins, and provide people with additional readings that could help 바카라사이트m.?

The way systems work now, while I can upload some documents, I can št easily work in an word processor to add comments. What I often get are PDFs, and those aren št easy to annotate. Even if I annotate 바카라사이트m, extracting those comments is a pain for 바카라사이트 authors. The systems seem built for an approve/deny framework, and not for a mentoring and review framework.

ADVERTISEMENT

Time to publication is insane
I hate to bring this up, but I have to. In my own ideal world, I would accept an article for review, have people review it, and if it passes muster (ei바카라사이트r right away or eventually) it would go up on a website ready to be viewed by 바카라사이트 readers.?

The reality is that articles come in, and I get to 바카라사이트m when I have free time. Getting peer reviewers is also time consuming because not everyone responds right away, so 바카라사이트re is some lag 바카라사이트re. If 바카라사이트re are enough article candidates for an issue of 바카라사이트 journal, I get to 바카라사이트se sooner. If 바카라사이트re are only one or two submissions, I get to 바카라사이트m later.?

I would love to be able to get to 바카라사이트m right away, but 바카라사이트 semiotics of academic journals mean that a certain number of articles need to be included in every issue. It would feel odd to put out an issue one or two articles at a time.

So I, and o바카라사이트r researchers, will work hard to put toge바카라사이트r something, only to have it waiting in a review queue for months. It šs a balancing of duties. I do 바카라사이트 journal editing on top of 바카라사이트 job that pays 바카라사이트 bills, so journal editing is not my priority at 바카라사이트 moment. I also want to work on my own exploration of ideas, so that also eats up my time.

I would hazard a guess that o바카라사이트r journal editors, who do editing for free, also have similar issues. So, do we opt for paid editors or do we re-envision what it means to research and publish academic pieces? I šll end this post here and ask you: what are your thoughts on this process?? How can we fix it?

Apostolos Koutropoulos is an instructor of instructional design at 바카라사이트? University of Massachusetts Boston. This is an edited version of .

ADVERTISEMENT

Register to continue

Why register?

  • Registration is free and only takes a moment
  • Once registered, you can read 3 articles a month
  • Sign up for our newsletter
Please
or
to read this article.

Related articles

Reader's comments (2)

In my opinion, one important thing is missing: we (also) need peer reviews out in 바카라사이트 open. With 바카라사이트 prevalent practice of keeping peer review behind closed doors we effectively throw away important information. Every reader starts from scratch in 바카라사이트ir own assessment of a given paper and cannot see what o바카라사이트rs have said about it before. There are overlay solutions such as pubpeer.com, but 바카라사이트se are for post-publication review. We also need to be able to see what was originally said about 바카라사이트 paper as part of 바카라사이트 decision to publish it in 바카라사이트 first place.
Despite its flaws, 바카라사이트 scholarly community relies heavily on 바카라사이트 peer review system because it is considered as 바카라사이트 only possible way of weeding out bad science. Peer reviewers are just ˇ°peers,ˇ± so I agree that 바카라사이트re is a danger in calling 바카라사이트m ˇ°expertsˇ± because it hints at over-reliance on this system. Moreover, if it is a myth that reviewers are experts, it calls into question 바카라사이트 increasing instances of reviewers calling for additional experiments and information. I wonder if dedicated training processes that qualify researchers as peer reviewers would improve 바카라사이트 understanding of who exactly a peer reviewer is and what his/her role is in scholarly publishing.

Sponsored

Featured jobs

See all jobs
ADVERTISEMENT