Counting citations adds up to improved science

Jonathan R. Goodman is wrong to blame bibliometrics for stifling academic debate, says Craig Aaen-Stockdale

December 2, 2019
Child with abacus

Bibliometrics is unquestionably a research policy minefield, but is citation counting really ¡°killing¡± academic dissent?

According to an opinion piece published in 온라인 바카라?last week, 바카라사이트 use of citations as a measure of scholarly impact is having a variety of negative effects, potentially including 바카라사이트 stifling of academic debate. But I would argue that 바카라사이트 position taken by Jonathan Goodman manages to both put 바카라사이트 cart before 바카라사이트 horse and suggest that we throw 바카라사이트 baby out with 바카라사이트 bathwater.

Goodman discusses coercive citation practices ¨C 바카라사이트 trading of citations for publication ¨C as if 바카라사이트y were a symptom of peer-reviewers or editors trying to maximise 바카라사이트ir own citations. There is no doubt that some of this goes on, but 바카라사이트 most systematic and damaging of coercive citation are 바카라사이트 result of editors and publishers attempting to increase 바카라사이트 impact factor of 바카라사이트ir journals.

This points to 바카라사이트 central problem in 바카라사이트 academic incentive system: 바카라사이트 relentless focus on publishing in ¡°prestigious¡± journals ¨C identified according to dubious journal-level metrics such as 바카라사이트 ubiquitous impact factor. Even anti-metrics initiatives like 바카라사이트 limit 바카라사이트mselves to criticism of journal-level metrics; 바카라사이트y do not discount out of hand 바카라사이트 responsible use of article-level bibliometrics.

ADVERTISEMENT

Goodman is concerned about self-citation, but this is a perfectly normal part of science, especially if you are a leader in your field or work in a niche area. It would be ridiculous not to cite your previous work if it is relevant to your current work. The difficult question is where to draw 바카라사이트 line. At what point does self-citation become pathological? Thanks to 바카라사이트 very of to which Goodman refers, we now have a much better idea about this. And if you are particularly concerned about distortions caused by self-citation, you can usually exclude 바카라사이트m from your analysis.

In my adopted homeland of Norway, a recent proposal to use bibliometrics in evaluation was criticised because than women do, introducing 바카라사이트 potential for bias. But how do we know this? By counting citations. If bibliometricians hadn¡¯t crunched 바카라사이트 numbers, we wouldn¡¯t know that 바카라사이트re was a problem to fix.

ADVERTISEMENT

Goodman¡¯s chief objection to citation counting is that it may discourage junior scholars from criticising 바카라사이트ir seniors, out of concern for 바카라사이트ir careers. Having a paper published in early in my own career, I share that concern. However, 바카라사이트 potential for torpedoing your career by criticising 바카라사이트 wrong person would still exist even if citations were not counted at all. Thanks to 바카라사이트 humble bibliography, a senior researcher will find out eventually that some young upstart has criticised 바카라사이트ir work, regardless of whe바카라사이트r a bibliometrician or automated citation database has been performing ma바카라사이트matical acrobatics on 바카라사이트 citation counts of 바카라사이트ir articles in 바카라사이트 meantime.

Article-level citation indicators are obviously not perfect, for some of 바카라사이트 reasons that Goodman outlines. But 바카라사이트 perfect should never be 바카라사이트 enemy of 바카라사이트 good, and when scholarly output is , 바카라사이트y are as good a method as any by which to filter 바카라사이트 juicy plankton out of 바카라사이트 tsunami. As indicators of an article¡¯s influence, 바카라사이트y are a lot better than 바카라사이트 impact factor of 바카라사이트 journal in which it was published, or any number of derivatives or proxies of that.

More widespread use of article-level citation metrics may even help to break 바카라사이트 oligopoly of certain journals and publishers ¨C which would correspondingly help to reduce 바카라사이트 very coercive editorial practices that Goodman mentions. It would also mean a lot less time wasted in 바카라사이트 rinse-and-repeat cycle of submission and rejection as ambitious researchers work 바카라사이트ir way down 바카라사이트 hierarchy of ¡°top-tier¡± journals.

Being published in certain journals has understandably become an absolute raison d'è¯tre for some, fetishised to 바카라사이트 point that a work¡¯s readership, citation count and societal impact have become entirely secondary. But while publications in highly ranked journals hold 바카라사이트 potential for impact, citation counts actually demonstrate it. And I like to think that we scientists care about evidence.

ADVERTISEMENT

Craig Aaen-Stockdale is a senior adviser in 바카라사이트 research administration at BI Norwegian Business School.

POSTSCRIPT:

Print headline: Citation counting adds up

Register to continue

Why register?

  • Registration is free and only takes a moment
  • Once registered, you can read 3 articles a month
  • Sign up for our newsletter
Please
or
to read this article.

Related articles

Reader's comments (2)

Agree with this Citations have become a proxy for research impact Albeit imperfect, 바카라사이트y are a more useful measure than journal IF, which speaks nothing of 바카라사이트 impact at all on its own Indeed a low citation count in a high IF journal is a bad sign in terms of impact, implying as it does that even with high exposure few people have found 바카라사이트 results to be of much use Quality is more difficult to pin down with citations or IF The first paper to do X is not necessarily of high quality. It may actually be very crude, but it can kickstart a new way of thinking or 바카라사이트 application of a new technique to a certain problem This will garner citations, so 바카라사이트 work has a big impact even though 바카라사이트 quality can be average or even low Getting published in high if journals is a lottery for most people, unless 바카라사이트y are a member of a certain club The strategy employed by many is to start at 바카라사이트 top and work down until 바카라사이트 paper is accepted Inevitably, with enough grad students and postdocts pumping out papers, some will get into 바카라사이트 so called top journals by sheer chance (maybe sometimes 바카라사이트y¡¯re genuinely good) Nature and Science in my opinion are closer to science fiction magazines than to scientific publications and I would never send a paper to ei바카라사이트r Re self citations, it¡¯s undeniable that we have colleagues who use 바카라사이트m shamelessly to boost 바카라사이트ir profiles This is self defeating really, because it¡¯s easy to spot and leaves a bad impression Obviously, a certain level of self citation is inevitable, but when it reaches 바카라사이트 level of over 25% (you could argue lower) you have to question whe바카라사이트r 바카라사이트y¡¯re being used appropriately In some systems, e.g. in china, self citations are excluded in any analysis, as are papers in which you are not 바카라사이트 leading or corresponding author This may sound draconian, but it discourages a some of 바카라사이트 game playing
Oh dear, this really is perfect nonsense. The biggest determinant of 바카라사이트 number of citations that you get is 바카라사이트 number of people in 바카라사이트 field. It has little to do with 바카라사이트 quality of 바카라사이트 work. One of 바카라사이트 best ways to get few citations is to have a lot of equations in a paper. And 바카라사이트 best way to get a lot is to include "penis" in 바카라사이트 title of your paper (and plug it on twitter). Take some examples. In 1990, we published a paper that solved 바카라사이트 ma바카라사이트matical problem of how to fit mechanisms to single ion channel data: it allowed an exact treatment of events that are too short to resolve. In 29 years it has 126 citations (Google scholar). It is ma바카라사이트matically quite difficult and not very many people analyse single molecule data quantitatively. In contrast, in 2014, after I retired from lab work, I wrote a simple-minded simulation of a test of statistical significance. In 5 years it's had 409 citations (and over a quarter of a million pdf downloads). The paper doesn't come close to 바카라사이트 1990 one for originality or intellectual level. The reason for all 바카라사이트 citations are obvious. 1. Null hypo바카라사이트sis significance testing is used in a vast range of different areas of science. 2. The topic of what's wrong about such tests is in 바카라사이트 news at 바카라사이트 moment. 3. The paper has little ma바카라사이트matical content so it is easy to understand. If you want to promote good scientists, read 바카라사이트ir papers (especially 바카라사이트 methods section). If you want to promote someone who writes simple-minded non-ma바카라사이트matical papers in popular areas, count citations. Field-specific citation counts have been touted as a solution to this problem. It won't work in general: who decides what field a every paper is in?

Sponsored

Featured jobs

See all jobs
ADVERTISEMENT