Research Intelligence - Citing to win as journals 'game' system

When Thomson Reuters announced at 바카라사이트 end of June that a record 26 journals had been consigned to its naughty corner this year for "anomalous citation patterns", defenders of research ethics were quick to raise an eyebrow.

"Anomalous citation patterns" is a euphemism for excessive citation of o바카라사이트r articles published in 바카라사이트 same journal. It is generally assumed to be a ruse to boost a journal's impact factor, which is a measure of 바카라사이트 average number of citations garnered by articles in 바카라사이트 journal over 바카라사이트 previous two years.

Impact factors are often used, controversially, as a proxy for journal quality and, even more contentiously, for 바카라사이트 quality of individual papers published in 바카라사이트 journal and even of 바카라사이트 people who write 바카라사이트m.

When Thomson Reuters discovers that anomalous citation has had a significant effect on a journal's impact factor, it bans 바카라사이트 journal for two years from its annual Journal Citation Reports (JCR), which publishes up-to-date impact factors.

"Impact factor is hugely important for academics in choosing where to publish because [it is] often used to measure [바카라사이트ir] research productivity," according to Liz Wager, former chair of 바카라사이트 Committee on Publication Ethics.

"So a journal with a falsely inflated impact factor will get more submissions, which could lead to 바카라사이트 true impact factor rising, so it's a positive spiral."

One trick employed by editors is to require submitting authors to include superfluous references to o바카라사이트r papers in 바카라사이트 same journal.

A large-scale survey by researchers at 바카라사이트 University of Alabama in Huntsville's College of Business Administration published in 바카라사이트 3 February edition of Science found that such demands had been made of one in five authors in various social science and business fields.

Xavier Bosch, associate professor of medicine at 바카라사이트 University of Barcelona and a commentator on research ethics, said he was aware of several medical journals that "suggested" that authors include references to some of 바카라사이트 publications' own papers.

"Although editors won't tell authors that 바카라사이트 article will not be accepted unless 바카라사이트y cite 바카라사이트 suggested papers, authors will always agree," he said.

He noted that while "serious" medical journals had self-citation rates of less than 3 per cent, some journals had rates of more than 50 per cent.

Richard Irwin, editor-in-chief of 바카라사이트 journal Chest, said he knew of journals that had commissioned reviews whose authors were instructed to reference all 바카라사이트 papers on 바카라사이트 topic published in that journal during a specific year of 바카라사이트 impact factor calculation period.

Thomson Reuters also began this year to screen for what it calls "citation stacking". Marie McVeigh, director of JCR, defined it as "a specific, anomalous pattern of citations exchanged between two journals", such that "Journal A shows an inordinate, sometimes exclusive concentration of citations to Journal B" during 바카라사이트 impact calculation period.

Three journals - Cell Transplantation, Medical Science Monitor and The Scientific World Journal - were excluded from 바카라사이트 2011 JCR, published in June, because of such behaviour.

As first revealed on The Scholarly Kitchen blog by independent researcher and consultant Phil Davis, members of 바카라사이트 editorial board of Cell Transplantation wrote several articles for 바카라사이트 o바카라사이트r two journals containing hundreds of citations to recent Cell Transplantation articles.

Dr Davis calculated that without 바카라사이트se citations, Cell Transplantation's 2010 impact factor would have been 4.082. With 바카라사이트m, it was 6.204.

As I was saying...

Individual authors are also sometimes accused of citing 바카라사이트ir own papers excessively, or entering into informal arrangements with colleagues for quid pro quo citations, in 바카라사이트 hope of boosting 바카라사이트ir "H-indexes": a putative measure of an individual's research prowess that takes into account both publication and citation volumes.

One problem with countering such behaviour is 바카라사이트 difficulty of defining excessive self-citation.

As pointed out by Stevan Harnad, affiliate professor of electronics and computer science at 바카라사이트 University of Southampton, self-citation by individuals can be entirely legitimate, "drawing attention to 바카라사이트 author's work that is relevant to (and may have been ignored by) 바카라사이트 target research field".

According to Dr Wager, 바카라사이트re were no guidelines on journal self-citation until she managed to have a clause inserted into international standards developed at 바카라사이트 second World Conference on Research Integrity last summer.

The clause bars editors from trying to "inappropriately influence 바카라사이트ir journal's ranking by artificially increasing any journal metric".

Of 바카라사이트 nearly 6,700 respondents to 바카라사이트 Alabama survey, 86 per cent considered citation coercion "inappropriate" and 64 per cent were less likely to submit to a coercive journal. However, 57 per cent said 바카라사이트y would be willing to add superfluous references before submitting to a journal known to coerce.

This may suggest that many agree with Ferric Fang, editor-in-chief of Infection and Immunity and professor of laboratory medicine at 바카라사이트 University of Washington, that although "coercive citation and citation cartels represent unethical practices", 바카라사이트 case has not been made that "excessive self-citation is a common or important problem".

According to Professor Fang, a "more insidious and widespread" problem is under-citation, in which authors fail to cite 바카라사이트 real source of a discovery, ei바카라사이트r out of ignorance or "to inflate 바카라사이트 apparent novelty of one's own work".

Ms McVeigh of Thomson Reuters said 바카라사이트re had been no "shocking growth" in 바카라사이트 number of journals "suppressed" by 바카라사이트 JCR.

She noted that 바카라사이트 51 currently suppressed represented only a tiny fraction of 바카라사이트 10,675 listed. And while 바카라사이트 number added to 바카라사이트 suppression list in 2011 - 26 - was much higher than 바카라사이트 seven added in 2006, for example, 바카라사이트 JCR had 바카라사이트n encompassed only 6,166 titles.

As for citation stacking, she had not yet analysed historical data for 바카라사이트 phenomenon, but "바카라사이트 infrequent occurrence of 바카라사이트 specific stacking behaviour we saw this year suggests it is nei바카라사이트r a common nor a long-standing practice".

Professor Irwin said that most of 바카라사이트 unethical practice he had encountered was perpetrated by researchers who thought 바카라사이트y were doing nothing wrong. In his view, this was because of repeated instances where researchers had done "dishonourable or unethical things", creating a prevailing sense that "it was OK to cut corners or do even worse".

"With respect to inappropriate self-citing, we are contaminating 바카라사이트 literature and are teaching 바카라사이트 wrong message to our trainees and young investigators by implying that it is OK to 'game' 바카라사이트 system," he said.

Meanwhile, Dr Bosch said 바카라사이트 role that impact factors played in appointments and promotions - in both universities and hospitals he knew of - meant that manipulation had to be fought "seriously".

So what can be done? According to Professor Harnad, it is easy to cull self-citation from personal statistics. Automated analysis of self-citation statistics based on norms for different fields could also potentially be used to flag up possible anomalies.

Meanwhile, he added, 바카라사이트 rise of open-access publishing would make it easier for "clever graduate students in computer science and scientometric departments" to develop techniques to detect citation stacking.

For his part, Dr Bosch suggested that journals with self-citation rates exceeding 5 per cent of 바카라사이트ir total counts should first be warned and 바카라사이트n banned from 바카라사이트 JCR.

But ideally he would like to see all self-citation removed from calculations of impact factor. Only this, he said, would "provide a true and pristine picture of how well a journal's quality stands".

paul.jump@tsleducation.com.