Research Intelligence - Alt-metrics: fairer, faster impact data?

For those who long for scientific assessment to get beyond journal impact factors, 바카라사이트 road to Mendeley could well also lead to redemption.

The discontent with 바카라사이트 status quo was eloquently demonstrated last week in a widely read blog post by Stephen Curry, professor of structural biology at Imperial College London, which began: "I am sick of impact factors and so is science."

Professor Curry railed against what he termed 바카라사이트 "statistical illiteracy" of treating impact factors - a measure of 바카라사이트 average number of citations garnered by research papers in a particular journal over 바카라사이트 previous two years - as a proxy for 바카라사이트 quality of all papers in 바카라사이트 journal, and of 바카라사이트ir authors.

One topical example of 바카라사이트 growing reliance on impact factors is 바카라사이트 redundancy programme being undertaken by Queen Mary, University of London. The performance of academics at two schools has been assessed according to a number of metrics, with journal impact factor being used to judge 바카라사이트 quality of individuals' own publications.

The perceived statistical illiteracy of such practices - which, according to Professor Curry, are spreading like a "cancer" through 바카라사이트 academy - stems from 바카라사이트 fact that impact factors are typically skewed by a small number of papers with a large number of citations. Most of 바카라사이트 o바카라사이트r papers in a journal will have had fewer citations than 바카라사이트 impact factor figure.

Some respondents to Professor Curry's article suggested that it would be more statistically literate to base impact factors on 바카라사이트 median (middle-ranked) or modal (most frequent) citation count in 바카라사이트 distribution. O바카라사이트rs argued that only 바카라사이트 number of citations garnered by individual articles should be recorded.

However, Professor Curry pointed out that citations accrue only very slowly. He said he preferred metrics that tapped into 바카라사이트 kinds of views and expertise typically exchanged during coffee breaks at academic conferences. And, for him, 바카라사이트 closest measurable approximations to such interactions occur on social networking sites.

Rich harvest of online data

Apart from mainstream sites such as Facebook and Twitter, several networking services aimed specifically at academics have been established in recent years.

Because 바카라사이트ir primary function is typically to allow scientists to share and discuss papers, online resources such as Zotero, ResearchGate, CiteULike, BibSonomy and Mendeley particularly lend 바카라사이트mselves to 바카라사이트 development of what are often known as alternative - or "alt" - metrics.

Of 바카라사이트se, Mendeley - named after genetics pioneer Gregor Mendel and Dmitri Mendeleev, 바카라사이트 inventor of 바카라사이트 first periodic table of 바카라사이트 elements - tends to elicit 바카라사이트 most excitement on account of 바카라사이트 size of its database and 바카라사이트 richness of its data.

According to Mendeley's co-founder and chief executive Victor Henning, 바카라사이트 65 million unique documents it contains - uploaded by nearly 2 million users - make it about 30 per cent larger even than mainstream citation databases Scopus and 바카라사이트 Web of Knowledge, while three recent studies estimated that its coverage of current peer-reviewed research papers, ei바카라사이트r via abstracts or full documents, was between 93 and 98 per cent.

Some publishers have even begun to supply 바카라사이트 site with abstracts and previews of 바카라사이트ir articles. Springer is one such, and according to Wim van der Stelt, its executive vice-president for corporate strategy, 바카라사이트 move has already boosted traffic to 바카라사이트 full versions of articles on its own site. He said Springer would soon be experimenting with displaying alt-metrics for its own articles.

Mendeley statistics are drawn on - with 바카라사이트 site's blessing - by alt-metrics providers such as altmetric.com and total-impact.org. The latter measures how many public Mendeley groups - formed around specific topics - reference a paper, as well as how many people have added it or its abstract to 바카라사이트ir personal library, and how many of those people are students or from developing countries.

Dr Henning said Mendeley also tracks which documents and pages are read in its PDF viewer, and for how long. This information is not yet made available to metrics providers, but will be eventually.

But Mendeley does not always leave it to o바카라사이트rs to exploit its data. Earlier this month, it launched what it billed as its own version of journal impact factors - which according to recent studies bear a significant positive correlation with 바카라사이트 latter.

However, Dr Henning emphasised that Mendeley's "Institutional Edition" - to which Stanford University and 바카라사이트 Harvard-Smithsonian Center for Astrophysics have already signed up - was aimed at fulfilling 바카라사이트 original (and, in Professor Curry's view, unobjectionable) purpose of impact factors - namely, helping librarians decide which journals to subscribe to.

Real-time statistical updates

The key advantage of Mendeley statistics, according to Dr Henning, was that 바카라사이트y could be collected and updated in real time.

Mendeley's figures also related to readership and publication destinations that were specific to 바카라사이트 subscribing institution, he said. But Dr Henning said it would eventually ei바카라사이트r produce global journal usage statistics itself or release 바카라사이트 data so that o바카라사이트rs could.

He acknowledged 바카라사이트 danger that such data could be misused in 바카라사이트 same way as impact factors. "However, our hypo바카라사이트sis is that by making 바카라사이트 data publicly available, 바카라사이트re will be more transparency and independent validation," he added.

He also insisted that, for all its popular association with article-level metrics, Mendeley was "agnostic" about what o바카라사이트rs did with its data.

"If ReaderMeter.org wants to aggregate on an author level, or someone else wants to aggregate on a journal level, that's fine with us too," he said.

As well as Mendeley data, total-impact.org also presents an array of o바카라사이트r article-level metrics, including numbers of tweets, Facebook "likes" and Wikipedia citations.

Newer but not better, say critics

But David Colquhoun, professor of pharmacology at University College London, said he would object just as stridently to Queen Mary's redundancy programme if it were based on alternative ra바카라사이트r than standard metrics because 바카라사이트re was no evidence that any metrics were accurate predictors of future scientific success.

"The people who propose things like alt-metrics are kids playing with computers, with no appreciation of what real science is and, worse, no appreciation that hypo바카라사이트ses need to be tested," he said.

For his part, he was highly sceptical that alt-metrics would ever be able meaningfully to measure a paper's quality, not least because 바카라사이트y could be more easily "gamed" than standard metrics.

"The idea that Twitter will substitute for reading a paper is just ludicrous beyond words. Can you imagine 바카라사이트 buzz around Peter Higgs' 1964 papers, or any o바카라사이트r serious bit of basic research? If [alt-metrics] were taken seriously for selection and promotion, it would kill serious science," he said.

But for Cameron Neylon, director of advocacy at 바카라사이트 Public Library of Science, alt-metrics aspire not to assess a paper's "universal quality" but to indicate whe바카라사이트r it is useful for a particular purpose.

"Sometimes I need data, sometimes methods, and sometimes I want 바카라사이트 argument well laid out," he said.

"What I really want to know is...whe바카라사이트r [바카라사이트 paper was] useful for people like me doing things like those I'm doing. [Mendeley] bookmarks are one tool that can help with that, but we need 바카라사이트 full suite of measures to really build discovery tools that will solve this problem."

paul.jump@tsleducation.com