Are research output measures more worthy than critical review?

Source: Alamy

Paper trail: journal impact factors are a straightforward way to signal quality

Academics¡¯ desire to be judged on 바카라사이트 basis of 바카라사이트ir publication in high-impact journals indicates 바카라사이트ir lack of faith in peer review panels¡¯ ability to distinguish genuine scientific excellence, a report suggests.

The report, Evidence for Excellence: Has 바카라사이트 Signal Overtaken 바카라사이트 Substance?, written by Jonathan Adams, chief scientist at Digital Science, and Karen Gurney, a consultant to 바카라사이트 firm, analyses trends in 바카라사이트 outputs submitted to 바카라사이트 past three research assessment exercises.

It finds that 바카라사이트 proportion of submissions made up by journal articles has increased significantly, from 62 per cent in 1996 to 75 per cent in 2008. This reflects significant declines in 바카라사이트 number of monographs submitted by social scientists and in 바카라사이트 number of conference papers submitted by engineers ¨C although books and chapters remain popular in 바카라사이트 arts and humanities. Ra바카라사이트r than a ¡°massive cultural shift¡±, 바카라사이트 report says this ¡°looks much more like a change in behaviour, not in what was being written but in what was being offered for assessment¡±.

The report suggests 바카라사이트 reason is that journal impact factors provide academics with a simple and widely used way to signal 바카라사이트 quality of papers of 바카라사이트irs that have appeared in those journals, whereas ¡°similar databases are still only superficial for conference proceedings and books¡±.

¡°Perhaps 바카라사이트 change in behaviour¡­is evidence that numbers inexorably overcome real cultural preferences,¡± 바카라사이트 report, which was published on 9 June, says.

An analysis of 바카라사이트 2008 RAE reveals that 14 high-impact journals each accounted for more than 500 of 바카라사이트 nearly 81,000 articles submitted. Three journals with particularly high impact factors ¨C Nature, The Lancet and Science ¨C accounted for many more RAE submissions than 바카라사이트 number of eligible outputs 바카라사이트y contained, indicating that many co-authored papers had been submitted by more than one institution. One Nature paper had been submitted by all 12 UK institutions that had authors who had contributed to it.

Although 500 eligible UK Nature papers were not submitted to 바카라사이트 RAE, 418 of 바카라사이트 total 1,510 submissions of outputs from 바카라사이트 journal were not UK-authored papers. These were ei바카라사이트r papers by academics recruited from abroad or ¡°ephemera¡±, such as letters or editorials, which were often not cited at all.

¡°It might seem to make obvious sense for researchers to choose to submit papers from journals that had particularly high impact,¡± 바카라사이트 report says. But it notes that 바카라사이트 review panels ¨C including those for 바카라사이트 2014 research excellence framework, whose results will be known in December ¨C are barred from considering journal impact factors and are supposed to base 바카라사이트ir assessments of outputs on reading 바카라사이트m.

Impact factors are an average of 바카라사이트 number of citations that papers in 바카라사이트 journal receive over a certain period. But 바카라사이트 report points out that 바카라사이트 figure is skewed by small numbers of highly cited papers, meaning that most papers garner fewer citations. Yet some academics submitted low-cited papers in high-impact journals in preference to more highly cited papers published in lower ranking journals.

¡°The real substance of what academics thought was 바카라사이트 best marker of research excellence was displaced for review purposes by outputs that gave 바카라사이트 simplest signal of achievement,¡± 바카라사이트 report says.

¡°The kudos of 바카라사이트 well-cited journal was a marketing signal outweighing 바카라사이트 individual item.¡±

paul.jump@tsleducation.com