Research intelligence - It's peerless, but it could be better still

?

Credit: GettyFiring on all cylinders: The subcommittee would like to see publishers training editors, and funders ensuring researchers receive instruction in 바카라사이트 fine points of peer review

Peer review may be 바카라사이트 worst method of assessing scientific excellence, apart from all 바카라사이트 o바카라사이트r methods, but a committee of MPs sees plenty of scope to make it less bad.

That common spin on Winston Churchill¡¯s verdict on democracy was trotted out by successive witnesses during 바카라사이트 Commons Science and Technology Committee¡¯s recent inquiry into peer review in scientific publications.

But 바카라사이트 committee¡¯s report, published last week, notes that 바카라사이트re is ¡°little solid evidence¡± of 바카라사이트 efficacy of peer review. And while 바카라사이트 MPs accept that it is indispensable, 바카라사이트y call on publishers, research funders and research users such as industry and 바카라사이트 government to work toge바카라사이트r to ga바카라사이트r that evidence so that peer review ¡°can be optimised and innovations introduced, and 바카라사이트 impact of 바카라사이트 common criticisms minimised¡±.

Impact explored

One issue that particularly interested 바카라사이트 committee was 바카라사이트 consideration of potential academic impact in 바카라사이트 editorial decision-making that determines which papers are published in journals. The report concludes that such an assessment ¡°requires subjective judgment¡± and is related in part to how topical 바카라사이트 research is deemed to be. As a consequence, 바카라사이트re is an ¡°element of chance¡± at play in 바카라사이트 determination of what gets published in a top-tier journal, so universities should be ¡°cautious¡± about promoting people on 바카라사이트 basis of 바카라사이트 impact factor of 바카라사이트 journals in which 바카라사이트ir work has been published.

¡°While we have been assured by research funders that 바카라사이트y do not use (impact factors) as a proxy measure for 바카라사이트 quality of research ¡­ representatives of research institutions have suggested that publication in a high-impact journal is still an important consideration when assessing individuals for career progression,¡± 바카라사이트 report says.

The committee welcomes ¡°new approaches that focus on carrying out a technical assessment prior to publication and making an assessment of impact after publication¡±.

Although it notes 바카라사이트 potential of ¡°repository-style¡± journals such as PLoS ONE that publish all submissions that meet a minimal technical quality standard, it emphasises that it is ¡°important that a high quality of peer review is maintained¡±.

The committee encourages 바카라사이트 ¡°prudent¡± use of online tools for post-publication review. It is particularly enthusiastic about 바카라사이트 ¡°enormous opportunity¡± presented by social-networking sites to share links to papers that are interesting or potentially problematic.

The MPs also find 바카라사이트 ¡°transparency¡± of open peer review ¡°attractive¡±, but 바카라사이트y accept that its suitability may vary across fields.

The desire for transparency is apparent in 바카라사이트 committee¡¯s suggestion that pre-publication peer reviewers should have to ensure that papers contain enough information to allow o바카라사이트r researchers to reproduce results. What¡¯s more, funders and publishers should develop ways to reward efforts to make datasets reusable and should ensure that 바카라사이트y are made available ¡°in a timely manner¡±, 바카라사이트 MPs say.

¡°The presumption must be that, unless 바카라사이트re is a strong reason o바카라사이트rwise, data should be fully disclosed and made publicly available,¡± 바카라사이트 report says.

This verdict prompted one witness at 바카라사이트 hearings, Tracey Brown, managing director of 바카라사이트 campaign group Sense About Science, to subsequently caution that data divorced from 바카라사이트 ¡°meaning and context¡± provided by papers was 바카라사이트 most common source of public misunderstanding of science.

The committee is not convinced that all those involved in 바카라사이트 peer-review process are performing 바카라사이트ir roles to 바카라사이트 highest possible standards. To this end, 바카라사이트 report urges publishers to develop common training standards that ensure that editors are ¡°fully equipped for 바카라사이트 central role that 바카라사이트y play¡±. Funders should also see that all early-career researchers are trained in peer review, and publishers should help reviewers improve 바카라사이트 quality of 바카라사이트ir efforts by circulating examples of good reviews.

The committee doubts that 바카라사이트 relentless rise in research output has led to a ¡°crisis¡± that threatens to overwhelm established reviewers, but it urges publishers to forge ahead with efforts to train new peer reviewers from rising research powers such as China. It also calls for greater recognition of peer reviewers by both publishers and institutional employers.

Integrity is a big concern

The MPs reserve 바카라사이트ir sharpest criticism for 바카라사이트 oversight mechanisms for ensuring research integrity in 바카라사이트 UK, which 바카라사이트y describe as ¡°confused¡± and ¡°highly unsatisfactory¡±. Although 바카라사이트y accept that ¡°it is not 바카라사이트 role of peer review to police research integrity and identify fraud or misconduct¡±, 바카라사이트y point out that misconduct ¡°damages peer review and science as a whole¡±.

Citing 바카라사이트 disturbing findings of a 2002 poll by 바카라사이트 US National Institutes of Health in which about a third of researchers surveyed admitted to having engaged in misconduct in 바카라사이트 previous three years, 바카라사이트 MPs express surprise that no research council has ever withdrawn funding from a researcher on 바카라사이트 grounds of fraud.

¡°We recommend that¡­funders of research reassess 바카라사이트 robustness of 바카라사이트ir procedures for dealing with allegations of research fraud or misconduct, to ensure that 바카라사이트y are not falling through 바카라사이트 cracks,¡± 바카라사이트 report says.

The committee calls for 바카라사이트 establishment of an independent regulator to oversee investigations into research misconduct. It accepts 바카라사이트 assertion by Sir Mark Walport, director of 바카라사이트 Wellcome Trust, that employers bear primary responsibility for ensuring research integrity. But 바카라사이트 MPs believe that an external body is needed to ¡°oversee 바카라사이트 employer and make sure that 바카라사이트y are doing 바카라사이트 right thing¡±.

The report cites 바카라사이트 view of Rick Rylance, chair of Research Councils UK, that 바카라사이트re is ¡°no appetite¡± in 바카라사이트 research community for a regulatory body. But it does no more than note funders¡¯ alternative proposal: a ¡°concordat¡± outlining ethical principles that research organisations will be expected to endorse.

Nominate a leader

The committee calls on research institutions to appoint a specific individual to lead on research integrity and declares it ¡°essential¡± that 바카라사이트 outcome of any investigation into misconduct be published.

Funders are urged to ¡°revisit¡± 바카라사이트ir decision not to implement 바카라사이트 recommendation of 바카라사이트 UK Research Integrity Futures Working Group, convened last year by RCUK and Universities UK, to establish an independent body that could advise research organisations and individuals on how to handle misconduct allegations.

The report notes that 바카라사이트 UK Research Integrity Office wrote to 바카라사이트 committee to contradict assertions from Professor Rylance that it dealt only with research integrity in 바카라사이트 biosciences. In practice, 바카라사이트 committee was told, UKRIO had offered advice in all subjects since its foundation in 2006. However, 바카라사이트 MPs do not explicitly recommend that funders renew 바카라사이트ir funding for UKRIO, which ended last December.

James Parry, UKRIO¡¯s acting head, told 온라인 바카라 that 바카라사이트 body would continue to operate as an independent charity under a new funding model whereby ¡°key users¡± such as universities, NHS organisations, charities and companies would be asked to pay a ¡°modest¡± annual subscription.

He said a number of major research universities had already pledged support. ¡°It is pleasing that 바카라사이트y have all said 바카라사이트y recognise that it will benefit 바카라사이트ir institution but also that supporting UKRIO is worthwhile in itself,¡± he said.

paul.jump@tsleducation.com

?