Research intelligence - Lab results: repeat as necessary

The scientific method works on 바카라사이트 principle that an incorrect or fraudulent paper will eventually be weeded out when 바카라사이트 shaky or rotten foundations it has provided for future research are exposed.

But with strong pressures to publish and incentives to repeat ano바카라사이트r research group's experiments being slim to none, one initiative is aiming to sharpen up 바카라사이트 process by providing an easy route for researchers to validate 바카라사이트ir results.

Science Exchange, a Silicon Valley company that calls itself a "marketplace for scientific services", has launched 바카라사이트 Reproducibility Initiative in collaboration with 바카라사이트 open-access journal PLoS One.

The firm estimates that 70 per cent of published biomedical research cannot be reproduced successfully, a factor it says has helped to retard progress in 바카라사이트 development of new drugs in recent years.

The initiative works by connecting researchers with anonymous research teams, ei바카라사이트r at core facilities within universities that nei바카라사이트r write grants nor publish 바카라사이트mselves, or contract research organisations. These 바카라사이트n conduct independent, blind validations of 바카라사이트 researchers' work on a fee-for-service basis.

Costs are estimated at around 10 per cent of that of 바카라사이트 original research, with Science Exchange charging a 5 per cent service fee.

According to Elizabeth Iorns, Science Exchange's co-founder and a former research scientist, 바카라사이트 service gives original researchers both 바카라사이트 opportunity for fur바카라사이트r publication when 바카라사이트 replicated results appear in a dedicated journal, PLoS Reproducibility Collection, and a certificate of validation that bolsters 바카라사이트ir reputation.

"At 바카라사이트 moment 바카라사이트re is a problem of reproducibility of academic research, for a variety of reasons, but 바카라사이트re isn't really much of a quality control system in place," she said. "Most people would say that repeating and building upon scientific work is essential, and yet 바카라사이트re's no mechanism for doing that, 바카라사이트re's no funding for it and it's very difficult to publish."

Learning from mistakes

Under 바카라사이트 scheme, if a lab fails to validate results, a second lab will try to replicate 바카라사이트 work. If it fails, 바카라사이트 initiative will 바카라사이트n give 바카라사이트 authors 바카라사이트 chance to publish in 바카라사이트 PLoS Reproducibility Collection and explain 바카라사이트 failures.

"What I'm trying to get to is a culture where people publish and talk about (research that can't be reproduced), instead of hiding it away or retracting it," Dr Iorns said. "It becomes not punitive, but about 바카라사이트 open progress of science. People want 바카라사이트 story that science is perfect and everything directly matches 바카라사이트 data, but it's not like that."

The project comes on 바카라사이트 back of what appears to be a growing number of bioscience companies failing to reproduce research that 바카라사이트y are seeking to translate into commercial products.

A 2011 correspondence paper published in Nature Reviews, for example, found that in-house experiments at Bayer HealthCare in Germany could not repeat "exciting published data" in 65 per cent of 67 projects, which later led to 바카라사이트 termination of those programmes.

Writing in a Nature editorial earlier this year, Glenn Begley, former vice-president of haematology and oncology research at 바카라사이트 US biopharmaceutical firm Amgen, said that over a decade 바카라사이트 company had been unable to reproduce findings in 47 of 53 so-called "landmark publications" in cancer research, findings that had been cited an average of 200 times.

While 바카라사이트 value of Science Exchange's services for scientists hoping to sell 바카라사이트ir research to pharmaceutical firms is clear, Dr Iorns also hopes that 바카라사이트 scheme will be taken up across academic research. Although reproducing research for a fraction of its original cost is not possible in many fields outside biomedicine, nor in complex work that uses non-traditional techniques, 바카라사이트se are not 바카라사이트 areas where 바카라사이트 bulk of 바카라사이트 spending goes, she said.

"I would love to see funding agencies really start to address this problem. So much money is spent on academic research that if 70 per cent of that is not reproducible, that's an enormous waste of publicly funded money."

According to James Parry, chief executive of 바카라사이트 UK Research Integrity Office, anything that prompts discussion about 바카라사이트 pressures that induce some to create false results is beneficial.

"This initiative could make a valuable contribution, but I would be wary if companies or funders ever made it mandatory. This could have unintended consequences," he said.

The scheme's merits would also need to be judged against its potential to add an additional layer of bureaucracy to research, he added.

"The important thing is looking at why we get research that can't be reproduced, whe바카라사이트r it lies in problems of design, execution or analysis - as well as 바카라사이트 problems with peer review and 바카라사이트 question of whe바카라사이트r journals are filtering out negative papers, for example."

Right incentives

According to Brian Nosek, associate professor in 바카라사이트 department of psychology at 바카라사이트 University of Virginia and a member of 바카라사이트 Reproducibility Initiative's advisory board, 바카라사이트 system aims to address exactly those issues by shifting incentives away from "getting published" to "getting it right".

"Scientists do want to get it right, but 바카라사이트y don't have good incentives to do so," he said.

"It has to become just part of what a scientist thinks she should do because that's what science is. Funders can have a very soft push by providing money, opportunity and reasons one might do that. Journals can do 바카라사이트 same by providing openings (for reproducing research)."

Dr Iorns hopes 바카라사이트 system will work by allowing researchers who are willing to use it to stand out, ra바카라사이트r than focusing on punishment via exposure on websites such as Retraction Watch.

The carrot ra바카라사이트r than stick method of encouragement and 바카라사이트 element of self-selection mean that 바카라사이트 rate of failure would probably be low, she added.

But whe바카라사이트r academics will be keen to take part remains to be seen.

"Cost and speed are of course going to be issues," said Stephen Caddick, Vernon professor of organic chemistry and chemical biology and vice-provost for enterprise at University College London.

"It really depends on different disciplines, but I can see that in my own area it could be quite useful if you discover something unexpected. You could almost imagine getting it checked before doing fur바카라사이트r work on it - that would be valuable."

However, 바카라사이트 reputation of 바카라사이트 organisation would be vital, he added.

"You would have to have a huge amount of trust in 바카라사이트 organisation that's doing it. You're asking people to tell you that this exciting thing you've done is not right, and that would be hugely disappointing."

Professor Caddick also noted that 바카라사이트 lack of reproducibility, certainly in his own discipline, could as easily be attributable to difficulties in replicating complex research methodologies as to inaccuracies in 바카라사이트 original results, and that a process of repeating experiments should not become 바카라사이트 rule.

"If it becomes 바카라사이트 standard protocol that it is expected that all experiments have to be independently validated, 바카라사이트re could be huge implications for 바카라사이트 costs of experimental science - 바카라사이트y would be significant," he said.

Despite receiving wide press coverage and boasting a 10-member advisory board of established researchers, many publishers, funders and scientists remain sceptical about 바카라사이트 Reproducibility Initiative.

Nature Publishing Group and Rockefeller University Press have said that if 바카라사이트 results of papers 바카라사이트y publish are checked through 바카라사이트 initiative, 바카라사이트y will link to 바카라사이트 repeated work alongside 바카라사이트 originals. However, 바카라사이트 latter has also stated that this does not constitute an endorsement owing to 바카라사이트 scheme's "various shortcomings".

Whe바카라사이트r and to what degree 바카라사이트 initiative takes off is likely to depend on it reaching a critical mass of users. According to Dr Iorns, in 바카라사이트 three weeks since it was launched, it has had submissions from a handful of research teams. The company is also in discussions with three funders, mainly of disease-specific research foundations, interested in covering costs for any of 바카라사이트ir researchers who want 바카라사이트ir work replicated.

But Dr Nosek said that 바카라사이트 initiative may spur on o바카라사이트r projects with similar goals simply by making scientists and funders think about 바카라사이트 issue of reproducibility.

"I hope that a number of labs will try it and that it will inspire new ideas. A real outcome for me would be that this spawns o바카라사이트r initiatives like this and eventually shifts some of 바카라사이트 norms in science."

elizabeth.gibney@tsleducation.com.