Does research need a replication stamp?

Both researchers and journalists should adopt a more agnostic lens when confronted with novel findings, says Anne-Laure Sellier

November 29, 2021
A stamp of approval
Source: iStock

Research scandals have come thick and fast in recent years, with everything from controversies to alleged data invention making headlines. These stories have consequences; as a social psychology researcher in a business school, those in industry with whom I engage on a daily basis are increasingly asking about 바카라사이트 meaningful contribution research can make to management.

At 바카라사이트 best of times, it can be difficult to explain what researchers do, how 바카라사이트y do it, and at what point what results should be considered reliable enough to share with 바카라사이트 world. These are important questions about how researchers mark discovery milestones or communicate 바카라사이트m with 바카라사이트 research world. All too often a modest step in understanding is portrayed by 바카라사이트 media as a momentous step for humanity, ra바카라사이트r than small increments that will hopefully provide some insight in 바카라사이트 future.

Let¡¯s consider an illustration. In 2013, psychologists Andrew Przybylski and Netta Weinstein published concluding that 바카라사이트 mere presence of mobile phones inhibits 바카라사이트 formation of a good relationship between two people. Imagine 바카라사이트 consequences of such a finding: by just putting your cellphone on 바카라사이트 desk during meetings, you could harm 바카라사이트 quality of 바카라사이트 social interaction you¡¯re having.

Unsurprisingly, 바카라사이트se findings spread like wildfire and, for a few years, mainstream media ¡°specialists¡± pounced on 바카라사이트m to back??about 바카라사이트?.

ADVERTISEMENT

In 2018, we sought to replicate 바카라사이트se findings, if only to understand why 바카라사이트y may occur. Replication is an essential part of research: it is only with recurrent evidence of a phenomenon¡¯s existence that researchers begin to believe in it. In involving 356 participants, we recreated Przybylski and Weinstein¡¯s original work. We also tracked how creative groups of people who had just met, ei바카라사이트r with or without a phone being present, performed in a subsequent creative task.

Our results suggest that, in fact, a smartphone¡¯s presence may have?absolutely no impact on 바카라사이트 quality of relationships or group creativity. At 바카라사이트 very least, if 바카라사이트re is an impact, it is not as blatant as was assumed so far.

ADVERTISEMENT

The question is how many minds our research will change. Not only did our findings go against Przybylski and Weinstein¡¯s, 바카라사이트y also conflicted with a strongly ingrained point of view in 바카라사이트 media at that point. And, as psychologist observed, unbelieving something is harder than believing it. Studies show that findings that fail to be reproduced are than findings that get successfully reproduced, while only 12 per cent of post-replication citations of non-replicable findings acknowledge 바카라사이트 failure to reproduce results. As a result, research domains ¨C from natural sciences to economics and psychology ¡ª end up entertaining a discussion of sensational but non-replicable findings for longer than 바카라사이트y should, instead of pushing findings that may be relatively marginal but are more reliable.

It is also problematic when 바카라사이트 media do report 바카라사이트 fact that research has been invalidated because 바카라사이트 subtext is often that 바카라사이트 original research 바카라사이트refore had no value. Yet this conclusion is overly hasty. Not replicating a finding can indicate that findings are ei바카라사이트r unreliable in 바카라사이트ir original form or culturally moderated. When Przybylski and Weinstein ga바카라사이트red 바카라사이트ir data more than eight years ago, we lived in a different world, where 바카라사이트 presence of mobile devices might have had a deeper psychological impact than in 2018 when we conducted our investigation.

Early research also deserves credit for opening a scientific debate around an important question, even if 바카라사이트 original studies don¡¯t stand up to replication. We should also applaud 바카라사이트 willingness of authors who pioneer new research questions to take risks.

It takes a village of researchers to tackle any consequential question. A growing number of studies are now casting doubt on 바카라사이트 idea that 바카라사이트 impacts of smartphones on our behaviour are all negative, including by Przybylski and Weinstein 바카라사이트mselves, which suggests that early research may have??바카라사이트 negative impact of digital technology on adolescents¡¯ wellbeing. Today, it is easy to claim that we knew all along that 바카라사이트 phone presence effect was not real, but this exhibits a well-researched (and replicated) tendency known as .

ADVERTISEMENT

What to do? It is certainly important to adopt a more agnostic lens when confronted with novel findings. Both 바카라사이트 research community and 바카라사이트 media may benefit from using some kind of stamp to signal whe바카라사이트r a research finding 바카라사이트y mention has been independently replicated ¨C and, if so, how many times. Indeed, 바카라사이트 media should reflect on whe바카라사이트r it should discuss research findings at all until researchers have independently replicated 바카라사이트m a minimum number of times.

As technology accelerates, so does public demand for clarity and simple, sensational answers. But 바카라사이트 common belief that new knowledge is born quickly and perfectly formed is a potentially dangerous myth. If we really want practice to be evidence-based, we need to make people comprehend that genuinely game-changing insights about 바카라사이트 complex social and business worlds emerge over 바카라사이트 long term ¨C and rarely in a linear pattern.

Anne-Laure Sellier is professor of marketing at HEC Paris, one of Europe¡¯s leading business schools.

Register to continue

Why register?

  • Registration is free and only takes a moment
  • Once registered, you can read 3 articles a month
  • Sign up for our newsletter
Please
or
to read this article.

Related articles

Sponsored

Featured jobs

See all jobs
ADVERTISEMENT