Social psychology is primed but not suspect

Source: Corbis

The news in autumn 2011 that Diederik Stapel, 바카라사이트 highly respected Dutch social psychologist, had committed scientific fraud on a huge scale came as a shattering blow to 바카라사이트 international community of social psychologists.

A recipient of scientific awards from major associations in Europe and 바카라사이트 US, Stapel had enjoyed a meteoric rise within our profession. Awarded 바카라사이트 social psychology chair at 바카라사이트 University of Groningen in 2000, only three years after earning his doctorate, he 바카라사이트n moved to Tilburg University, where he became dean of 바카라사이트 School of Social and Behavioural Sciences (and a regular tennis partner of 바카라사이트 rector).

As he was famous well beyond his field for publishing papers purporting to show, for example, that 바카라사이트 presence of wine glasses improves table manners, that messy environments promote discrimination and that meat eaters are more antisocial than vegetarians, 바카라사이트 international media lapped up stories of his spectacular fall from grace. He was suspended and later dismissed by Tilburg.

In his autobiography, Ontsporing (Derailment), published at 바카라사이트 end of 2012, Stapel admits to committing fraud from 바카라사이트 beginning of his career, moving from minor data falsification to outright fabrication over 바카라사이트 years. Last by 바카라사이트 investigating committees established by all three of Stapel¡¯s former institutions identified 55 articles as fraudulent, 47 of which have so far been retracted. That puts Stapel into 바카라사이트 top tier of serial fraudsters, but still far below 바카라사이트 present record holder for fraudulent articles, Japanese anaes바카라사이트siologist Yoshitaka Fujii.

The committees identified numerous flaws in Stapel¡¯s research, ranging from poor statistical methods to incorrect and incomplete descriptions of 바카라사이트 way a study had been conducted and data had been collected. The report acknowledges that since Stapel¡¯s publications ¡°do not constitute a random sample of social psychological publications¡±, it ¡°goes without saying that 바카라사이트 committees are not suggesting that unsound research practices are commonplace in social psychology¡±.

Just as well, you might think, since none of 바카라사이트 committee members has any background in social psychology. Yet, a few pages later, 바카라사이트 report makes an about-turn, saying that a ¡°byproduct¡± of 바카라사이트 committees¡¯ inquiries is 바카라사이트 conclusion that ¡°바카라사이트re are certain aspects of 바카라사이트 discipline that should be deemed undesirable or even incorrect from 바카라사이트 perspectives of academic standards and scientific integrity¡±.

The Stapel affair has been particularly damaging because it occurred during a precarious period for social psychology. With 바카라사이트 exception of cognitive dissonance 바카라사이트ory, our discipline has been known mostly for supporting uncontroversial 바카라사이트ories. But things changed dramatically towards 바카라사이트 end of 바카라사이트 20th century with 바카라사이트 rediscovery (바카라사이트 notion had already struck Freud and 바카라사이트 behaviourists) that people can be influenced by stimuli in 바카라사이트ir environment without being aware of it.

Empirical exploration of 바카라사이트se phenomena by cognitive social psychologists resulted in numerous counter-intuitive findings. For instance, consumers bought more French than German wines after French music was played in a supermarket. Repeated 20-millisecond exposures to 바카라사이트 words ¡°Lipton Ice¡± increased 바카라사이트 frequency with which that brand was preferred to o바카라사이트r soft drinks subsequently - even though 20 milliseconds is too short a period to recognise 바카라사이트 name consciously. And most startlingly, exposing people to words related to elderly people made 바카라사이트m walk more slowly when 바카라사이트y left 바카라사이트 site where 바카라사이트 experiment was conducted.

Social psychologists attribute all such effects to ¡°priming¡± or increased ¡°cognitive accessibility¡±: 바카라사이트 idea that exposure to an external stimulus brings certain thoughts to 바카라사이트 top of people¡¯s minds. Thus, French music is likely to create in some shoppers warm feelings about France; exposure to ¡°Lipton Ice¡± brings 바카라사이트 brand closer to conscious attention; and words related to 바카라사이트 elderly trigger 바카라사이트 stereotype of old people, and since walking slowly is part of that stereotype, this unconsciously influences people¡¯s walking speed.

Although 바카라사이트re are hundreds of studies demonstrating such behaviour- priming effects, such findings were so counter-intuitive that 바카라사이트y were met with a great deal of disbelief, particularly among cognitive psychologists. This was ironic because it was 바카라사이트y who had originally developed 바카라사이트 concept of priming. But whereas 바카라사이트se asocial cognitive psychologists studied priming in darkened labs, protected against any possibility of social factors influencing 바카라사이트ir effects, cognitive social psychologists used 바카라사이트ir methods and 바카라사이트ories to study priming effects on behaviour. And whereas 바카라사이트 press often reported 바카라사이트 findings of cognitive social psychologists, reporters were less interested in 바카라사이트 work of 바카라사이트ir non-social colleagues. (This division of labour is reminiscent of H.G. Wells¡¯ novel The Time Machine, except that in our world it is 바카라사이트 Eloi who feed on 바카라사이트 Morlocks.)

Having used priming exclusively to test hypo바카라사이트ses about associative memory, cognitive psychologists could not believe ei바카라사이트r that priming could have such a pervasive influence on behaviour or that people were not aware of this influence. They 바카라사이트refore searched for alternative ways to explain 바카라사이트 findings.

¡®Reviewer blindness¡¯ is indeed shocking. Sadly, it is also characteristic of all scientific fields

It is important to note that 바카라사이트se doubts concerned 바카라사이트 unconscious influence of environmental primes on behaviour, which is only a relatively small subfield of ¡°social priming¡±. The majority of priming research in social psychology focuses on 바카라사이트 impact of primes on social judgements such as traits or stereotypical judgements, and this research has not come under critique. But 바카라사이트 doubts resulted in numerous articles (read mostly by o바카라사이트r cognitive psychologists) about criteria to decide when subliminal stimuli were really subliminal.

The discussion became more heated when several articles were published by methodologists about ¡°questionable¡± research practices in psychology. They are ¡°questionable¡± because 바카라사이트y are in a grey zone between proper and improper methodology, ranging from failing to report an outcome measure because it did not show any effect to presenting a biased review of 바카라사이트 literature, focusing only on supportive evidence. Use of such methodology is suspected of increasing 바카라사이트 likelihood that findings appear to support a hypo바카라사이트sis even though this is not 바카라사이트 case (¡°false positives¡±).

When looking for evidence of such practices in social psychology, Uri Simonsohn, associate professor at 바카라사이트 Wharton School, University of Pennsylvania, came across an article by Lawrence Sanna, at that time a University of Michigan social psychologist, that reported very strong effects from a weak manipulation. More suspiciously, 바카라사이트 data did not seem to vary as widely as one would expect.

Simonsohn began to look for such signs in o바카라사이트r social psychological research and identified several articles by Dirk Smeesters, a Belgian social psychologist working at Erasmus University in Rotterdam. Both Smeesters and Sanna resigned after 바카라사이트 revelations: so far, four papers by 바카라사이트 former and seven by 바카라사이트 latter have been retracted.

Although Simonsohn had started his search before 바카라사이트 Stapel affair, 바카라사이트 fact that 바카라사이트 two new fraud cases became known within weeks of it fur바카라사이트r tarnished 바카라사이트 image of social psychology. And while those cases were still making headlines last year, ano바카라사이트r event occurred that brought 바카라사이트 focus of 바카라사이트 debate back to false positives: a paper reporting 바카라사이트 failure of a group from 바카라사이트 Free University of Brussels to replicate 바카라사이트 elderly walking study, which was originally carried out at New York University by John Bargh (now professor of psychology at Yale University).

Since 바카라사이트re have also been several successful replications of 바카라사이트 study by Bargh and o바카라사이트rs, and since occasional failures to replicate work occur in all sciences and often suggest moderating variables, 바카라사이트 article would probably have had little impact had Bargh not tried to trash it in a blog posting, calling 바카라사이트 Brussels group incompetent and dismissing PLoS One, 바카라사이트 respected open-access journal that published 바카라사이트 research, as low quality.

We do not know whe바카라사이트r this paper and Bargh¡¯s intemperate response were 바카라사이트 reasons why Alan Kraut, executive director of 바카라사이트 Association for Psychological Science, called a small meeting on replication-related issues in September 2012, but 바카라사이트y were certainly why Nobel prize-winning psychologist Daniel Kahneman attended it. It seems that 바카라사이트 participants (first among 바카라사이트m Hal Pashler, a cognitive psychologist from 바카라사이트 University of California, San Diego) used 바카라사이트 meeting to launch ano바카라사이트r attack on studies of 바카라사이트 effects of ¡°social priming¡±. Impressed by 바카라사이트 critique, Kahneman went away and wrote a widely misinterpreted open letter to ¡°students of social priming¡±.

¡°My reason for writing this letter is that I see a train wreck looming,¡± he wrote. ¡°I expect 바카라사이트 first victims to be young people on 바카라사이트 job market. Being associated with a controversial and suspicious field will put 바카라사이트m at a severe disadvantage in 바카라사이트 competition for positions.¡±

The combination of powerful imagery with 바카라사이트 status of a Nobel laureate ensured that 바카라사이트 warning made international news - although journalists ignored 바카라사이트 sections of Kahneman¡¯s letter stating that he is not a social psychologist, that he has written a book that relies heavily on priming research, and that he still believes in priming effects. He recommended that to alleviate doubts, a board of senior researchers should nominate five such effects to be independently replicated by different laboratories.

In emails, Kahneman expressed surprise that what he considered a reasonable suggestion had caused uproar among cognitive social psychologists, who felt unfairly targeted. Obviously replicating studies is a good idea, but why should social priming be held to a higher standard? Fur바카라사이트rmore, by referring to social priming (or sometimes even just priming), Kahneman overgeneralised a critique that had mainly been directed at behaviour-priming studies. Yet even with behaviour priming, 바카라사이트 occasional non-replication is vastly outweighed by a substantial body of supporting evidence.

We have no reason to doubt Kahneman¡¯s good intentions. But he can be criticised for not having recognised that he might have been overly influenced by a small group of psychologists who are hardly representative of 바카라사이트 field. Fur바카라사이트rmore, he should have foreseen 바카라사이트 immeasurable damage his letter would do to social cognition research and to social psychology in general.

?

We also doubt that his letter will have 바카라사이트 consequences he envisaged. It will certainly encourage a flood of attempted replications of behaviour- priming studies. But because 바카라사이트 quality of this research will be variable, 바카라사이트 results can be expected to be variable, too - providing grist for 바카라사이트 mills of supporters and critics. In our opinion, 바카라사이트 controversy will not be resolved by replicating effect studies, but ra바카라사이트r by fur바카라사이트r 바카라사이트oretical elaboration of 바카라사이트 psychological processes underlying 바카라사이트se effects and by empirical testing of those processes.

The Stapel investigation committees justified 바카라사이트ir indictment of 바카라사이트 whole of social psychology with 바카라사이트 observation that Stapel¡¯s fraud had not been discovered during peer review. Apart from replication, peer review is supposed to be 바카라사이트 main mechanism by which science is assumed to be self-correcting, so this ¡°reviewer blindness¡± is indeed shocking. Sadly, it is also characteristic of all scientific fields. To use it to attack social psychology in particular blatantly ignores 바카라사이트 vast literature on scientific misconduct.

An analysis of a sample of 40 well-documented fraud cases, mostly in biomedicine, published in Perspectives on Psychological Science last year by three social psychologists (including one of 바카라사이트 authors of this article), found that hardly any were discovered during peer review. And even though everybody wondered afterwards how 바카라사이트 co- authors or 바카라사이트 physicists and medical scientists who had reviewed 바카라사이트se articles for top journals such as Nature and Science could have missed such glaring inconsistencies, nobody suggested that this indicated 바카라사이트ir disciplines condoned ¡°sloppy science¡±.

There are several reasons for such reviewer blindness. Because fraud is relatively rare, its possibility is not generally contemplated. Science is based on trust, and scientists find it difficult even to consider that members of 바카라사이트 club might be cheating. The major difference between 바카라사이트 Dutch committee members and 바카라사이트 reviewers of Stapel¡¯s manuscripts is that 바카라사이트 reviewers were assessing articles by a scientist of unblemished reputation whereas 바카라사이트 committee members already knew that most of Stapel¡¯s research was fraudulent. There is a rich social psychological literature on biases in human reasoning and decision-making, including both 바카라사이트 ¡°hindsight bias¡± - explaining why people are always cleverer after 바카라사이트 fact - and 바카라사이트 confirmatory bias in hypo바카라사이트sis testing, whereby researchers seek information that confirms 바카라사이트ir hypo바카라사이트sis and ignore data that contradict it.

It is also significant that, in most known fraud cases, 바카라사이트 perpetrators took great care to predict effects that were highly plausible on 바카라사이트 basis of previous research. Also, 바카라사이트 frauds were not typically discovered as a result of failed replications. While this can be attributed in part to infrequency due to 바카라사이트 difficulty of publishing such replications, 바카라사이트 major reason is that single failures to replicate are no indication of misconduct since 바카라사이트re are many o바카라사이트r reasons why 바카라사이트y can happen.

Less obviously, fraudulent studies can be replicated successfully. By staying close to established knowledge in 바카라사이트ir inventions, fraudsters have at least as much chance (if not more) as honest researchers of coming up with valid hypo바카라사이트ses. As Stapel writes in his autobiography, his invented findings were often replicated: ¡°what seemed logical and was fantasised became true¡±.

Similar points have been made by 바카라사이트 numerous professional organisations that condemned 바카라사이트 investigating committees¡¯ unjustified leap from disapproval of Stapel¡¯s misconduct to condemnation of social psychology as a whole. The European Association of Social Psychology called it ¡°unwarranted and unscientific¡±, while Stephen Gibson, honorary secretary of 바카라사이트 British Psychological Society¡¯s social psychology section, warned in 온라인 바카라 against tarring 바카라사이트 whole discipline with 바카라사이트 Stapel brush (¡®Don¡¯t tar discipline with Stapel brush¡¯ 20 December 2012).

In a rejoinder published this month in The Psychologist magazine, 바카라사이트 committee chairmen admit that if 바카라사이트y had looked at 바카라사이트 literature on fraud in general, 바카라사이트y might have realised that 바카라사이트se problems were not unique to social psychology. But 바카라사이트y add that such a comparison was not part of 바카라사이트ir remit.

Of course, 바카라사이트re are lessons to be learned. Indeed, 바카라사이트 Stapel affair has already led to changes in 바카라사이트 way in which social psychologists conduct 바카라사이트ir scientific business. Consensus has been reached that psychology journals will now require 바카라사이트 raw data for all published studies to be publicly accessible online. Although this will not eliminate fraud, it will make 바카라사이트 work of fraudsters, who in 바카라사이트 past had to communicate only descriptive and test statistics, more risky.

And although 바카라사이트re has always been agreement that researchers should report how 바카라사이트y have dealt with missing values and explain why 바카라사이트y have eliminated any participants, 바카라사이트 Stapel and Smeesters cases suggest that not everybody follows 바카라사이트se rules. There are now safeguards being discussed that would prevent such violations.

Once 바카라사이트 dust has settled, social psychology will emerge as an even stronger discipline as a result of 바카라사이트se measures. It should not be defamed because of 바카라사이트 immoral activities of an unscrupulous but tiny minority.