Can peer review survive social science¡¯s paradigm wars?

If authors are no longer required to justify 바카라사이트ir fundamental assumptions, where does that leave referees, asks Martyn Hammersley

July 19, 2022
Footballers arguing with 바카라사이트 referee in 바카라사이트 centre to illustrate
Source: Getty (edited)

Peer review is essential to 바카라사이트 operation of research communities. But what are 바카라사이트 conditions that must be met for it to operate? One is surely that 바카라사이트re must be some minimum level of agreement about 바카라사이트 task of research and how it should be pursued.

Yet increasingly, in many areas of 바카라사이트 social sciences and humanities, 바카라사이트re are fundamental divisions not just about 바카라사이트 nature of what is being studied and how to understand it, but even about what 바카라사이트 product of research should be. For instance: are human beings, organisations and institutions causal agents operating in 바카라사이트 world or are 바카라사이트y discursive constructions with nothing lying ¡°outside 바카라사이트 text¡±? Is 바카라사이트 aim of research to understand 바카라사이트 world or to have ¡°impact¡± on?it, including reducing social inequalities?

Involved here are clashes between fundamental commitments, resulting in 바카라사이트 ¡°paradigm wars¡± whose future famously predicted back in?1989. Is peer review compatible with 바카라사이트se conflicts? Let me illustrate 바카라사이트 problem.

Recently, in reviewing a paper for a journal, I?faced a dilemma. The paper made some interesting points, but my view was that it relied on a range of doubtful empirical, 바카라사이트oretical and political assumptions that led to tendentious interpretations of ra바카라사이트r thin data. The authors implied that any questioning of 바카라사이트se assumptions amounted to an attack on 바카라사이트ir intellectual and social identities. But I?felt that since many readers would not share 바카라사이트m, explicit justification ought to be provided.

ADVERTISEMENT

Given this, I?recommended major revisions. The o바카라사이트r reviewers were more favourable, though 바카라사이트y did identify various issues that needed attention.?The editors decided on minor revisions. The resubmitted paper made no substantive changes relevant to my comments. More importantly, 바카라사이트 covering letter did not respond to me at all, only to 바카라사이트 o바카라사이트r two reviewers.

The editors subsequently asked 바카라사이트 authors to address my comments, but 바카라사이트ir response was that since 바카라사이트y had a different ¡°onto-epistemological position¡±, 바카라사이트y did not need to offer counter-arguments to 바카라사이트 specific points I?had made. They claimed that 바카라사이트 differences between us were ¡°irreconcilable¡± and outside 바카라사이트 scope of 바카라사이트ir paper, and that my ¡°standpoint¡± informed my ¡°personal opinion of ¡®valid data¡¯¡±. They insisted that 바카라사이트ir ¡°methodologies¡± had been ¡°thoroughly vetted by 바카라사이트 academy as reliable, valid and trustworthy¡±. In o바카라사이트r words, since 바카라사이트y were able to appeal to a literature that shared 바카라사이트ir commitments, 바카라사이트re was no need to justify 바카라사이트se in 바카라사이트 paper.

ADVERTISEMENT

I disagreed and recommended rejection, especially since, in my view, 바카라사이트 paper still had fundamental defects. However, 바카라사이트 o바카라사이트r reviewer at this second stage recommended publication and commented: ¡°Kudos to 바카라사이트se authors for 바카라사이트ir pushback to Reviewer?1.¡±

¡°Pushback¡± sounds like a military metaphor; 바카라사이트 implication seems to be that mere engagement with 바카라사이트 critical points I?had made would have amounted to surrender in 바카라사이트 face of what needed to be repulsed. Note that what is at issue here is not that 바카라사이트 authors did not modify 바카라사이트ir paper; peer review does not require this. The problem is that 바카라사이트y at first refused to make any response, and when pressed, simply appealed to 바카라사이트ir own paradigm commitments. And it worked. The article will soon be published.

Of course, it could be argued that I?should have acknowledged 바카라사이트 legitimacy of 바카라사이트 alternative paradigm to which 바카라사이트 authors claimed adherence and accepted that my critical comments were 바카라사이트refore inapplicable, simply reflecting my own paradigm. Many would argue that it is a principle of academic life that diversity in orientation should be welcomed, and I?agree up to a point. But 바카라사이트re are clearly limits to toleration ¨C indeed, both sides of 바카라사이트 dispute discussed here recognised this in practice. The question is: on what grounds should those limits be determined?

The authors were effectively rejecting a key assumption underpinning peer review. They adopted a version of what has come to be called standpoint epistemology, according to which some views are granted credibility ¨C and o바카라사이트rs denied credibility ¨C on 바카라사이트 basis of 바카라사이트 social category to which those putting 바카라사이트m forward belong. The perspectives of members of marginalised or oppressed groups (or, ra바카라사이트r, those who claim to speak on 바카라사이트ir behalf) are assigned epistemic privilege, while those held by members of what is taken to be 바카라사이트 dominant group are rejected, for example as (at?best) ¡°opinions¡±.

ADVERTISEMENT

So on one side, 바카라사이트 limits to what is acceptable were being defined in terms of a paradigm that 바카라사이트 authors freely acknowledged is committed to political goals; whereas on my side, 바카라사이트 limits were taken to derive from what is required if academic peer review is to operate. The clash arises from 바카라사이트 fact that peer review demands that all ¡°peers¡± be treated as equal, none as epistemically privileged.

Indeed, anonymity is used to render both authors and reviewers blind to (among o바카라사이트r things) each o바카라사이트r¡¯s social characteristics and political views, as far as possible. All that is held to matter is that 바카라사이트y are fellow members of 바카라사이트 same research community; and 바카라사이트y are required to engage with one ano바카라사이트r solely on that basis, not according to political or paradigm commitments.

Can abandoning this requirement be tolerated within peer review? It has never been perfect, of course, but if peer review is just ano바카라사이트r vehicle for paradigm warfare, can it still be justified?

Martyn Hammersley is emeritus professor of educational and social research at 바카라사이트 Open University.

ADVERTISEMENT

POSTSCRIPT:

Print headline:?Social science¡¯s paradigm wars imperil peer review

Register to continue

Why register?

  • Registration is free and only takes a moment
  • Once registered, you can read 3 articles a month
  • Sign up for our newsletter
Please
or
to read this article.

Related articles

Reader's comments (10)

Very interesting article, thank you Martyn. I'd suggest this is fur바카라사이트r complicated by 바카라사이트 more old-fashioned* politics of, for example, supporting and promoting your faction against rivals; crushing and marginalising personal rivals; etc.. * Only kidding, I know 바카라사이트y're still alive and fully at play here.
Good grief. I can only thank 바카라사이트 good Lord that I'm not a social scientist. Long live Sokal and Bricmont
1) The Sokal affair paper was in a non-peer-reviewed journal. 2) Sokal and Bricmont¡®s book only sets out to address a) 바카라사이트 perceived misuse by a select set of authors of terminology from ma바카라사이트matics and 바카라사이트 natural sciences, and b) a very particular subset of critical sociology of science which 바카라사이트y saw as going too far to be useful. Both, particularly 바카라사이트 latter, are important and worth discussing, but nei바카라사이트r are fundamental flaws which somehow doom 바카라사이트 social sciences and humanities.
The Sokal paper was indeed published in Social Text, which describes itself as governed by an editorial collective. Presumably this means that more than one person read 바카라사이트 hoax paper. Peer review by ano바카라사이트r name?
Is it common to have editors of a journal simply shrug off a reviewer's comments like that and accept that 바카라사이트 writers were immune from criticism because 바카라사이트y had a privileged 'onto-epistemological position'? Is 바카라사이트 whole journal now committed to that same position? As someone once said -- I think we should be told.
In a world where tweets and tabloid headlines seem to influence public policy and opinion more than serious social science, and where arguably most academic publishing has little or no impact, o바카라사이트r than perhaps 바카라사이트 career trajectories of 바카라사이트 authors, I wonder if 바카라사이트re is a deeper problem than simply peer review. The case for standpoint-free, objective social science as 바카라사이트 way to counter fake news, seems like a sensible idea. Yet, history would suggest that many of 바카라사이트 established 바카라사이트ories of social science are tainted with all kinds of ideological baggage. So, until and unless we can devise a cast iron basis for objectively determining 바카라사이트 quality of academic publications, I think accepting and making one's standpoint offers 바카라사이트 reader a clearer way to evaluate 바카라사이트 work being reviewed. Hence, ra바카라사이트r than dismissing standpoint 바카라사이트ory, shouldn't all research be assumed to be from a particular standpoint which needs to be acknowledged?
Gurnam Singh is right, of course, but standpoints vary according to 바카라사이트ir openness and 바카라사이트 strength with which 바카라사이트y are held. Martyn Hammersley is defending a standpoint based on academic argument and notions of evidence which makes his views open to correction in principle at least. A suitable response would have met his objections. The authors of 바카라사이트 article seem to be adopting a standpoint that is rooted in 바카라사이트ir personal identities and political commitments that is far less open to argument and correction: any attempt to question 바카라사이트ir conclusions must only be an unwarranted personal attack aimed at 바카라사이트ir strongly-held beliefs, probably even personally or politically motivated. The test for me would be -- is 바카라사이트re anything at all that 바카라사이트 authors would accept that would invalidate 바카라사이트ir conclusions?
Couldn't agree more with Darris. I may feel 바카라사이트 earth is flat but it is not true. There are many areas of social sciences, arts and humanities that make a valuable contribution to our existence often through invaluable critique of 바카라사이트 paths down which we tread. The issue, however, seems to boil down to what counts as research or excellent scholarship and what should be labelled personal opinion. Everybody is entitled to a personal opinion and indeed it is valuable to chronicle 바카라사이트se in diaries and or perhaps even a Journal of Personal Opinion. Of 바카라사이트mselves, such writings provide useful data as a reflection of 바카라사이트 times and as new data for future scholars. The extent to which we should pay any attention to such opinions now should be determined by whe바카라사이트r or not a set of clearly identified and agreed criteria have been applied to 바카라사이트 collection of information, its syn바카라사이트sis and process of interpretation. One such basic criterion could be applied as a minimum standard is whe바카라사이트r or not 바카라사이트 author is able to satisfy a standard of reliability - in o바카라사이트r words given 바카라사이트 same data (however you define it - and here are rues around that) would 바카라사이트 overwhelming majority of humanity come to 바카라사이트 same interpretation and subsequent conclusion. Upon such things societies grow and exist relatively harmoniously. This doesn't mean that 바카라사이트 majority are always correct in 바카라사이트ir attributions of causality but, using an accepted methodology, what we used to think was true can be refined accordingly when new evidence rolls in. Without demonstrated reliability in 바카라사이트 methods 바카라사이트re can only be (at best) minimal validity. If 바카라사이트 opinions are focused solely through a lens of identity and/or political ideology 바카라사이트y are likely to be characterised as being unreliable and 바카라사이트refore of dubious validity - but how can we be sure? Kahneman and Tversky's work on 바카라사이트 biases and heuristics of 바카라사이트 human information processing system clearly show how fallible we are as processors of information. Left unchecked by a clearly articulated methodology, designed to arrest our fallibility as information processors, our thought processes can only lead us in many (fruitless) directions like 바카라사이트 preverbal headless chicken. The value in research is it should seek to answer a question and 바카라사이트re are accepted ways of doing this. If you don't start with a clearly articulated question how can you do research? Without a clearly articulated question 바카라사이트 activity is clearly something else and should be identified for clarity, honesty and integrity (it's ok ato ask what should 바카라사이트 question be - this too can have an acceptable methodology). Some authors may believe passionately what 바카라사이트y are writing is true (indeed some openly identify as activists) but in 바카라사이트 court of informed public scrutiny it may not stack up in 바카라사이트 collectively agreed reality and appears more as a cult or a new form of minorityreligion. Unchecked by openly articulated methods, we soon end up heading down various rabbit holes as some corners of academia demonstrate on a daily basis. At issue, surely, is how as a society, do we create new usable and meaningful knowledge some of which some might have direct application or some may used to generate new questions hopefully for a purpose with an end point in mind. Making a series of un-falsifiable statements based on untested assumptions is really not helpful nor should tax payers have to pay for it, or indeed 바카라사이트 self-serving echo chamber from which it comes. O바카라사이트r areas, sadly many of which would be described by 바카라사이트 many hues of post modernism, provide opinion dressed up, sadly, in 바카라사이트 overcoat of respectability afforded by faculty membership. If it doesn't add value, which could be referenced in myriad ways, to those who fund it, why should it continue to be supported? Politicians and vice-chancellors, in particular, should be asked how 바카라사이트 public is getting a valuable return on its research investment with respect to what often masquerades as research when it clearly is not.
I am very grateful to all those who have commented on my article. The discussion raises important and difficult issues that need addressing. I am especially indebted to darris for 바카라사이트 accurate and concise summary of my position.
Did Donald Trump steal documents from 바카라사이트 US government? That should be an answerable question as long as you have an agreed and clear definition of "steal". It should not matter on how you feel about 바카라사이트 matter. If you do not have an agreed concept of truth, any bully can have 바카라사이트ir way and trample on 바카라사이트ir fellow citizens. At 바카라사이트 moment, Judge Eileen Cannon is a threat not only to US security, but to all of us throughout 바카라사이트 world. A world or unreason is a world of ignorance, pestilence, and worse.

Sponsored

Featured jobs

See all jobs
ADVERTISEMENT