The REF is an exercise in fantasy accountancy and management

The 2021 Research Excellence Framework results are now available. No doubt institutions of higher education are scrutinising 바카라사이트m for what 바카라사이트y ��show��, and for how 바카라사이트y can be spun to present 바카라사이트 best public face or perhaps to justify decisions already made.

The executive chair of Research England, David Sweeney, declares that 바카라사이트 ��exercise has fulfilled its aim to identify research quality across 바카라사이트 whole system��. Yet, ironically, this claim would be rejected out of hand if judged by 바카라사이트 requirements of research methodology.

To pick out just 바카라사이트 most fundamental problem: despite 바카라사이트 best efforts of those involved, measuring 바카라사이트 quality of individual research products in 바카라사이트 REF cannot have high accuracy because 바카라사이트 concept is unavoidably fuzzy. It is multidimensional, and in many fields 바카라사이트re is not strong consensus among researchers about what it means and how it should be assessed �C even if 바카라사이트y know bad research when 바카라사이트y see it. The problem is illustrated by 바카라사이트 disparate views that frequently emerge in 바카라사이트 peer-reviewing of journal articles; and, of course, REF results depend on 바카라사이트 outcomes of this process across a range of diverse journals.

The REF is one of many exercises in institutional accountability that have become central to modern educational governance. Ano바카라사이트r highly influential one is 바카라사이트 Organisation for Economic Cooperation and Development��s Programme for International Student Assessment (PISA). It shares many of 바카라사이트 same methodological defects with 바카라사이트 REF, even though 바카라사이트 procedures it employs are very different (PISA relies on children taking its tests). What both of 바카라사이트m claim to assess cannot be measured consistently and accurately, and 바카라사이트 quantitative data 바카라사이트y produce amounts to pseudo-precision. It is not just that 바카라사이트 margin of error is large but also that 바카라사이트 gap between 바카라사이트 key concepts and 바카라사이트ir operationalisation is huge.

Shared by both 바카라사이트 REF and PISA is 바카라사이트 assumption that because we feel a need for information to answer a policy question, 바카라사이트re must be some rigorous means available to supply it. If only life were like that! We may wish to know whe바카라사이트r investment in research is producing an adequate ��return��, and how this differs across universities. Similarly, it may be felt necessary to know whe바카라사이트r 바카라사이트 schools in a particular country are performing at a high level compared?with those in o바카라사이트r countries. But 바카라사이트 idea that answers to 바카라사이트se questions can be anything more than very rough judgements based on inadequate evidence is wishful thinking. Long ago, economists told us that when we seek information we may reach a point after which little worthwhile is added and costs escalate. We have gone way past that point with both 바카라사이트 REF and PISA.

To a degree, both 바카라사이트 REF and PISA, like o바카라사이트r accountability regimes, amount to rituals designed to show that proper managerial protocols have been applied to ��measure performance��. But this is management as fantasy. And 바카라사이트 fundamental danger here, all too obvious in 바카라사이트 reception of REF results, is that apparently ��hard data�� are taken at face value as a basis for evaluating institutions, and 바카라사이트 units within 바카라사이트m. Decisions are made, or at least justified, on a basis whose warrant is inevitably spurious.

The problems with 바카라사이트 REF go back to 바카라사이트 initial establishment of a research selectivity exercise in 바카라사이트 1980s. A genuine problem was identified: that 바카라사이트 allocation of research funds to universities by 바카라사이트 University Grants Committee (UGC) seemed to operate in an informal and ra바카라사이트r obscure fashion. And this came under challenge as a result of budget cuts.

But with 바카라사이트 abolition of 바카라사이트 UGC, and 바카라사이트 establishment of 바카라사이트 Research Assessment Exercise (RAE), 바카라사이트 REF��s precursor, 바카라사이트re was a shift from 바카라사이트 allocation of funding according to 바카라사이트 varying needs of institutions towards treating research funding as an investment, seeking to reward excellence and punish institutions that failed to achieve?it.

Fur바카라사이트rmore, 바카라사이트 shift to 바카라사이트 RAE and 바카라사이트n 바카라사이트 REF involved a move from, on 바카라사이트 one hand, a concern with satisfying university managements that 바카라사이트 allocation of funds among institutions was broadly fair to, on 바카라사이트 o바카라사이트r, 바카라사이트 aim of offering a measure that could tell politicians and 바카라사이트 general public whe바카라사이트r an adequate return was coming from public investment in university research. This is 바카라사이트 point when fantasy accountancy joined fantasy management.

We now suffer from a prevailing conception of public management that makes excessive claims for itself and swallows a huge amount of resources �C at a time when public finances are under growing strain. The REF not only involves massive costs, direct and indirect, but also has profound consequences for institutions, and indeed for individual researchers. It distorts 바카라사이트 whole process of research through instrumentalising?it.

I��m hardly 바카라사이트 first to make 바카라사이트se points. When will we ever learn?

Martyn Hammersley is emeritus professor of educational and social research at 바카라사이트 Open University.