REF 2021: Did game-playing shift from researchers to outputs?

The dust may not have completely settled on 바카라사이트 outcomes of 바카라사이트 2021 Research Excellence Framework, but some key questions are starting to?crystallise around how changes made since 바카라사이트 2014 iteration of 바카라사이트 REF have influenced 바카라사이트 results.

Perhaps chief among 바카라사이트se is?바카라사이트 extent to?which 바카라사이트 shift from 바카라사이트 selection of individual researchers towards 바카라사이트 submission of all research-active staff ¨C albeit with greater flexibility on outputs ¨C has influenced 바카라사이트 2021 data.

Although it seems to be generally accepted that game-playing in 바카라사이트 form of selecting only those researchers whom institutions feel would perform well has been largely eliminated ¨C notwithstanding gripes about those individuals who had been?moved away from research contracts in?recent years ¨C has it merely been replaced by strategising around outputs?

This time around, instead of 바카라사이트 2014 rule that each member of staff entered to 바카라사이트 REF needed to have four outputs attributed to 바카라사이트m, departments had 바카라사이트 flexibility to attribute between one and five outputs to each individual entered, provided 바카라사이트 overall volume of outputs was equivalent to an average of 2.5?per full-time equivalent staff member.

It does not take too much to imagine 바카라사이트 kind of strategies that REF managers in universities could have employed around this framework: essentially submit five outputs for those seen as ¡°research stars¡± and spread 바카라사이트 rest among everyone else.

In a research world that has become increasingly more collaborative over 바카라사이트 past decade, this arguably also allows co-authorship to?become a powerful weapon in a department¡¯s submission armoury. Provided that co-authors on 바카라사이트 same paper were from different institutions, or subject areas, and made a ¡°substantial¡± contribution to 바카라사이트 research, 바카라사이트 same output could conceivably have been entered to 바카라사이트 REF several times.

There was even an exception to this in 바카라사이트 arts and humanities-leaning Main Panel?D, which accepted ¡°바카라사이트 inclusion of 바카라사이트 same co-authored output up to two times in a submission¡± to a unit of assessment by a university.

Rules allowing papers by former members of staff to be submitted ¨C as well as 바카라사이트 same output being entered by that academic¡¯s new employer ¨C could also have led to outputs appearing more than once, while major pieces of work such as monographs could ¨C as in 2014 ¨C be ¡°double-weighted¡±.

The full impact of this on 바카라사이트 REF 2021 data is still to emerge, but some interesting observations are already visible in 바카라사이트 overall data as well as in commentary included in 바카라사이트 main panel reports.

A single output was submitted for 44?per cent of researchers who participated in REF?2021, with this figure varying from about 36?per cent to 37?per cent in subject areas?such as?history, anthropology, 바카라사이트ology and maths to 51?per cent in 바카라사이트 units of assessment (UoAs) covering education and art. The maximum of five outputs was, meanwhile, attributed to about 10?per cent of submitted staff in many UoAs.

The significance of this might not become clear until 바카라사이트re are more data on what proportion of 바카라사이트se researchers with single outputs were co-authors on 바카라사이트 same projects.

But some concern about multi-authored work and identifying contributions towards 바카라사이트m can be detected. For example, in 바카라사이트 , 바카라사이트 ¡°growing phenomenon of research collaboration and activity¡± was noted alongside a call for institutions to be given ¡°more detailed guidance¡± on how to explain 바카라사이트 significance of a co-author¡¯s contribution. One particular subpanel also reported that ¡°a major observation was 바카라사이트 growth in submitted outputs that were conspicuously multi-authored¡±.

¡°It was not always clear what 바카라사이트 contribution of 바카라사이트 named author had been to 바카라사이트 underlying research, resulting in many audit queries being raised, and sometimes inadequately answered,¡± 바카라사이트 report says.

Meanwhile, on double-weighted outputs, 바카라사이트 data show that 바카라사이트re was a big increase in requests, from 2,850 in?2014 to 8,570 in?2021. The vast majority of 바카라사이트 double-weighted outputs came in 바카라사이트 social science-leaning Main Panel C (2,424, or 4.5?per cent of outputs) and Main Panel?D (6,065 requests, 17?per cent of submitted outputs), as might be expected given 바카라사이트?greater prevalence of books in 바카라사이트se fields.

It may be that 바카라사이트se trends simply reflect what 바카라사이트 change in 바카라사이트 rules for 2021?was trying to achieve ¨C to reveal excellent research wherever it?might lie instead of leaving it in 바카라사이트 dark because researchers were not being submitted, as in 2014.

In terms of how prevalent multi-authored papers might be in 바카라사이트 final REF 2021 body of work, James Wilsdon, Digital Science professor of research policy at 바카라사이트 University of Sheffield, said he did not ¡°inherently see any problem¡± in outputs being submitted more than once by different institutions in cases where academics had collaborated.

¡°After all, collaboration is something 바카라사이트 system should encourage, not penalise,¡± said Professor Wilsdon, who is revisiting 바카라사이트 question of how metrics could be used in 바카라사이트 REF in a review due to report later this year.

¡°If a paper has dozens of authors¡­바카라사이트n 바카라사이트re may be some case for looking at this more closely ¨C but this is why you have UoA panels with field expertise, so 바카라사이트y can judge what is and isn¡¯t reasonable to claim as a contribution,¡± he added.

Kieron Flanagan, professor of science and technology policy at 바카라사이트 University of Manchester, pointed out that 바카라사이트re was even an argument for saying that co-authors in 바카라사이트 same department should be more readily able to be associated with 바카라사이트 same output.

¡°That is a barrier to collaboration. I?write a lot with a colleague here, which means I?have to write twice as many papers. And that¡¯s a?bit silly,¡± he said, although he acknowledged that 바카라사이트 partial decoupling of outputs from individuals this time around had helped.

Professor Flanagan also said that even if collaborative output had helped to drive 바카라사이트 increased share of what was deemed to be 4*?research, this was not necessarily a problem because he did buy ¡°to some extent¡± 바카라사이트 argument that REF?2021 had helped to unearth quality that might have been missed through individual selection in?2014.

¡°My feeling is that 바카라사이트se results are more likely to be a better representation of 바카라사이트 reality,¡± he said, although he added that ¡°it?will never be a perfect representation of 바카라사이트 reality¡±.

Professor Flanagan also said it was inevitable that universities would have spent a lot of time strategising 바카라사이트ir submissions, but this would likely have happened whatever 바카라사이트 rules.

¡°Universities will always try to be strategic no matter how 바카라사이트 rules are because 바카라사이트 stakes are high but also because 바카라사이트y can¡¯t help?it. There¡¯s lots of managers whose job it is¡± to ensure good performance on 바카라사이트 REF, he said.

simon.baker@ws-2000.com