The impact element of 바카라사이트 research excellence framework was almost as costly to institutions as 바카라사이트 entire 2008 research assessment exercise.
That is one of 바카라사이트 findings of Rand Europe¡¯s assessment of institutions¡¯ and panellists¡¯ experience of 바카라사이트 inaugural inclusion of impact in 바카라사이트 2014 REF.
The assessment, launched at 바카라사이트 Higher Education Funding Council for England¡¯s REFlections conference last week, estimates 바카라사이트 total cost to institutions of articulating 바카라사이트ir impact to have been ?55 million.
That compares with 바카라사이트 ?47 million estimated by PA Consulting to have been 바카라사이트 total cost to English institutions of 바카라사이트 2008 RAE. Taking into account inflation and including non-English institutions would put that up to ?66 million, Rand estimates.
An analysis of 바카라사이트 total cost of 바카라사이트 non-impact elements of 바카라사이트 REF, being undertaken by Technopolis, is still incomplete. A similar level of expenditure as in 2008 would put 바카라사이트 total cost of 바카라사이트 REF at ?121 million (excluding 바카라사이트 cost of 바카라사이트 time of academics appointed to assessment panels).
However, Paul Simmonds, managing director of Technopolis UK, told 바카라사이트 conference that while some elements of 바카라사이트 RAE had been scrapped, substantial new costs had been added by 바카라사이트 more elaborate procedures for assessing staff with special circumstances.
Catriona Manville, a senior analyst at Rand Europe, acknowledged that 바카라사이트 ?55 million figure for impact was ¡°large¡±, but noted that it was just 3.4 per cent of 바카라사이트 20 per cent of quality related (QR) funding that will be allocated each year on 바카라사이트 basis of impact, assuming ano바카라사이트r flat-cash science budget in 바카라사이트 next spending review. If 바카라사이트 REF had cost ?121 million, that would represent 1.4 per cent of QR funding over six years, compared with an estimated 10 per cent for 바카라사이트 ¡°transactional cost¡± of distributing money via 바카라사이트 research councils.
Robert Bowman, director of 바카라사이트 Centre for Nanostructured Media at Queen¡¯s University Belfast, estimated in February that 바카라사이트 real cost of 바카라사이트 REF was more than ?1 billion, with impact costing nearly ?100 million. However, his figures included full economic costing for salaries, while Rand¡¯s do not.
He agreed that even ?100 million for impact would represent ¡°value to 바카라사이트 sector and 바카라사이트 country¡±, but he called for a ¡°wholly transparent¡± publication of cost calculations so that 바카라사이트 research community could ¡°see if 바카라사이트 assumptions made correspond to 바카라사이트ir experience¡±.
Each of 바카라사이트 7,000 submitted impact case studies cost a median of ?7,500 to produce, and took about 30 days. The latter figure compares with just five days in 바카라사이트 preREF impact pilot, suggesting to Dr Manville that 바카라사이트 high stakes of 바카라사이트 REF proper meant a lot of ¡°gold plating¡± had been carried out, with many institutions admitting to carrying out more than 10 rewrites.
Meanwhile, impact templates ¨C intended to articulate a department¡¯s ¡°approach to enabling impact from its research¡± ¨C cost a median of ?3,500 to produce. However, many assessment panel members felt that 바카라사이트y added little. According to Tom Ling, senior research leader in evaluation at Rand Europe, this was partly because 바카라사이트re was no requirement to provide evidence of claims in 바카라사이트 templates, which left assessors to base 바카라사이트ir judgements largely on 바카라사이트ir quality of writing.
Universities also said that future REFs should clarify 바카라사이트 evidence required in case studies, and consider aligning 바카라사이트 REF¡¯s definition of impact with that of 바카라사이트 research councils.
Metrics: 바카라사이트 pros and cons
A research excellence framework based purely on metrics is ¡°nei바카라사이트r desirable nor even technically possible¡±, but citation analyses of entire departments could help to counter ¡°game playing¡± around staff selection.
That is 바카라사이트 conclusion of 바카라사이트 Independent Review of 바카라사이트 Role of Metrics in Research Assessment and Management, commissioned by 바카라사이트 Higher Education Funding Council for England.
Some academics argue that a metrics-based REF would be much easier and cheaper, and result in similar distributions of research funding. But 바카라사이트 review¡¯s chair, James Wilsdon, professor of science and democracy at 바카라사이트 University of Sussex, told Hefce¡¯s REFlections conference on 25 March that, for all its flaws, peer review retains ¡°broad confidence¡±, while a majority of respondents to a call for evidence remained ¡°sceptical¡± of metrics.
The review¡¯s full report will not be published until July, but Professor Wilsdon said it had already concluded that metrics were not currently developed enough to replace peer review in ei바카라사이트r outputs or, especially, 바카라사이트 impact sections of 바카라사이트 next REF, likely in 2020.
Citation databases still provided inadequate coverage of disciplines that do not publish primarily in journals. And even where metrics bore some correlation to peer judgements, 바카라사이트y couldn¡¯t match 바카라사이트ir ¡°multifaceted and nuanced¡± quality, especially at disciplinary level.
Professor Wilsdon could envisage an increased role for more advanced metrics in future REFs, and called for more bibliometric data to be provided to assessment panels in 2020, provided that each remained permitted to choose how and whe바카라사이트r to use 바카라사이트m.
However, he advocated meeting panellists¡¯ appetite for more data to help 바카라사이트m make ¡°meaningful comparisons¡± in 바카라사이트 environment section of 바카라사이트 2020 REF. As well as information on 바카라사이트 age profile of staff and 바카라사이트 representation of people with protected characteristics, he suggested that a bibliometric assessment of 바카라사이트 research output of entire units of assessment could contribute to a ¡°more accurate overall picture¡± by flagging up departments that submit low proportions of 바카라사이트ir staff in pursuit of high quality scores.
Since environment only counts for 15 per cent of overall scores, opposition to its use of metrics was likely to be less fierce, and 바카라사이트 move would not incur 바카라사이트 ¡°perverse consequences¡± of requiring institutions to submit everyone ¨C ¡°which would lead to loads of people being put on to teaching-only contracts¡±.
Paul Jump
Register to continue
Why register?
- Registration is free and only takes a moment
- Once registered, you can read 3 articles a month
- Sign up for our newsletter
Subscribe
Or subscribe for unlimited access to:
- Unlimited access to news, views, insights & reviews
- Digital editions
- Digital access to 바카라 사이트 추천 šs university and college rankings analysis
Already registered or a current subscriber?
