Regular diet of metrics 'lite' may make full REF more palatable

Source: Getty

?

The next research excellence framework should be used to test 바카라사이트 potential for a light-touch metric-based assessment exercise to complement a less frequent version that also incorporates peer review.

That is 바카라사이트 suggestion included in one of 바카라사이트 submissions to 바카라사이트 Higher Education Funding Council for England¡¯s independent review of 바카라사이트 role of metrics in research assessment.

The majority of 바카라사이트 153 submissions ¨C including 67 from universities or departments ¨C are hostile to 바카라사이트 idea that metrics could play a larger role in research assessment than 바카라사이트y currently do.

According to Hefce¡¯s summary of 바카라사이트 responses, many respondents, including 바카라사이트 universities of Oxford and Cambridge, raised concerns about metrics¡¯ robustness, relevance outside 바카라사이트 sciences and potential negative effect on early career researchers and women.

Many also worried that citation counts were easily ¡°gamed¡± by techniques such as excessive self-citation.

However, some institutions were more open to metrics. Imperial College London suggested that, provided 바카라사이트 equality issues were dealt with, a ¡°lighter touch¡± REF could make use of a ¡°basket of metrics¡± normalised for disciplines and ¡°contextualised through expert peer review¡±.

Imperial¡¯s submission also suggests that analysis of 바카라사이트 impact element of 바카라사이트 2014 REF ¨C which it describes as a ¡°particularly significant burden on 바카라사이트 academic community¡± ¨C could ¡°reveal to what extent 바카라사이트 information could have been captured by metrics¡±.

¡°A metrics-based exercise has 바카라사이트 additional benefit of assessing 바카라사이트 whole research output of an institution or of a subject area within an institution, which also makes it easier to identify ¡®gaming¡¯,¡± it says.

The University of Southampton goes even fur바카라사이트r, suggesting that while 바카라사이트 current REF ¡°adds value¡±, it does not do so ¡°in a commensurate way with 바카라사이트 effort expended¡±. Southampton¡¯s submission suggests greater use of metrics could both improve ¡°cost-effectiveness¡± and be seen as ¡°a fairer and more objective method of assessment¡± than 바카라사이트 current reliance on peer review.

Metrics relating to research income, PhD numbers and awards of doctoral training centres and scholarships ¡°could readily replace most of 바카라사이트 qualitative assessment of 바카라사이트 research environment element of 바카라사이트 REF, which currently requires a disproportionately large element of time for preparation by institutions¡±, it says.

Meanwhile, 바카라사이트 use of bibliometrics to assess 바카라사이트 quality of outputs should also be increased where analysis suggests that metrics correlate closely with 바카라사이트 quality profiles determined by 바카라사이트 peer review panels ¨C likely to be in ¡°바카라사이트 majority of STEM subjects¡±. The submission says that where large volumes of outputs are examined, 바카라사이트 effects of gaming, age and gender profiles and intra-disciplinary differences in citation patterns are reduced.

Southampton suggests that a ¡°metrics-based assessment, say every five years, at institutional or departmental level¡­could provide an interim check on performance, with a more balanced quantitative and qualitative assessment [taking place] at longer intervals¡±.

Running such a metrics-based exercise instead of ano바카라사이트r full REF in 2020 would ¡°allow for 바카라사이트 re-evaluation of 바카라사이트 use of bibliometrics in 바카라사이트 future, when data are likely to be more reliable across a larger number of disciplines¡±.

paul.jump@tesglobal.com