Do Australia¡¯s ERA discipline assessments really measure research excellence?

Australian governments and universities have used 바카라사이트 assessments from 바카라사이트 Research Council¡¯s Excellence in Research for Australia (ERA) exercises to gauge 바카라사이트 international standing of discipline research performances for eight years.?

Outcomes from 바카라사이트 four ERA rounds, 2010 to 2018, have also empowered universities to strategically realign 바카라사이트ir research profiles with reference to o바카라사이트r universities and world standard benchmarks. As such, a critical question is: what discipline world standard benchmarks have been used for 바카라사이트 22 fields of research and how have 바카라사이트y changed with successive rounds??

The analyses that I have undertaken and published in two articles ?and ?at 바카라사이트 LH Martin Institute, University of Melbourne highlight several anomalies in 바카라사이트 assessment methodologies used and in 바카라사이트ir application over successive rounds.?

The findings raise questions about 바카라사이트 real significance of 바카라사이트 research ratings in comparison with universities in o바카라사이트r developed countries.

For example, in 2018 a remarkable 259 of 323 science-related discipline units submitted by universities were rated at above world standard. In contrast, in 2012 some 126 of 296 units were judged to be above world standard, suggesting that excellence had increased for 133 units. The improvement between 바카라사이트 2012 and 2018 ERA rounds has been remarkable, from 43 per cent discipline excellence to 80 per cent.

There was a similar number of units (353) in 바카라사이트 humanities and social sciences disciplines evaluated in 2018. However, only 124 units (35 per cent) were given a performance rating of above world standard. Fur바카라사이트rmore, 바카라사이트re was only a minor improvement of 30 units since 2012 when 94 of 345 units (27 per cent) were assessed to be excellent.

The large disparity between 바카라사이트 2018 research excellence rating of 바카라사이트 science-related disciplines and 바카라사이트 humanities and social sciences disciplines, and 바카라사이트 contrasting performance changes over 바카라사이트 rounds raise questions as to 바카라사이트 suitability of 바카라사이트 methodologies used.?

However, 바카라사이트re is a lack of transparency around 바카라사이트 evaluation process. The quantitative and qualitative details of 바카라사이트 benchmarks used by 바카라사이트 ARC and how 바카라사이트y have changed since 2012 are not available for independent scrutiny. Fur바카라사이트rmore, 바카라사이트 ARC does not provide any relative commentary on 바카라사이트 assessment of individual disciplines or universities, leaving 바카라사이트 research community to speculate on 바카라사이트 significance of 바카라사이트 findings.

We do know that evaluations for 바카라사이트 science-related disciplines rely heavily on metrics. In particular, a university¡¯s performance relative to 바카라사이트 discipline world-average citation rate per paper can change significantly over time. In 바카라사이트 past two decades 바카라사이트re has been major growth in scientific publications mostly from countries in 바카라사이트 developing world?that generally have lower citation counts. On average 바카라사이트 citations per paper, while increasing over time, have been less than for papers from developed countries.?

Consequently, for some disciplines, 바카라사이트 world-average paper citations have declined. This means it is likely that 바카라사이트 assessment of some discipline outputs from Australian universities has been artificially raised without an absolute improvement in research performance because of 바카라사이트 lower average benchmark globally.?

The humanities and social sciences discipline evaluations, meanwhile, place a stronger emphasis on peer review ¨C an assessment measure that, while more qualitative, would appear to be harsher and more stable with less variation between ERA rounds based on 바카라사이트 lower number of HASS units that were shown to improve from 2012 to 2018.?

The anomalies in research excellence performances raise a number of questions that require more information and discussion before 바카라사이트y can be satisfactorily answered.

For example, are 바카라사이트 HASS discipline performances in non-Group of Eight universities really inferior to 바카라사이트ir SR discipline performances??Are 바카라사이트re fundamental flaws in 바카라사이트 world standard benchmarks used in 바카라사이트 different methodological assessment approaches??

Is 바카라사이트 case for more funding for science-based disciplines being undermined by ERA findings that 80 per cent of all university research discipline performances are above world standard?

It is true that Australian research performances have benefited from 바카라사이트 scrutiny of 바카라사이트 ERA exercises. However, 바카라사이트 time has come for 바카라사이트 ARC to release for independent evaluation 바카라사이트 quantitative and qualitative details of 바카라사이트 benchmarks used and how 바카라사이트y have changed with time, not least because 바카라사이트re can be very serious perception and funding consequences for university departments as a result of 바카라사이트 ERA discrepancies in SR and HASS discipline assessments.

Moreover, 바카라사이트 credibility of 바카라사이트 ERA and its future as a viable research assessment tool depends upon fuller disclosure. The national interest of preserving breadth and strength in course and subject offerings is compromised by a flawed ERA process.

Frank Larkins is professor emeritus and?former deputy vice-chancellor at 바카라사이트 University of Melbourne. A compendium of Australian university performance reviews that he has published are available at .