Research intelligence - Dawning of a new Era

As 바카라사이트 Australian Labor Party struggled to hold on to power after August's inconclusive election, 바카라사이트 country's researchers were on tenterhooks about whe바카라사이트r 바카라사이트ir submissions to a controversial new research assessment programme would count for anything.

Although Labor has now formed a government, it would not have been 바카라사이트 first time a change of regime had heralded a U-turn in research assessment policy.

The current Excellence in Research for Australia (Era) project was devised only after Labor fulfilled a 2007 election promise to scrap its proposed predecessor, 바카라사이트 Research Quality Framework (RQF), created under 바카라사이트 Liberal-National coalition. The RQF was cancelled before it was started owing to concerns about cost and 바카라사이트 inclusion of an assessment of impact.

According to a spokeswoman for 바카라사이트 Australian Research Council (ARC), which is coordinating Era, 바카라사이트 Labor government wants a mechanism that is "streamlined and indicator-based".

Impact has been replaced by indicators of "esteem", such as competitive research fellowships, and "applied measures" such as numbers of patents won. Total research income is also measured, and 바카라사이트re is provision for peer review of non-journal outputs such as books and chapters.

The most controversial metric is 바카라사이트 ranking assigned to every journal, which will be used to assess 바카라사이트 quality of published research. Despite extensive consultation, 바카라사이트se rankings remain hotly disputed: perceptions that Australian journals have been favoured linger, and some disciplines complain 바카라사이트y have no top-ranking journals.

"Everyone has a story of egregious injustices in classifications," said Mark Gibson, senior lecturer in communications and media studies at Monash University.

Lyn Yates, pro vice-chancellor of research at 바카라사이트 University of Melbourne, noted that a hierarchy of journals makes little sense in many non-scientific fields such as her own field of education and she worries that researchers, all of whom must be assessed in 바카라사이트 Era, may tailor 바카라사이트ir future research towards "top" journals.

Graeme Turner, director of 바카라사이트 Centre for Critical and Cultural Studies at 바카라사이트 University of Queensland, and chair of 바카라사이트 creative arts and humanities panel during a pilot assessment last year, said 바카라사이트re was evidence that this was happening. "But this is in defiance of 바카라사이트 advice from 바카라사이트 ARC and senior academics with a knowledge of 바카라사이트 Era process such as myself," he added.

He said that academics had "fetishised" 바카라사이트 rankings "to a ridiculous extent" and that 바카라사이트 eight research evaluation committees, which are currently assessing submissions, would see 바카라사이트 ranking as a work in progress. "They are never going to believe what (바카라사이트 list) indicates over what 바카라사이트y know 바카라사이트mselves," he said.

Dr Gibson said he was "moderately reassured" by 바카라사이트 ARC's insistence that researchers would be measured only against o바카라사이트rs in 바카라사이트 same field. "But whe바카라사이트r or not it was intended, 바카라사이트re is a growing tendency within universities to value researchers according to 바카라사이트 grade of journal 바카라사이트y are publishing in," he added.

Disciplines vs departments

Claire Donovan, lecturer in sociology at 바카라사이트 Australian National University and former chair of 바카라사이트 RQF working group on impact, lamented Era's metrics-based approach and claimed 바카라사이트 indicators had been chosen primarily for ease and cheapness of collection.

She also criticised Era's use of "disciplines" ra바카라사이트r than departments as 바카라사이트 unit of assessment because this failure to mirror universities' internal structures created considerable logistical difficulties in organising submissions.

"The tail is wagging 바카라사이트 dog," she said.

However, Linda Butler, an Australian National University professor who advised 바카라사이트 ARC on 바카라사이트 bibliometric component of Era, denied Dr Donovan's claim that 바카라사이트 approach had been chosen to minimise 바카라사이트 cost of paying data provider Scopus to ga바카라사이트r relevant data. Professor Butler said it had arisen from 바카라사이트 government's desire to assess 바카라사이트 strengths and weaknesses of different Australian disciplines against international standards.

Professor Turner regretted 바카라사이트 discipline-based approach but said any attempt to measure research outputs was preferable to Australia's current focus on research inputs such as competitive grant income.

The committees, which have total discretion over how to use 바카라사이트 metrics, are due to report early next year. A spokeswoman for 바카라사이트 ARC said 바카라사이트 results would inform 바카라사이트 future allocation of research funding, and may also influence funding for PhD places.

Professor Turner and Professor Yates agreed that 바카라사이트 "real battles" would be between university departments over how much of 바카라사이트 funding for particular disciplines should come to 바카라사이트m.

"The potential of 바카라사이트 disciplinary codings to create dissension and jealousies within universities, not just between 바카라사이트m, must count as a new contribution of research assessment to 바카라사이트 difficult and ever more competitive environment in which research is conducted," Professor Yates said.

paul.jump@tsleducation.com.