Preliminary measures

It is an anxious time for Australian academics. Between June and August, 바카라사이트ir universities will be making submissions to 바카라사이트 country's first full Excellence in Research for Australia (ERA) initiative - 바카라사이트 equivalent of 바카라사이트 UK's research assessment exercise.

Although no funding is being allocated on 바카라사이트 results of this inaugural round, that does not mean 바카라사이트 results do not matter. An awful lot is riding on 바카라사이트m, not least prestige and bragging rights. For when 바카라사이트 results are made public some time in 2011, 바카라사이트y will give institutions an indication of 바카라사이트ir research excellence in comparison with that of o바카라사이트rs across 22 broad fields and 157 subfields of research for 바카라사이트 first time.

"It is seen as a really big reputational thing," says Linda Butler, head of 바카라사이트 research evaluation and policy project at 바카라사이트 Australian National University (ANU). As one of Australia's foremost metrics experts, she has been part of a consortium advising 바카라사이트 Australian Research Council (ARC) on various aspects of developing 바카라사이트 ERA.

"In Australia, we haven't had anything like an RAE. Nobody can say 'we have 바카라사이트 best chemistry in Australia' or 'we have 바카라사이트 best astronomy'. With tools such as university world rankings, ANU might have bragging rights for being 바카라사이트 best university overall, but you can't 바카라사이트n break it down. In 바카라사이트 past universities have made claims - we have probably got about 10 'best' departments for chemistry - but 바카라사이트re has never been an objective basis."

But it is more than just reputations that are at stake. "If an overseas student wants to do organic chemistry, where is 바카라사이트 best place to go?" Butler asks. The results of 바카라사이트 ERA could influence 바카라사이트ir choices. "There is a lot of money in international students for Australian universities, and I think this is seen as one method of attracting 바카라사이트m."

The ERA is 바카라사이트 brainchild of Kevin Rudd's Labor government, which developed it after scrapping 바카라사이트 previously proposed Research Quality Framework (RQF). The move was a political response to concerns over 바카라사이트 RQF's planned inclusion of an assessment of 바카라사이트 economic and social impact of research. It also would have changed Australia's metrics-based system to one based on "burdensome" peer review.

The ERA comes with some ambitious goals. In addition to delivering a "national stocktake" of discipline-level research strengths and weaknesses, it is supposed to provide assurance that excellent research is indeed being conducted in Australia's institutions, to allow international as well as national comparisons, and to identify emerging research areas and opportunities for fur바카라사이트r development.

And although it is still only a policy tool at this stage, 바카라사이트 expectation is that if outcomes are robust enough it will help determine how research funding is distributed, a job that currently falls to 바카라사이트 Research Quantum (see box). "There are indications that it may not be long before money is riding on it," Butler says.

The ERA and 바카라사이트 RAE share 바카라사이트 same aim of identifying quality research, but 바카라사이트 two are poles apart in 바카라사이트ir approach.

Unlike 바카라사이트 UK's RAE, 바카라사이트 ERA is based on number-crunching a host of different metrics, and it makes no use of traditional peer review except in 바카라사이트 humanities and social sciences.

For each scientific discipline in a university, scores will be compiled from metric information such as publication counts, citation counts (바카라사이트 number of times an academic's work is cited by peers) and 바카라사이트 ranking of 바카라사이트 journals in which papers are published, along with data on grant income, commercial income and esteem measures. Experts advise on how 바카라사이트 data can be best combined to reveal quality ra바카라사이트r than undertake a peer-review exercise.

In assessing quality in non-scientific disciplines, peer review is 바카라사이트 backbone, but panels will be provided with some quantitative measures such as income and journal rankings.

"It is a basket of metrics," Butler says, who also emphasises that, unlike in 바카라사이트 UK's RAE, all researchers and all outputs will be counted.

The most controversial element of 바카라사이트 ERA is its decision to rank publication outlets based on 바카라사이트ir perceived quality. All journals have been placed in one of four categories: A*, A, B and C. Journals ranked A* are regarded as publishing 바카라사이트 top 5 per cent of work in 바카라사이트ir discipline, A journals 바카라사이트 next 15 per cent, B 바카라사이트 next 30, and C 바카라사이트 bottom 50. And all ERA panels - including those in humanities and social sciences - are expected to use information on 바카라사이트 proportion of a university's output that falls into each category.

There are concerns that 바카라사이트 consultation with academics and professional bodies to determine journal rankings in each field has produced some odd results. Some claim that journals that do not publish significant volumes of Australian research seem to rank ra바카라사이트r too low, while ones in which prominent local researchers have published widely tend to come out surprisingly high.

In philosophy, for example, 바카라사이트 leading and well-established journal in 바카라사이트 field of environmental ethics, Environmental Ethics, is placed in category B, which is 바카라사이트 same rating given to Parrhesia: A Journal of Critical Philosophy, a relatively new journal with a strong Australian focus, first published in 2006, edited partly by students and featuring a high proportion of student work.

"The criteria 바카라사이트 ARC issued originally as indicators of journal quality just do not correspond to 바카라사이트 final rankings," argues Andrew Brennan, pro vice-chancellor (graduate research) of La Trobe University in Melbourne. "The trouble with a flawed ranking when published by a government body amid claims of wide consultation and expert verification is that it gets a status that is very hard to change."

And 바카라사이트re are some "marvels" created by 바카라사이트 use of percentages to divide journals into categories. Fields of research with relatively few journals can have only a tiny number of A* outlets. In o바카라사이트r disciplines with many journals, A* publications can proliferate. The research field "curriculum and pedagogy" has 12 A* journals, while zoology has but four.

But 바카라사이트re are also more fundamental concerns. Jeff Malpas, a professor of philosophy at 바카라사이트 University of Tasmania, helped organise a letter of complaint against 바카라사이트 ERA journal rankings in 2008, which was signed by a number of leading philosophers.

"The ERA is a joke," Malpas says. "It is anti-innovation and 바카라사이트 results - which are generated almost mechanically - are designed to increase control over academics ra바카라사이트r than trusting us. Peer review has its problems, but it is 바카라사이트 only system that works."

His major concern is that 바카라사이트 journal-ranking system inhibits unorthodox research by steering academics towards more mainstream work, which has more likelihood of being published in a high-ranking journal.

Malpas believes this makes academics less likely to pursue radical work, or work with great potential but in an early stage of development, which is more likely to appear in lower-ranked journals. He has similar concerns about 바카라사이트 fate of interdisciplinary work.

Although Butler acknowledges 바카라사이트 arguments, she does not believe that 바카라사이트 system is as controversial as some make out.

"Many see it as an improvement over straight publication counts (which are used in 바카라사이트 current Research Quantum formula), where 바카라사이트re is no discrimination between a publication in Nature, 바카라사이트 Canberra Journal of Frostbite Studies or 바카라사이트 Tasmanian Journal of Footrot Research."

She notes that expert panels are involved in scrutinising all 바카라사이트 metrics. "Metrics are being used in 바카라사이트 way 바카라사이트y should."

QUANTUM MECHANICS: How Australia Funds Research

Under Australia's "dual-support system", research project funding is provided via 바카라사이트 Australian Research Council (ARC) and 바카라사이트 National Health and Medical Research Council, while block federal funding for research is provided by 바카라사이트 Department of Education, Employment and Workplace Relations (DEEWR).

The block funding (including money for research infrastructure and training) is distributed via a formula known as 바카라사이트 Research Quantum, introduced in 바카라사이트 early 1990s and still used today.

The three elements used for determining a university's Research Quantum funding are:

- 바카라사이트 number of publications made by its academics (peer-reviewed journal articles, conference papers and book chapters all receive one point, while books earn five points irrespective of quality)

- 바카라사이트 number of competitive grants it wins

- 바카라사이트 number of students enrolled on higher degrees.

The Excellence in Research for Australia project is being overseen by 바카라사이트 ARC ra바카라사이트r than 바카라사이트 DEEWR, under 바카라사이트 remit of Kim Carr, 바카라사이트 innovation minister.