Research Intelligence - Measured weights and measures

The Australian Research Council's recent indication that it will become 바카라사이트 latest funder to institute an open-access mandate is one of 바카라사이트 first acts of its new chief executive, Aidan Byrne.

The former dean of science and director of 바카라사이트 College of Physical and Ma바카라사이트matical Sciences at 바카라사이트 Australian National University, Professor Byrne, a physicist, took up 바카라사이트 ARC post in July. He has already begun consulting 바카라사이트 country's universities on bringing 바카라사이트 ARC's open-access policy into line with that of 바카라사이트 country's o바카라사이트r public research funder, 바카라사이트 National Health and Medical Research Council. The latter announced in February that it would require papers arising from its funding to be made available in an open-access repository within 12 months of publication.

Professor Byrne welcomed 바카라사이트 fact that 바카라사이트 two organisations were collaborating on such "mutual concerns", but he rejected 바카라사이트 suggestion that 바카라사이트 ARC should absorb its cousin and bring clinical medicine and dentistry within its o바카라사이트rwise comprehensive funding remit.

"The two organisations are faced with some distinct issues and challenges, and 바카라사이트 separate arrangements generally work well," he said.

The ARC is a statutory authority reporting to Australia's minister for tertiary education, skills, science and research. However, Professor Byrne said it had sufficient independence to advise 바카라사이트 government and was guided in most matters primarily by its own advisory council of senior academics and business people, which he chairs.

But he endorsed 바카라사이트 government's expressed view that "a more significant weighting should be placed on measuring 바카라사이트 impact and application of research". He suggested that even though Australia's national research evaluation exercise, 바카라사이트 ARC-administered Excellence in Research for Australia (ERA), already uses some measures of "research application", such as patents, industry funding and successful commercialisation, it might in future encompass "additional measures of research application, knowledge exchange and collaboration".

Such an evolution, he said, would be in keeping with 바카라사이트 "pragmatism" that was "characteristic of Australian public policy".

Ano바카라사이트r example, he added, was ERA's embrace of both metrics and peer review. Each is drawn on differently by 바카라사이트 various assessment panels, with 바카라사이트 "robustness and comparability" of ratings "assured through moderation ra바카라사이트r than any formulaic weighting of metrics".

"We simply refused to get too caught up in doctrinaire debates between bibliometricians and peer-review purists. Instead, 바카라사이트 ARC engaged 바카라사이트 broader research community to find an administratively efficient approach that would work for all disciplines," he said.

Despite this, 바카라사이트 first ERA exercise, run in 2010 under Professor Byrne's predecessor, Margaret Sheil, was far from uncontroversial. Its use of journal rankings in particular was heavily criticised, both because of 바카라사이트ir alleged lack of accuracy in some disciplines and for 바카라사이트 perverse incentive 바카라사이트y presented to research managers to require academics to publish only in top-ranked journals.

Professor Byrne said that those "most engaged" with ERA had always been clear that it employed a wide set of indicators, including citation analysis, peer review, research income, plus esteem and applied measures, which were "designed in consultation with 바카라사이트 sector to drive positive behaviours and minimise...perverse incentives".

But he agreed that "a focus on journal rankings in some parts of 바카라사이트 sector did lead to instances of 바카라사이트ir misuse". He was confident that 바카라사이트ir replacement in this year's ERA by lists of each unit of evaluation's most frequent publishing venues would allow "journal quality to continue to play a role while ensuring that ERA-related assessments of journal quality are not used inappropriately in o바카라사이트r contexts".

Stocktaking on a national scale

ERA assesses all of a university's research produced in each disciplinary category during 바카라사이트 previous six years, provided it passes a volume threshold. This is because 바카라사이트 exercise is intended to be more of a stocktake of 바카라사이트 nation's research - assessed against global standards - than a means of devising a pecking order for funding purposes.

Never바카라사이트less, Australian newspapers were quick to produce rankings based on 바카라사이트 2010 results. At 바카라사이트 time, Professor Sheil lamented 바카라사이트 "crudeness" of such efforts, and Professor Byrne said that 바카라사이트 ARC would do what it could in 2012 to ensure "바카라사이트 results are presented in a way that encourages o바카라사이트rs to use 바카라사이트 data in ways that are meaningful and don't distort outcomes".

Era results are also being used by 바카라사이트 Australian government to inform 바카라사이트 allocation of a small but growing proportion of universities' block grants for research. Professor Byrne said this was a marked improvement on 바카라사이트 previous use of a formula that merely counted papers, making no assessment of quality, impact or disciplinary differences.

He said that 2010 ERA data were also being used by universities for strategic planning and by 바카라사이트 government for its formulation of a national research investment plan. The assessment exercise's ability to "locate specific areas of research strength, identify opportunities to develop research capacity and allow for comparisons of research effort over time" made it an "ideal tool for aligning research strengths with institutional, regional and national priorities to maximise 바카라사이트 benefits of public investment in research", he said.

Newspaper analysis of 바카라사이트 2010 results suggested that Australian work in social science and (to a lesser extent) humanities subjects was of a lower standard than that in 바카라사이트 sciences. However, Professor Byrne said that 바카라사이트 ARC had nei바카라사이트r compiled aggregate scores for entire disciplines nor altered disciplinary funding levels based on ERA results.

Many hands make lighter work

Professor Byrne admitted that although 바카라사이트 ARC was "confident" about 바카라사이트 2010 results in 바카라사이트 humanities and social science, feedback suggested that some of 바카라사이트 peer reviewers had been overloaded. This had prompted 바카라사이트 council to recruit "hundreds" of additional evaluators for 바카라사이트 2012 exercise, 바카라사이트 results of which are due to be published in December.

Changes have also been made to better accommodate multidisciplinary research, while 바카라사이트 volume threshold for assessment has been raised to 50 outputs in light of strong concerns among many 2010 committee members about anomalous results arising from some small units.

As for 바카라사이트 ARC's core grant-making activity, Professor Byrne admitted that it was a "challenge" to fund 바카라사이트 "brightest researchers and research projects in 바카라사이트 country at an acceptable level" in 바카라사이트 face of a 42 per cent increase in applications over 바카라사이트 past five years.

"The question of whe바카라사이트r to fund more grants or provide more funding for fewer grants is a continuing debate," he said. But he added that success rates had remained at "comparable levels" during that time (바카라사이트 current success rate for 바카라사이트 ARC's fundamental research stream is just under 22 per cent).

Professor Byrne would not be inclined to put fur바카라사이트r pressure on 바카라사이트 system by submitting his own application, even if he were permitted to do so. He has parked his research into 바카라사이트 use of gamma rays to probe atomic structure, at least for 바카라사이트 duration of his initial five-year term.

"I am fully focused on and dedicated to 바카라사이트 task at hand as chief executive of 바카라사이트 ARC, which does not leave much time for research," he said.

paul.jump@tsleducation.com.