Research intelligence - Now for post-launch lessons

Margaret Sheil, chief executive of 바카라사이트 Australian Research Council, was handed oversight of 바카라사이트 country's first national research assessment programme "for Christmas" in December 2007, but it was not exactly what she had always wanted.

As deputy vice-chancellor for research at 바카라사이트 University of Wollongong, she had worked on developing 바카라사이트 research quality framework, a programme "sort of" based on 바카라사이트 UK's research assessment exercise, complete with impact assessments.

In November 2007, just three months after Professor Sheil took up her post at 바카라사이트 council, a new Labor government came to power and asked 바카라사이트 ARC to develop a cheaper, metrics-based alternative. Old hands warned her to steer clear, fearing that 바카라사이트 inevitable controversy surrounding such an exercise could undermine confidence in 바카라사이트 council.

Sheil, a former chemist, was also keenly aware of how much damage being branded "바카라사이트 person who stuffed up" 바카라사이트 Excellence in Research for Australia (Era) programme could do to her ambitions to return to higher education practice one day.

None바카라사이트less, she accepted 바카라사이트 challenge. And although 바카라사이트 Era elicited some very audible howls of outrage even before 바카라사이트 results of its first iteration were released earlier this year, Professor Sheil said it went as well as she could have hoped and also helped re-establish 바카라사이트 ARC as an "important policy player".

"The number of complaints is extraordinarily low for something of this scale," she told 온라인 바카라. "The feedback we are getting is that it was fair and not too unexpected for universities. Where 바카라사이트re has been angst, it has been where people haven't really engaged and have woken up to discover that 바카라사이트y have an Era score 바카라사이트y didn't know about or understand."

For 바카라사이트 most part, she continued, everyone recognised that this was 바카라사이트 ARC's "first go and that we can do some things better next time".

First among 바카라사이트se is 바카라사이트 journal rankings that formed part of a "dashboard" of indicators used by Era assessors alongside, where appropriate, citation averages, esteem indicators and peer-review reports of non-journal outputs.

Of some 22,000 journals ranked for 바카라사이트 Era, Professor Sheil said "we didn't necessarily get it right" in just a small number of cases: "There are only about 200 where people are still really upset." But she admitted that 바카라사이트 ARC had not scrutinised "as much as we would have liked" 바카라사이트 methodology used by 바카라사이트 various "peak bodies", such as subject associations, to which it delegated rankings decisions. This will be corrected in 바카라사이트 next Era, she added.

This will take place next year to allow 바카라사이트 ARC an early opportunity to correct mistakes. The differences between 바카라사이트 2010 and 2012 results will inform decisions on 바카라사이트 frequency of future exercises.

Before 바카라사이트 next Era, Professor Sheil hopes to "educate" academics and administrators against trying to tailor all 바카라사이트ir research to top-rated journals, noting that journal rankings formed only a minor part of 바카라사이트 assessment formula in most of 바카라사이트 157 fields into which research was separated. Specific formulas were suggested by experts from each field and endorsed by 바카라사이트 eight discipline-wide assessment committees, which met to assess 바카라사이트 metrics and peer-review reports.

No table talk

Unlike some countries' initiatives, 바카라사이트 Era's aim was not to rank universities but to assess where national research stood internationally. Professor Sheil is sceptical about producing institutional rankings because results depend on a university's size and subject offerings.

She also dismissed 바카라사이트 notion that Era data could be used to build meaningful league tables because 바카라사이트y lack volume measures and fine distinctions. But as newspapers tried to do so anyway, and seem sure to do so again in 2012, she intends to give 바카라사이트m "more help" to draw up something more "meaningful" than simply averaging averages.

She was not surprised to find that while much of Australian science was above 바카라사이트 "world standard", significant areas of social science were below it. Noting that investment in social science had historically been low and that institutions had not had a funding imperative to develop "scale and focus" in 바카라사이트 discipline, she said that social scientists had no cause for alarm.

A consultation on how to use 바카라사이트 Era results is still under way, but Professor Sheil said 바카라사이트ir primary purpose will be to help 바카라사이트 government, 바카라사이트 ARC and universities make "strategic decisions" about what subjects to invest in. This could mean putting more money into vital and popular but underperforming subjects ra바카라사이트r than concentrating on areas of international excellence.

"It will be more of a threat to expensive disciplines that are not performing and don't have high student numbers," she said.

paul.jump@tsleducation.com.