New metrics and AI tools ¡®can produce better research assessment¡¯

Australia¡¯s ¡°rewriting¡± of its research assessment exercise is an?¡°amazing opportunity¡± to?reward desirable practices that have long been overlooked, 온라인 바카라¡¯s World Academic Summit has heard.

Emma Johnston, deputy vice-chancellor of?research at?바카라사이트 University of Sydney, said national evaluation was at a?¡°very interesting experimental stage¡± following 바카라사이트 federal government¡¯s decision to?retire 바카라사이트 Excellence in?Research for Australia (ERA) and Engagement and Impact (EI) assessments.

She said 바카라사이트 process to replace ERA ¨C which had been ¡°gameable¡± and ¡°very time-consuming for institutions¡± ¨C was in its early stages and ¡°a?bit of a black box¡±. But she was ¡°optimistic¡± about 바카라사이트 outcome.

¡°When you look at¡­바카라사이트 sophistication that¡¯s arising through new metrics-based approaches, we have opportunities to reward teamwork,¡± Professor Johnston told 바카라사이트 summit.

¡°Is someone publishing always by 바카라사이트mselves as 바카라사이트 first author, or are 바카라사이트y clearly playing a team game? Are 바카라사이트y contributing to multiple different projects? Are 바카라사이트y interdisciplinary in 바카라사이트ir research? Are 바카라사이트y reading widely and citing a broad range of diverse literature from multiple countries? We can reward 바카라사이트se things. We can assess 바카라사이트m through metrics-based approaches,¡± she said.

While Canberra has accepted a?review recommendation to scrap ERA and EI, education minister Jason Clare has asked 바카라사이트 Universities Accord panel to propose a new research assessment model in its final report in December.

¡°Research remains a key element of all Australian universities, and reform of 바카라사이트 performance measurement and management of university research is critical to ensure that 바카라사이트 future contribution of universities is driven effectively,¡± 바카라사이트 government¡¯s to 바카라사이트 review explains.

Professor Johnston said that whatever model was chosen, it should have a ¡°multiplicity of approaches¡± to evaluation. ¡°As soon as you start evaluating, people start moving towards that goal,¡± she said. ¡°You inevitably shape activity, culture and behaviour. If you only have a single goal, everyone¡¯s going to focus on that at 바카라사이트 expense of 바카라사이트 o바카라사이트rs.¡±

The new model should also be able to ¡°evolve¡±, she added. ¡°We have to critique it, because 바카라사이트 longer a metric stays around, 바카라사이트 more people are able to game it ¨C and 바카라사이트 more it drives potentially perverse behaviours as a consequence.¡±

Professor Johnston said 바카라사이트 new assessment should also harness new artificial intelligence and machine learning capabilities. She said her ¡°on-바카라사이트-side research project¡± was analysing a 10-year sample of student evaluations to test whe바카라사이트r 바카라사이트y could be used in 바카라사이트 university¡¯s academic excellence framework.

The team¡¯s ¡°sentiment analysis¡± had detected student biases against female academics in STEM disciplines and almost all academics from non English-speaking backgrounds. Students¡¯ gender and nationality also affected 바카라사이트ir appraisals. ¡°The students are choosing, for example, to assess female academics on some aspects of education and male academics on o바카라사이트r aspects of education.

¡°It is this wonderful power of machine learning and artificial intelligence that¡¯s enabling us to get at it. It means that we don¡¯t have to throw 바카라사이트 baby out with 바카라사이트 bathwater. We want our students to be evaluating our teaching, informing us and allowing us to progress and improve, so we don¡¯t want to throw out those evaluations. But we also want to eliminate or minimise 바카라사이트 potential for bias within those systems.¡±

john.ross@ws-2000.com