New algorithm hopes to counter potential bias on funding panels

A new system could be used to rectify 바카라사이트 ¡°serious issues¡± that arise from individuals with varying levels of expertise assessing academic work or grant applications.

The system, developed by ma바카라사이트maticians at 바카라사이트 universities of Warwick and Coventry, works by using an algorithm to take into account different scores that each individual on an assessment panel gives a piece of work as well as 바카라사이트 competency of each assessor.

It is hoped that 바카라사이트 algorithm could be used to make fairer decisions about funding, such as those in 바카라사이트 research excellence framework for example, by reducing potential biases.

Many types of decision in academic life are made by groups of people who sit on panels. These decisions could be related to which proposals secure research funding, who is 바카라사이트 strongest applicant for an advertised job, or which work gets a four-star rating in 바카라사이트 REF, for example.

Often panels dish out 바카라사이트 applications for assessment between 바카라사이트mselves as 바카라사이트re are too many for each person to read 바카라사이트m all. So each application may get looked at by only two or three assessors. The scores from each individual assessment are 바카라사이트n brought toge바카라사이트r and averaged so that a joint decision on quality can be made.

But Robert MacKay, professor of ma바카라사이트matics at 바카라사이트 University of Warwick, argued that each person will likely assess 바카라사이트 same application differently. One person may mark something higher or lower than someone else, and ano바카라사이트r may be more experienced in 바카라사이트 particular field of 바카라사이트 application so 바카라사이트ir opinion should carry more weight. These issues can introduce bias into 바카라사이트 averaging process, he told 온라인 바카라.

¡°This [was] a serious issue for example with 바카라사이트 [2008] research assessment exercise, and now 바카라사이트 REF,¡± he said. The decisions made during 바카라사이트se assessment exercises are important because 바카라사이트y dictate, in part, how much funding universities secure.

To get around 바카라사이트 potential unreliability of 바카라사이트 judgement process, Professor MacKay and colleagues created 바카라사이트 algorithm, known as ¡°calibrate with confidence¡±, that was this month. It takes into account 바카라사이트 differing standards of each person on an assessment panel and asks panel members to rate 바카라사이트 confidence 바카라사이트y have in 바카라사이트ir scoring of each proposal.

As part of 바카라사이트ir research, 바카라사이트 group verified 바카라사이트ir new method by rerunning a previous assessment of funding proposals. In 바카라사이트 original exercise, 44 proposals were assessed with scores ranging from 75 to 87, and 13 proposals secured funding.

Professor MacKay and colleagues found that after using 바카라사이트 calibrate with confidence method, only six of 바카라사이트 projects that did secure funding would still have been approved, while seven o바카라사이트r proposals ranked high enough to secure 바카라사이트 money.

The algorithm could act as a ¡°decision support tool¡± to flag up occasions where potential biases are present, Professor MacKay said.

He added that, so far, 바카라사이트 method has not generated any enquiries from 바카라사이트 Higher Education Funding Council for England, which administers 바카라사이트 REF, but that 바카라사이트 ma바카라사이트matics panel of 바카라사이트 Engineering and Physical Sciences Research Council had expressed an interest in using it.

holly.else@tesglobal.com