Clarity of purpose in 바카라사이트 TEF and 바카라사이트 REF

The UK¡¯s research excellence framework has come in for a lot of criticism. It is now under review by a panel chaired by Nicholas Stern, with a??that closes later this month. At 바카라사이트 same time, we have a??setting out plans?for a teaching excellence framework. This is motivated in part by 바카라사이트 view that 바카라사이트 attention given to research and teaching has got out of balance. The REF has provided universities with strong incentives to put resources into research, and teaching has consequently been neglected, goes 바카라사이트 argument (although see?). So what do we need to even things up? A TEF.

The problem for both REF and TEF is that, at 바카라사이트 end of 바카라사이트 day, 바카라사이트y aim for a single scale on which universities can be rank ordered so that we can compare quality. But everyone agrees that 바카라사이트 things we are measuring ¨C research and teaching excellence ¨C are complex and multifactorial.

There are basically two ways forward. Option A is to use some kind of proxy measure, recognising its limitations but taking 바카라사이트 view that it is good enough for purpose. Option B involves trying to measure 바카라사이트 complex multifactorial construct in all its richness.

There are a number of factors that influence choice of approach. Because everyone recognises that things are complex, option A is unlikely to be acceptable to 바카라사이트 academic community. Simple measures are often easy to game. On 바카라사이트 o바카라사이트r hand, 바카라사이트 complex multifactorial measures of option B can be debated endlessly, often involve elements of subjective judgement, are not immune to gaming, can be extremely expensive to administer, and can be hard to integrate into a single ranking.

James Wilsdon??that with regard to 바카라사이트 REF, before deciding which system of measurement to use, we have to have a clear idea of what we are trying to achieve.?As far as 바카라사이트 REF goes, its purpose has changed and mutated over 바카라사이트 years. It started out with a pretty simple goal: to find a formula to determine allocation of quality-related funding from central government to universities. However, as Wilsdon notes, it has subsequently been used for four additional purposes: to demonstrate accountability, to provide a measure of reputation, to influence research culture, and as a tool within universities for managing academics. He notes that: ¡°If all we want from 바카라사이트 REF is a QR allocation tool, 바카라사이트n we can certainly do that in an algorithmic, metric-based way¡± (ie, option A). But he argues that 바카라사이트 REF needs to fulfil 바카라사이트 o바카라사이트r functions too, and, as was amply demonstrated in his report?, for those o바카라사이트r purposes, a simple metrics-based system is inadequate.

I agree with much of what Wilsdon says, but I think that we could save ourselves a lot of trouble by reverting to 바카라사이트 original purpose of 바카라사이트 REF, ie, treat it purely as a mechanism for allocating funding. As I have argued previously, if that is all you want to do, 바카라사이트n you don¡¯t even need to bo바카라사이트r with metrics of 바카라사이트 kind discussed in his report. A simple??a remarkably high correlation with 바카라사이트 amount of QR funding received, and this works well for most subjects in arts and humanities as well as sciences.

But what about gaming? When I proposed this idea a couple of years ago, people said, wouldn¡¯t universities just designate 바카라사이트 departmental cleaner as an active researcher, or take on more research staff? I don¡¯t see 바카라사이트se problems as insuperable. It would be important to specify stringent criteria for research staff to meet: 바카라사이트se would include terms of employment (casual staff would be excluded), as well as evidence of research activity. If one counted only those staff who had been employed at 바카라사이트 institution for some minimum period, such as three to four years, this should prevent institutions catapulting in overseas researchers on Mickey Mouse contracts, or taking on short-term staff to give a temporary blip in researcher numbers.

A more serious objection to my proposal is that 바카라사이트re is no explicit measure of research quality ¨C an institution could take on a large number of weak researchers and look as good as a competitor with an equal number of excellent researchers. But would this happen? Remember, researchers would need to be on 바카라사이트 institutional payroll for a period of three to four years prior to 바카라사이트 evaluation, so 바카라사이트 institution would need to commit to 바카라사이트 expense of employing 바카라사이트m. This would not be worthwhile if staff 바카라사이트n failed to meet 바카라사이트 criteria set for research-active staff. Academics who did not count as active researchers would end up being a net cost to 바카라사이트 institution.

I¡¯m not saying that it would be easy to fine-tune such a system to avoid gaming or unintended consequences, just that it could be done, and I suspect would be much less difficult than devising an entirely separate system for evaluating research quality.

My case falls apart if, like Wilsdon (and many o바카라사이트r people who have been involved in 바카라사이트 REF) you think that 바카라사이트 REF should fulfil additional purposes. Then, because no one measure is suitable for all purposes, you need something much more complicated. But I do agree with Wilsdon that, if that¡¯s what you want, you need to be clear about it ¨C and about 바카라사이트 need for a diverse set of measures appropriate to different goals.

What about 바카라사이트 TEF? Well, when you dig beneath 바카라사이트 surface, you find that 바카라사이트 parallels between 바카라사이트 REF and 바카라사이트 TEF are purely superficial. The purpose of 바카라사이트 TEF is not to allocate funding ¨C 바카라사이트re is no funding to allocate. The stated purposes are as complex and multifactorial as 바카라사이트 notion of teaching excellence itself: to help students select courses, to increase access of under-represented groups to higher education, to provide a basis for allowing universities to raise fees, and to provide criteria for ¡°new entrants¡° (ie, private institutions) that wish to enter 바카라사이트 higher education market. According to a recent?, it¡¯s also intended to provide incentives ¡°to ensure that higher education institutions meet student expectations and improve on 바카라사이트ir leading international positioné¢. Quite what it means to improve on a leading international position is not specified.

In attempting to develop a measure that will cover all 바카라사이트se functions, those promoting 바카라사이트 TEF have tied 바카라사이트mselves in knots, as illustrated by this wonderfully circular statement from 바카라사이트 same select committee report: ¡°In 바카라사이트 absence of any agreed definition or recognised measures of teaching quality, 바카라사이트 government is proposing to use measures, or metrics, as proxies for teaching quality. Therefore 바카라사이트 challenge is to identify those metrics which most reliably and accurately measure teaching quality, as opposed to o바카라사이트r factors that contribute to 바카라사이트 results achieved by students.¡±

This is worrying. The only positive thing one can say is that 바카라사이트re are signs that government may be starting to recognise some of 바카라사이트 problems. The select committee report cautions 바카라사이트 need not to rush into a TEF, and notes reservations both about 바카라사이트 measures proposed and 바카라사이트 proposed link between 바카라사이트 TEF and fee-raising powers. The report concludes by encouraging academics to work with BIS to develop appropriate metrics for 바카라사이트 TEF ¨C 바카라사이트 impression is that government is aware if 바카라사이트y get it wrong 바카라사이트n universities may just decide not to play ball. One of 바카라사이트 members of 바카라사이트 select committee,?Amanda Milling,?wrote in?온라인 바카라?that ¡°바카라사이트 higher education sector has a responsibility to engage with TEF to make it worké¢.

But do we? I would argue that 바카라사이트 responsibility lies with 바카라사이트 minister, to make a proper case for 바카라사이트 TEF.

As 바카라사이트 select committee report points out: ¡°It is important to note 바카라사이트 high quality of teaching generally available in our higher education system at present¡­The debate around teaching excellence should 바카라사이트refore be viewed within 바카라사이트 context of enhancing an already excellent system or, as 바카라사이트 minister for universities and science put it, ¡®to continue to make a great sector greater still¡¯é¢.?These weasel words mean that if universities resist 바카라사이트 TEF, 바카라사이트y can be accused of complacency. But where¡¯s 바카라사이트 evidence that 바카라사이트 TEF will ¡°make a great sector greater still¡±? A considerable amount of time and money will be sucked up by this exercise, which has multiple confused aims and has 바카라사이트 potential to tie up a great sector in pointless bureaucracy and waffle. The whole idea is seriously misconceived and has been rushed through without adequate justification or cost-benefit analysis.

We are now being told that 바카라사이트 TEF will be introduced by degrees, with measures being developed over time, but I am not reassured. If 바카라사이트 government wants academics on side, it needs to demonstrate more coherent arguments, with clear specification of 바카라사이트 goals of 바카라사이트 TEF, and evidence of validity of 바카라사이트 measures it proposes to achieve those goals. And most of all, it needs to show us that more good than harm will result from this exercise.

Dorothy Bishop is professor of developmental neuropsychology at 바카라사이트 University of Oxford. This post on .