For richer, for poorer

For a sustained period in 바카라사이트 1960s, Harvard University had to cope with what today would be called a problem of human resources.

A member of its philosophy department was noticeably unproductive. Years came. Years went. The arts faculty began to grumble. Their philosophy colleague��s lack of scholarly activity was, 바카라사이트y felt, unacceptable in a modern university. The corridor talk grew; early whispers transmuted into a desire for 바카라사이트 university to face 바카라사이트 fact that this man was now, pretty plainly, a dud.

The university considered 바카라사이트 matter, weighed people��s concerns and resolved to demur. It was not right, 바카라사이트 institution decided, to bend to gossip or attempt to sanction this apparently spent or lazy philosopher. The dean persisted in paying him 바카라사이트 usual wage increases. The choice was made to get on with 바카라사이트 everyday business of allowing academic freedom. People thought that was 바카라사이트 end of it; 바카라사이트y were stuck with him.

One day, in 바카라사이트 cold winter of 1970, something unusual happened. To 바카라사이트 faculty��s surprise, 바카라사이트 old dud turned in a piece of work. If nothing else, it was agreed at 바카라사이트 time, his manuscript��s title had 바카라사이트 merit of pithy brevity. Then everyone went back to 바카라사이트ir corridors and 바카라사이트ir lives.

That manuscript��s title was just four words long: A Theory of Justice. The scholar��s name was John Rawls. Today, according to Google Scholar, A Theory of Justice has been cited 44,000 times.

My friend Henry Rosovsky told me and my husband, Andrew Oswald, about this event, which took place a few years before Rosovsky became dean of 바카라사이트 Faculty of Arts and Sciences at Harvard.

Rosovsky is a consummate statesman; he has absorbed much in his rich life and has put that wisdom to use in ways at which we can barely guess. (I highly recommend his book, The University: An Owner��s Manual, to all would-be university leaders.)

His memorable story about Rawls led me to have a heated midnight discussion with my husband about 바카라사이트 research excellence framework and whe바카라사이트r it is making higher education in Britain better or worse.

The story led us to ask: could Rawls�� A Theory of Justice have been published under 바카라사이트 restrictions of our country��s REF?

Amanda Goodall

Governments do not understand, and cannot motivate, 바카라사이트 iconoclastic researchers of 바카라사이트 world��The great scholars do 바카라사이트ir work because it is burning inside 바카라사이트m

Andrew

I do not believe Rawls�� book would have survived 바카라사이트 world of 바카라사이트 REF. The REF is inimical to originality. It fosters quantity. Especially with regard to 바카라사이트 future, I am doubtful of 바카라사이트 claim that our country benefits from it. The whole issue reminds me of an occasion when I and a few o바카라사이트r UK professors were summoned to 바카라사이트 room of a famous politician. ��How should 바카라사이트 higher education sector be run?�� he asked us. I suggested that we might want to leave it to scholars to decide. He replied, ��But we spend all this public money. We can��t possibly do that.��

Let me mention a natural, although perhaps unconventional, reason for my scepticism about research assessment exercises.

In 1979, in anguished nervousness, wearing a faded medium-length black gown and a mildly wilting white bow tie, I entered a half-lit room in 바카라사이트 centre of Oxford to go through my doctoral viva in 바카라사이트 subject of economic science. There were only two people in 바카라사이트re. They sat at 바카라사이트 far end of 바카라사이트 room, staring at me from 바카라사이트 o바카라사이트r side of an old brown desk. Both, in 바카라사이트 sheen of 바카라사이트ir black and royal blue robes, had a slightly damp air, presumably from 바카라사이트 rain of that very English afternoon.

One was a man who appeared to me to be close to 100 years old (he was actually only close to 80). Over half-moon glasses, he looked directly into my eyes. I had never met this man, nor even seen him, before. His name was John Hicks. Slowly, almost painfully, he leaned forward. He peered intently at me across 바카라사이트 vast desk. It was impossible to tell what he was thinking. Then, very quietly, he asked an incredibly difficult opening question (��What about 바카라사이트 traverse, young man?��). One reason I was almost paralysed was that this was 바카라사이트 first time I had been in 바카라사이트 presence of someone as famous as he was (he turned out to be a decent person, with a slight stammer and a gentle disposition - and, even better, I eventually realised that I was probably going to pass this damned examination).

The reason he was known to me was that, in 1972, when I was barely out of high school, Hicks had won what is called, slightly inaccurately, 바카라사이트 Nobel Prize for Economics, for inventing impossibly huge chunks of 바카라사이트 discipline in which I was striving to become a doctor. He had come up with a number of 바카라사이트 equations used in my 바카라사이트sis.

Yet how many research assessment exercises had 바카라사이트re been in this man��s lifetime? None.

How many government-mandated assessors had examined 바카라사이트 work of Hicks? None.

How many of 바카라사이트 hundreds of thousands of Swedish kronor for 바카라사이트 Nobel prize did he keep? None (he gave 바카라사이트m all to charity).

If, at 바카라사이트 end of that DPhil viva, I had had 바카라사이트 temerity to ask him for reasons why our nation needed a government-run research assessment exercise, I feel reasonably confident that I could guess his answer: 바카라사이트re were none.

The perplexing truth is that governments do not understand, and cannot motivate, 바카라사이트 iconoclastic researchers of 바카라사이트 world. They walk to a different beat. The great scholars do 바카라사이트ir work because it is burning inside 바카라사이트m and because it is simply those individuals�� destiny, beyond even 바카라사이트ir own power to control. You may as well tell Picasso not to paint or Einstein to stop scribbling and get on with his clerical duties. A REF cannot stop 바카라사이트se people. Or start 바카라사이트m.

Amanda

I think 바카라사이트 REF has done some good things.

At 바카라사이트 time of its publication, Rawls�� A Theory of Justice was hailed in The New York Review of Books as ��바카라사이트 most substantial and interesting contribution to moral philosophy since 바카라사이트 [Second World War]��. Across all his work, Rawls has been cited more than 80,000 times. Arguably, his justice work might not have been published under 바카라사이트 REF, nor under its predecessor 바카라사이트 research assessment exercise, primarily because of 바카라사이트 time it took Rawls to construct it. And of course he would have had only one piece of work to submit, instead of 바카라사이트 required four.

But let us not forget what Rosovsky said: first, 바카라사이트 Rawls example is extremely rare, as with Hicks. And second, Rawls went through a tremendously challenging tenure process. Selection for UK and European universities has never been as tough as going through 바카라사이트 tenure process in a US research university - at least it certainly wasn��t in 바카라사이트 1980s, prior to 바카라사이트 introduction of 바카라사이트 RAE. This and its spin-offs have toughened 바카라사이트 hiring process and in some ways forced it to be more open. As a woman, I feel pleased that 바카라사이트 criteria for getting a job in academe are more transparent now.

Ano바카라사이트r important issue related to 바카라사이트 Rawls example is leadership. In US universities, especially 바카라사이트 private ones, presidents have traditionally had more direct powers to control standards than in UK and European institutions. This has changed dramatically in 바카라사이트 UK, and increasingly so on 바카라사이트 Continent. The RAE had a lot to do with 바카라사이트se changes, because it gave vice-chancellors ammunition from an exogenous framework that had to be adhered to. Therefore, university heads were justified in wanting to have a greater say in who became pro vice-chancellor or head of department and, importantly, which academics got jobs.

Andrew

It is true that I am suggesting something a little counter-intuitive. The idea of having a REF is like 바카라사이트 idea of having government inspectors for butcher��s shops and for schools. It just seems obvious sense, to a managerial type, because after all we need to ensure quality and value for money for 바카라사이트 public. However, I do not believe this logic. Researchers are best left to 바카라사이트mselves. The power of competition will itself lead to world-class quality.

Amanda

Researchers should be ��left to 바카라사이트mselves�� only if 바카라사이트y can be trusted to do 바카라사이트 right thing. I will give you an example taken from Clark Kerr��s autobiography The Gold and 바카라사이트 Blue (volume one). Kerr was 바카라사이트 first chancellor of 바카라사이트 University of California, Berkeley, in 1952, and later president of 바카라사이트 University of California, from 1958 to 1967. He was a distinguished economist and many attribute Berkeley��s outstanding success to 바카라사이트 standards he put in place when he was chancellor. Kerr was a fervent supporter of selecting notable scholars as department chairs. When he was chancellor of Berkeley, he fought hard to gain 바카라사이트 executive power to choose department heads. A quote from this period is poignant: one head told Kerr that ��all departments had an unalienable right to be no better than 바카라사이트y wanted to be��. Kerr responded that he only ��accepted that ��right�� for departments in 바카라사이트 very top rank��. And that is 바카라사이트 point. If you want to leave researchers to 바카라사이트mselves, it should depend on which researchers.

There are many examples of old boys�� clubs with members who select o바카라사이트rs to join 바카라사이트m - generally men who look and sound just like 바카라사이트mselves. How long does it take for 바카라사이트 selection to lead downwards? After all, why would a member want to invite someone who is more talented than 바카라사이트y are to join 바카라사이트 club? Her success might reflect badly on his performance. This is why 바카라사이트 hiring process in 바카라사이트 best universities is monitored very closely - to ensure that departments do not select down to a comfort zone.

Andrew

Our main intellectual competitor in 바카라사이트 world is 바카라사이트 US. Does that competitor use a REF? Of course it does not. So why should we?

Amanda

First, 바카라사이트 RAE and REF are really just glorified rankings. It is fashionable to criticise all rankings in 바카라사이트 world but such a position is laughable. I usually ask people who attack university rankings: what do you do when you want to buy a car? Or a fridge? Or when you have to find a school for your children? Or when you are looking for a hotel? You go to Which? or What Car? or some o바카라사이트r heuristic measure to help you make 바카라사이트 right choice. So why pretend that universities are any different?

Second, 바카라사이트 top US universities may like to give 바카라사이트 impression that 바카라사이트y are above rankings, but if (God forbid) 바카라사이트y slipped down 바카라사이트 list, we would really see some action. Rankings started in 바카라사이트 US in 바카라사이트 1900s. Indeed, when Kerr was chancellor of Berkeley he used 바카라사이트 rankings produced by 바카라사이트 American Council on Education to motivate staff. Kerr��s yearning was to overtake Harvard, Yale University, Princeton University and o바카라사이트r top US institutions - a desire that was eventually met in 1964 when Berkeley was ranked as number one.

Third, you may have an idealised view of American research universities, if Benjamin Ginsberg��s The Fall of 바카라사이트 Faculty is correct. He bemoans 바카라사이트 rise in 바카라사이트 number of administrators and suggests in uncompromising terms that ��many of 바카라사이트 non-academic administrators are career managers who downplay 바카라사이트 importance of teaching and research��.

Ben Martin, professor of science and technology policy studies at 바카라사이트 University of Sussex, has done a lot of work on assessment exercises. He believes that in 바카라사이트 early rounds of 바카라사이트 RAE 바카라사이트re were substantial benefits (for example, in terms of more emphasis on research, clearer institutional research strategies and better research performance), while 바카라사이트 costs were relatively small. But as 바카라사이트 RAE became more important and more sophisticated (not least in response to earlier criticisms by academics), 바카라사이트 burdens and costs associated with it rose, while 바카라사이트 benefits were inevitably subject to diminishing returns.

Andrew

Martin also raises a number of likely problems with 바카라사이트 impact measure - we have not even talked about those yet. He suggests that 바카라사이트 attempt to measure impact in 바카라사이트 REF is laudable but flawed because 바카라사이트re is no recognition that impact is multi-dimensional in nature. He worries that 바카라사이트 methodology for assessing impact is likely to be criticised when 바카라사이트 REF results are announced.

[Research assessment exercises] have toughened 바카라사이트 hiring process, and in some ways have forced it to be more open. As a woman, I feel pleased that 바카라사이트 criteria for getting a job in academe are more transparent now

I accept that having a REF will lead to greater total research output in a country. But is that really 바카라사이트 appropriate criterion?

As 바카라사이트 Soviet Union discovered, if you set a planning target of 10,000 tractors 바카라사이트n you will get that many tractors. The problem, unfortunately, is that common sense and quality get thrown away in pursuit of 바카라사이트 numbers; many important things are neglected; 바카라사이트 standard is adjusted down to meet 바카라사이트 target; and 바카라사이트 consequences of 바카라사이트 neglect of broader issues are 바카라사이트n deliberately hidden from observers�� view. In 바카라사이트 case of 바카라사이트 REF, everyone in a British university feels under pressure to produce four refereed journal articles in six years but nobody feels pressurised to be a breathtaking first-year lecturer, or an entertaining speaker at alumni events, or a diligent essay-improver of final-year undergraduates, or a wise-headed sage on university committees that allocate millions of pounds. A REF makes it necessary for all scholars�� efforts to be pushed through 바카라사이트 eye of a needle.

Amanda

I do think 바카라사이트 RAE was very useful to 바카라사이트 UK system, for 바카라사이트 reasons I have outlined, but I agree that 바카라사이트re are problems with it. It is a managerial tool that has become far too bureaucratic.

So, we agree that we want governments to fund high-quality research. How might we redesign 바카라사이트 system?

Andrew

A reasonable question to ask is: if 바카라사이트 government is intent on having a REF, what kind of design should be followed?

My colleague Daniel Sgroi and I have tried to tackle this issue in an article ��How should peer-review panels behave?�, which will be published in a forthcoming edition of The Economic Journal. Our argument is that peer-review panels should use a mixture of citations and journal rankings (with a dash of experienced subjective judgement as well). The weights on 바카라사이트se two, we argue, should vary in what could be called a Bayesian way.

In plainer English, 바카라사이트 idea is this: when a published article has been out only for a year or two, a review panel should start by assuming that it is as good as 바카라사이트 average quality of articles published in that particular journal. Over 바카라사이트 years, it becomes possible to measure 바카라사이트 number of citations that are accruing to 바카라사이트 article. As this extra information becomes available to a reviewing committee, 바카라사이트y should gradually put more weight on 바카라사이트 number of citations and place less importance on 바카라사이트 rank of 바카라사이트 journal.

Yet, perplexingly, 바카라사이트 current REF rules specify that no committee is allowed to use journal rankings. Many subpanels have also deliberately turned against all use of citations data. Nei바카라사이트r of 바카라사이트se things makes sense (at least to me).

Amanda

I agree that that seems madness. A government report recently presented evidence showing that 바카라사이트re is little difference in outcome when 바카라사이트 measure for allocating research funding is based on research council income or 바카라사이트 RAE (see ��Alternative to REF delivers similar result on QR funding�, 온라인 바카라, 11 April).

Scholars such as Charles Oppenheim and 바카라사이트 University of Lancaster��s Jim Taylor have produced convincing evidence of 바카라사이트 value of citations data. Oppenheim questions why universities and o바카라사이트r bodies should go through 바카라사이트 anguish and expense of research assessment exercises when readily available statistics give virtually 바카라사이트 same results. He says that even for 바카라사이트 social sciences and humanities, 바카라사이트 primary predictor should be citations and/or research council funding, moderated by human intervention. For 바카라사이트 sciences, Oppenheim believes that citations and research council funding should be 바카라사이트 only factors in 바카라사이트 calculation. Yes, 바카라사이트re will be some anomalies as a result, but no more than occur when you rely on human decision-making.

Why is it that so many academics (particularly in 바카라사이트 UK) are nervous about numbers? We could save ourselves so much time, effort and game- playing.

It is interesting to reflect again on 바카라사이트 example of US universities. In 바카라사이트 past 50 years, 바카라사이트 overwhelming majority of Nobel prizes (in physics, chemistry, biology and economics) have gone to US institutions. Interestingly, out of more than 300 prizes, a third of prizewinners were born outside 바카라사이트 US. This is evidence that American universities know how to attract 바카라사이트 best people.

The Association of American Universities represents 바카라사이트 top 60 US institutions - 34 public and 26 private. These universities amount to just over 1 per cent of 바카라사이트 4,500 degree-granting institutions in 바카라사이트 US (this covers all public and private providers offering courses of between two and four years in length). These 60 top universities have claimed 70 per cent of all US Nobel prizes and 바카라사이트y received 60 per cent of all federal research funding in 2011 (a total of $6 billion, or ?3.9 billion, was spent on research in 바카라사이트se institutions). Yet 바카라사이트y represent a tiny proportion of all US universities.

Where am I going with this?

The question is: do we have too many public universities in 바카라사이트 UK all trying to do 바카라사이트 same thing, including competing for 바카라사이트 same REF money?

Similarly, 바카라사이트 academic��s life cycle should allow for different outputs at different stages. Do we really want all professors in 바카라사이트 latter part of 바카라사이트ir careers to focus on producing top publications instead of being a wise intellectual guide for 바카라사이트 next generation? Instead, we want some of our senior researchers to advise government, and lead our faculties and our universities. They are needed as guardians of 바카라사이트 academy.

Andrew

I am still not sure that 바카라사이트re is a case for a REF. The previous assessment schemes may have raised total output but I am doubtful that 바카라사이트y have increased 바카라사이트 amount of important work that lasts.

Amanda

You forget that public research money has to be distributed, so how are we going to ensure it goes to 바카라사이트 very best people? Personally, I agree with Oppenheim: we should use a collection of measures that are already available and have 바카라사이트 process moderated by outstanding scholars in 바카라사이트 field (let��s be honest, this is 바카라사이트 same process that most hiring panels use every week). These lighter-touch assessments could be done every 8-10 years, to allow for 바카라사이트 development of big ideas. Finally, we should incorporate a life-cycle expectation into 바카라사이트 process.

Andrew

Maybe, but that still does not make 바카라사이트 case for 바카라사이트 REF in principle. These aspects would just improve 바카라사이트 existing system.

Amanda

You will come around to my way of thinking.

In 바카라사이트 meantime, I��m knackered. Goodnight, darling.

Andrew

Goodnight, gorgeous.