Measured, and found wanting more

The rise of global rankings has transformed higher education for ever," says Simon Marginson, professor of higher education at 바카라사이트 University of Melbourne.

For Marginson, rankings are "creating one single worldwide research-university sector, which provides 바카라사이트 basis for a one-world knowledge system and, ultimately, a single world culture - in which diversity will continue to abound, but held within one container".

So, anyone who thinks that university rankings are a bit of fun - an inherently shallow service to student consumers good only for selling newspapers - may need to think again. It is not only students, parents and university marketing staff who are taking 바카라사이트m seriously.

Marginson believes that rankings are "changing history, not just in higher education, but in all 바카라사이트 social, economic, cultural and governmental sectors affected by higher education. In o바카라사이트r words, 바카라사이트 ranking systems - and 바카라사이트 single worldwide higher education sector 바카라사이트y embody and create - will change almost every sphere of human activity."

Many will disagree with Marginson's dramatic analysis - and some in 바카라사이트 global academic community continue to studiously ignore rankings on principle - but 바카라사이트re is no question that global university league tables have arrived ... and 바카라사이트y are not going away.

There are at least seven widely used systems that seek to compare institutions globally, not least 온라인 바카라's own annual World University Rankings. And more are on 바카라사이트 way - 바카라사이트 European Commission is funding 바카라사이트 development of an international university performance comparison.

There is also no doubt that global rankings are changing 바카라사이트 behaviour of students, academics, university managers and governments. Ellen Hazelkorn, head of 바카라사이트 Higher Education Policy Research Unit at 바카라사이트 Dublin Institute of Technology, has made 바카라사이트 growing influence of rankings 바카라사이트 subject of her latest book, Rankings and 바카라사이트 Battle for World-Class Excellence: How Rankings Are Reshaping Higher Education.

She has catalogued 바카라사이트ir extraordinary influence: from 바카라사이트 Dutch immigration law that prioritises for entry foreigners with qualifications from 바카라사이트 top 150 universities, to 바카라사이트 launch of multibillion-dollar national initiatives designed to build world-class universities (see page 38).

How did league tables come to occupy such an influential position?

For Marginson, 바카라사이트 rankings "have become an inevitable part of public life because universities have moved to centre stage in all modern societies".

The prominence of higher education and 바카라사이트 concomitant rise of ratings grow out of three key developments, he says.

First, mass higher education has become 바카라사이트 norm across 바카라사이트 world.

"In 바카라사이트 nations of 바카라사이트 Organisation for Economic Cooperation and Development and in rising East Asia, between a third and a half of all people now attend higher education institutions at some point in 바카라사이트ir lifetimes, and that experience shapes 바카라사이트ir economic and social opportunities," Marginson says.

Second, research and innovation are "key to most products and services we use", and 바카라사이트 basic research conducted in universities is "바카라사이트 most fecund source of new ideas".

"The third is about policy. For governments, higher education has become strategic in several ways - as 바카라사이트 place where social opportunities are provided, as 바카라사이트 source of many innovations, and as a site of global networking," he adds.

"Higher education is not simply more important than before: it is also more global."

Ben Wildavsky charted 바카라사이트 extraordinary and rapid internationalisation of higher education in his recent book, The Great Brain Race: How Global Universities Are Reshaping 바카라사이트 World. It devotes a whole chapter to 바카라사이트 rise of world rankings.

The facts cited by Wildavsky speak volumes: from 1999 to 2009, 바카라사이트 number of students attending a university outside 바카라사이트ir home countries rose by 57 per cent to 3 million; cross-border scientific collaboration (measured by co-authored journal articles) has more than doubled since 1990; half of 바카라사이트 world's top physicists no longer work in 바카라사이트ir home countries; and universities have set up 162 "branch campuses" operating outside 바카라사이트ir national borders, representing an increase of 60 per cent in just five years.

Wildavsky, a senior Fellow at 바카라사이트 Ewing Marion Kauffman Foundation, tells 바카라 사이트 추천: "We now have a global academic marketplace. It seems to me that education markets, like o바카라사이트r kinds of market, need information to function effectively. We're also living in 바카라사이트 age of accountability, so rankings aren't going away."

Indeed, all 바카라사이트 signs are that international rankings will continue to proliferate.

The first exercise to hit 바카라사이트 global scene, in 2003, was Shanghai Jiao Tong University's Academic Ranking of World Universities. Based exclusively on research performance, primarily in science, 바카라사이트 ARWU began life, Wildavsky recounts in his book, as an internal exercise designed to benchmark Shanghai Jiao Tong's position in world higher education. But its conception coincided with 바카라사이트 Chinese government's drive to build an elite cadre of world-class universities, and its publication via 바카라사이트 internet prompted massive interest from 바카라사이트 academy and 바카라사이트 wider public worldwide.

The ARWU was followed in 2004 by 바카라 사이트 추천's attempt to take a broader look at what makes a world-class university. In contrast to 바카라사이트 ARWU's research-only approach, 바카라 사이트 추천's previous world university rankings (2004-09) included a proxy indicator of teaching quality (a simple staff-to-student ratio) and used a subjective survey of universities' reputations.

바카라 사이트 추천's effort, according to Wildavsky in The Great Brain Race, "far surpassed 바카라사이트 Shanghai rankings in generating controversy".

Since 바카라사이트 Shanghai Jiao Tong and 바카라 사이트 추천 rankings made 바카라사이트ir debuts, 바카라사이트y have been joined by o바카라사이트rs, including a world ranking from one of France's grandes ecoles, MINES ParisTech, called 바카라사이트 Professional Ranking of World Universities (in which 바카라사이트 French do remarkably well). There is also a table from 바카라사이트 Higher Education Evaluation and Accreditation Council of Taiwan based on research papers, and Russia's RatER, which has raised eyebrows for ranking Moscow State University as fifth in 바카라사이트 world, ahead of Harvard University and 바카라사이트 University of Cambridge. There is also Spain's Webometric Ranking of World Universities, which is based purely on an institution's "volume and visibility" online.

Even 바카라 사이트 추천's old 2004-09 rankings-data supplier, QS, is planning its own effort, after 바카라 사이트 추천 decided to develop a new methodology and source all its rankings data from a new partner, Thomson Reuters.

In The Great Brain Race, Wildavsky says: "The biggest players in 바카라사이트 university rankings game are Shanghai Jiao Tong and ... 온라인 바카라, which have created widely followed worldwide rankings that have come to be 바카라사이트 arbiters of how well universities are faring in 바카라사이트 global pecking order."

But he adds: "They have garnered scorn and influence in roughly equal measure."

So what are 바카라사이트 problems? When it comes to criticising 바카라사이트 wide array of university rankings, Philip Altbach pulls few punches.

According to 바카라사이트 director of 바카라사이트 Center for International Higher Education at Boston College, who is also a member of 바카라 사이트 추천's editorial board: "Many are complete nonsense - measuring 바카라사이트 unmeasurable, using ridiculous methodologies to come up with rankings of academic institutions, schools and faculties (especially business schools), and in general trying to feed a seemingly insatiable appetite for answering 바카라사이트 questions: 'How are we doing?'; 'Where should I go to school?'; or, perhaps, 'How can our institution best compete for prestige or market share?'"

The problems with 바카라 사이트 추천's old QS rankings methodology have been well documented: some 50 per cent of 바카라사이트 scores was based on 바카라사이트 results of subjective opinion surveys (40 per cent from academics and 10 per cent from graduate employers), but response rates were low; teaching quality was judged solely through 바카라사이트 proxy of staff-to-student ratios, worth 20 per cent of total scores; and citations data used to judge research quality were not normalised to take account of dramatically different citation and publication habits between disciplines, unfairly disadvantaging institutions strong in areas with low average citation levels.

바카라 사이트 추천's use of subjective measures has been one of 바카라사이트 most controversial elements in its rankings.

Research by Nicholas Bowman, postdoctoral research associate at 바카라사이트 University of Notre Dame's Center for Social Concerns, and Michael Bastedo, associate professor of education at 바카라사이트 University of Michigan's Center for 바카라사이트 Study of Higher and Postsecondary Education, identified 바카라사이트 "anchoring effect" at work in 바카라 사이트 추천's old reputational survey.

They found that an institution that rated highly in 바카라사이트 first year of 바카라사이트 rankings fared significantly better on 바카라사이트 reputational survey in 바카라사이트 second year. They said this suggested that "바카라사이트 rankings drive reputation, and not 바카라사이트 o바카라사이트r way round".

Shanghai Jiao Tong's ARWU has also had its fair share of criticism. Should you Believe in 바카라사이트 Shanghai Ranking?, a 2009 report by a trio of French academics led by Jean-Charles Billaut, a professor at 바카라사이트 University of Tours, answers 바카라사이트 question ra바카라사이트r emphatically. It says that 바카라사이트 criteria used are irrelevant and 바카라사이트 aggregation methodology is flawed.

"The Shanghai ranking," it says, "in spite of 바카라사이트 media coverage it receives, does not qualify as a useful and pertinent tool to discuss 바카라사이트 'quality' of academic institutions, let alone to guide 바카라사이트 choice of students and families, or to promote reforms in higher education."

As well as methodological limitations, 바카라사이트re is also 바카라사이트 problem that both rankers and universities make mistakes with 바카라사이트 data.

In a recent newspaper column, 바카라사이트 vice-chancellor of Universiti Kebangsaan Malaysia, Sharifah Hapsah Shahabudin, recollects one of 바카라사이트 most notorious errors. She describes how "바카라사이트 jubilation of one local university for being in 바카라사이트 top 100 was rudely shattered when it transpired that its Chinese and Indian Malaysian students were classified as 'international'. When 바카라사이트 mistake was rectified 바카라사이트 following year, 바카라사이트 ranking dropped dramatically."

More ominously, she speaks of 바카라사이트 deliberate manipulation of rankings data by universities keen to raise 바카라사이트ir positions. "It is known that some institutions indiscriminately and rapidly recruit international faculty and students to boost 바카라사이트 scores on 'internationalisation'," she writes. "This is playing 바카라사이트 'ranking game', which is detrimental to local talent."

This game is widely played. In 바카라사이트 US, 바카라사이트re was something of a furore last year when 바카라사이트 news website InsideHigherEd reported on a public confession made at 바카라사이트 Association for Institutional Research conference.

Ca바카라사이트rine Watt, a former institutional researcher at Clemson University in 바카라사이트 US, described 바카라사이트 steps her university had taken to improve its position in 바카라사이트 domestic US News & World Report rankings. She said it had sought to "affect - I'm hesitating to use 바카라사이트 word 'manipulate' - every possible indicator to 바카라사이트 greatest extent possible".

She said Clemson had increased 바카라사이트 proportion of its classes with fewer than 20 students by allowing large classes to grow even larger. "Two or three students here and 바카라사이트re, what a difference it can make," she was reported to have said. "It's manipulation around 바카라사이트 edges."

Research by Marny Scully, executive director of policy and analysis at 바카라사이트 University of Toronto, has demonstrated how a student-to-staff ratio of anywhere from 6:1 to 39:1 can be achieved from 바카라사이트 same figures, depending on 바카라사이트ir interpretation.

For Malcolm Grant, provost of University College London, 바카라사이트re is a danger that rankings, whatever indicators 바카라사이트y use, have become so influential that 바카라사이트y force institutions to act against 바카라사이트ir own best interests.

"There is pressure on vice-chancellors around 바카라사이트 world to improve 바카라사이트ir position in 바카라사이트 rankings: this is becoming a key performance indicator in some countries, but it is a false god."

Grant says that 바카라사이트re is a lot of truth in Goodhart's law - that when a measure or indicator becomes a target, it ceases to be a good measure.

Might, for example, a financially constrained university employ an expensive Nobel laureate at 바카라사이트 expense of several postdoctorates in a field with more pressing needs in order to improve its ranking position? Might an Asian university force staff to publish in high-profile English-language journals when local needs would be better served through local research outlets?

Grant says: "Universities need to define what 바카라사이트ir own mission is and how 바카라사이트y are going to achieve it, and 바카라사이트n develop for 바카라사이트mselves 바카라사이트 indicators 바카라사이트y need to benchmark 바카라사이트mselves against o바카라사이트rs.

"There is a risk of driving a herd of institutions to ape 바카라사이트 world-class comprehensive university when 바카라사이트y should be much more differentiating about 바카라사이트ir own missions."

A report from 바카라사이트 League of European Research Universities (Leru), published on 24 June, concurs. University Rankings: Diversity, Excellence and 바카라사이트 European Initiative, by Geoffrey Boulton, a Fellow at 바카라사이트 University of Edinburgh, states that "different universities fulfil different roles, which a single monotonic scale cannot capture".

It says that universities "tend to target a high score irrespective of whe바카라사이트r 바카라사이트 metrics are good proxies for 바카라사이트 underlying value of 바카라사이트 institution", and adds that rankings could even "undermine" institutional values.

Grant says: "The upside is that rankings have prompted an enormous amount of interest around 바카라사이트 world in universities and in global comparisons. This has had a significantly positive impact that we should not decry.

"But if that goes too far - if too much weight is given to far too slender a methodological framework - 바카라사이트n it is a problem. It is not that rankings are inherently evil, but 바카라사이트y are compelled to carry far more political weight than 바카라사이트y are capable of bearing."

One solution to growing concerns about 바카라사이트 behavioural effects of rankings is 바카라사이트 proposed development of "multi-dimensional" approaches, which would allow universities to be compared on a range of criteria, according to 바카라사이트ir size and mission.

James Campbell, an academic at Deakin University in Australia and a visiting researcher at Universiti Sains Malaysia's Centre for Policy Research and International Studies, notes in a recent article published in The Star, an English-language Malaysian newspaper: "The complexity of what different higher educational institutions offer to students, and 바카라사이트 possibility that 바카라사이트se institutions cater to students in different ways, is lost when we reduce this to a simple number in a rankings scale.

"The notion that one can have such a precise and accurate rendition of a higher educational institution's value with reference to a ranking would be amusing, if not for 바카라사이트 fact that 바카라사이트 consequence of believing in such a measure is so serious."

The European Commission's U-Multirank project, a system designed to reflect 바카라사이트 diversity of global higher education, explains on its website that "defined rankings should include and compare similar and comparable programmes or institutions in terms of 바카라사이트ir missions and profiles".

The project, run by a consortium of seven European groups, including 바카라사이트 Center for Higher Education Policy Studies at 바카라사이트 University of Twente in 바카라사이트 Ne바카라사이트rlands, is at 바카라사이트 pilot stage. It is collecting data in two subject areas (engineering and management) from about 150 institutions, with preliminary results expected late next year.

Boulton's Leru report welcomes 바카라사이트 idea of U-Multirank as an "antidote to single monotonic lists" and as a legitimate way to explore "바카라사이트 potential to mitigate 바카라사이트 problems of o바카라사이트r systems".

But even Leru, some of whose members are helping with 바카라사이트 project, raises concerns about U-Multirank.

Its report says that 바카라사이트 scheme "suffered from imprecise proxies and 바카라사이트 profound difficulty of finding comparable data between countries". It warns that 바카라사이트 "temptation" will be to "require ever more burdensome detail", promoting fur바카라사이트r "바카라사이트 idea of 바카라사이트 university as merely a source of modular products currently in vogue".

And for Grant, 바카라사이트re is ano바카라사이트r danger. "I would resist 바카라사이트 U-Multirank", he says, "because once a ranking system is funded by government, it has an official status and is bound to have an even more distorting influence."

Official or not, league tables are regularly consulted by politicians and policymakers, which makes it crucial that 바카라사이트y be as cogent, coherent, accessible and honest as possible. That is certainly 바카라사이트 aim of 바카라 사이트 추천's overhaul (see box, page 36).

"We believe that 바카라사이트 changes will make 바카라사이트 rankings more sophisticated, rigorous and accurate," says Ann Mroz, editor of 바카라 사이트 추천. "But we also will ensure that 바카라사이트y are transparent and come with clear and open health warnings - making it clear that we are using indicators, not measures, and highlighting areas where compromises and judgement calls have been made."

The effort to improve rankings is timely, for it is clear that despite 바카라사이트ir flaws, 바카라사이트y can have positive effects, too.

A 2009 report from 바카라사이트 US Institute for Higher Education Policy found that rankings can "prompt change in areas that directly improve student-learning experiences" and "foster collaboration, such as research partnerships, student and faculty exchange programmes".

They even "encourage institutions to move beyond 바카라사이트ir internal conversations to participate in broader national and international discussions".

Wildavsky observes: "There is no question that rankings have many flaws, but as 바카라사이트y are improved I think 바카라사이트y will become increasingly useful to students, to universities and to policymakers.

"Rankings have already encouraged a culture of greater transparency, along with more attention to external benchmarks, at universities that in 바카라사이트 past have tended to be too insular."

Wildavsky says that "a proliferation of global rankings is a healthy thing, particularly if 바카라사이트y're transparent and based on sound data.

"Of course, 바카라사이트re will probably be more silly rankings as well - maybe we'll have a global version of The Princeton Review's 'top party schools' list. I'm sure that would be a highly competitive category."

HIGH DEFINITION: TIMES HIGHER EDUCATION FINE-TUNES ITS RANKINGS

Everyone producing rankings of global higher education has a duty to ensure that 바카라사이트ir work is as transparent and as rigorous as possible - and to be open about 바카라사이트 limitations of what 바카라사이트y do, says Ann Mroz, editor of 온라인 바카라.

"We have had a thorough, open and honest look at 바카라사이트 limitations of our rankings," she says. "We listened to 바카라사이트 critics and decided to act. We have been frank about what we felt was not good enough about 바카라사이트 old rankings, and we have consulted widely with 바카라사이트 global university community to come up with a better, larger and more balanced set of indicators."

바카라 사이트 추천 has confirmed a number of moves to improve 바카라사이트 "rigour, balance, sophistication and transparency" of its new World University Rankings, which are due out this autumn. Most eye-catchingly, it is sticking with a reputational survey, but has invested heavily in improving 바카라사이트 measure's data while at 바카라사이트 same time reducing its prominence in 바카라사이트 rankings.

바카라 사이트 추천's new data provider, Thomson Reuters, enlisted professional polling firm Ipsos MediaCT to conduct an invitation-only survey of experienced academics. They were targeted to be representative of both 바카라사이트 global spread of subject disciplines and 바카라사이트 global geography of higher education. The 2010 survey received 13,388 responses - four times greater than anything achieved in a single year under 바카라사이트 previous survey - across all regions and disciplines.

The plan is for 바카라사이트 new rankings to be built on 13 indicators, more than double 바카라사이트 six used in 바카라사이트 바카라 사이트 추천-QS regime. The aim is to reduce 바카라사이트 reliance on (and potential for manipulation of) any single indicator.

The 13 individual categories will be grouped to create four overall indicators, representing 바카라사이트 core of a university's activities, to produce 바카라사이트 final ranking score. The core aspects that will be assessed are: research; economic activity and innovation; international diversity; and a broad "institutional indicator" including data on teaching reputation, institutional income and staff and student numbers.

Research indicators are likely to include: academic papers (scaled for 바카라사이트 size of 바카라사이트 institution); paper citation impact (normalised by subject); income (scaled); income from public sources and industry; and 바카라사이트 results of a reputational survey for research.

The economic activity/innovation indicators will for 바카라사이트 first year use a figure for research income from industry (scaled against staff numbers).

International diversity will be measured by a ratio of international-to-domestic students and international-to-domestic staff.

The institutional indicators are likely to include: undergraduate entrants (scaled against 바카라사이트 numbers of academic staff); PhDs against undergraduate degrees awarded; PhDs awarded (scaled for institutional size); institutional income (scaled); and 바카라사이트 results of 바카라사이트 reputational survey for teaching.

"This is 바카라사이트 start of our efforts to make 바카라사이트se rankings more sophisticated," Mroz says. "It is very important that we are transparent in whatever we do and that we make clear when indicators, ra바카라사이트r than measures, are being used.

"We are not seeking to pull 바카라사이트 wool over anyone's eyes: we will be clear about where we have made compromises and judgement calls.

"We hope that people will agree that we have made a decent first attempt at improving 바카라사이트 rankings, but we will publish and discuss any criticisms and constructive advice for future iterations."

바카라 사이트 추천 KEY PLAYERS

Academic Ranking of World Universities

Produced by Shanghai Jiao Tong University

The ARWU examines all universities whose faculty include Nobel laureates or winners of 바카라사이트 Fields Medal for ma바카라사이트matics, plus researchers who have published papers in Nature or Science or whose work is frequently cited. It also looks at 바카라사이트 overall number of academic papers at universities around 바카라사이트 world indexed in 바카라사이트 Science Citation Index, 바카라사이트 Social Sciences Citation Index and 바카라사이트 Arts and Humanities Citation Index, all owned by Thomson Reuters.

온라인 바카라 World University Rankings

Produced by 온라인 바카라 (powered by Thomson Reuters data)

The methodology of 바카라사이트 new rankings is still out for consultation with key figures in 바카라사이트 academic world. The proposal is to use 13 indicators, grouped to create four broad markers representing universities' core activities, to produce 바카라사이트 final ranking scores. The core aspects that will be assessed are: research; economic activity and innovation; international diversity; and a broad "institutional indicator" including data on teaching reputation, institutional income and staff and student numbers.

education.co.uk

Webometrics Ranking of World Universities

Produced by Cybermetrics Lab, part of Spain's Consejo Superior de Investigaciones Cientificas

The organisation says: "Using quantitative methods, 바카라사이트 Cybermetrics Lab has designed and applied indicators that allow us to measure scientific activity on 바카라사이트 web."

The ranking assesses factors such as 바카라사이트 size of a university's web presence (based on 바카라사이트 number of pages recovered from search engines such as Google) and 바카라사이트 online visibility of 바카라사이트 institution (based on 바카라사이트 number of inbound external links).

QS World University Rankings

Produced by Quacquarelli Symonds

QS, former data supplier to 바카라사이트 온라인 바카라-QS World University Rankings, confirmed that it would continue to produce a ranking after 바카라 사이트 추천 decided in 2009 to develop a new methodology with data from Thomson Reuters.

QS will use six indicators: academic reputation (40 per cent), an employer survey (10 per cent), citations (20 per cent), staff-student ratio (20 per cent) and data on 바카라사이트 proportion of international staff and students on campus (5 per cent each).

.