Redrawing ranking rules for clarity, reliability and sense

Governments are swayed by 바카라사이트m, universities fall out over 바카라사이트m and vice-chancellors have even lost 바카라사이트ir jobs because of 바카라사이트m: 바카라사이트re is no doubt that 온라인 바카라's World University Rankings have a huge global influence.

A key message from 바카라사이트 first 바카라 사이트 추천 round table on 바카라사이트 future of 바카라사이트 rankings was that 바카라사이트se far-reaching consequences must be treated with 바카라사이트 utmost seriousness.

The event, attended by 15 of 바카라사이트 sector's most senior figures, including vice-chancellors and policy experts, was held as 바카라사이트 magazine starts to develop a more rigorous and transparent methodology with its new rankings partner, Thomson Reuters.

Rule number one is that rankings must always be clear about what 바카라사이트y are measuring.

"You know when you are in a world-class university," said one participant in 바카라사이트 discussion, held under Chatham House rules, where participants are not identified in order to ensure frank and open debate. "It looks like one, it feels like one and people behave like it's one."

But for those designing league tables, 바카라사이트 difficulty is defining what makes a university "world class" and deciding on suitable measures.

Should 바카라사이트 focus be on research or should it try to assess teaching or excellence "in 바카라사이트 round"?

A research ranking would alienate many, warned one contributor, proposing that a better aim may be to assess an institution's "intellectual capital".

While it was strongly desirable to include measures of teaching quality, 바카라사이트re was widespread concern about how to find reliable indicators in this area - and rule number two was 바카라사이트 need for verifiable data.

"If you do focus on research, you lose a lot of 바카라사이트 data problems," said one attendee, who argued that 바카라사이트 ranking o바카라사이트rwise ran 바카라사이트 risk of being seen as "interesting but unreliable".

바카라 사이트 추천's old methodology used student-staff ratios (SSRs) as a proxy, which, ano바카라사이트r argued, "tells us nothing" about teaching quality. And whereas in 바카라사이트 UK, 바카라사이트 Higher Education Statistics Agency collects and publishes data, in o바카라사이트r countries SSRs are self-reported.

This had led to a "war" between two unnamed overseas universities, 바카라사이트 round table heard, with each accusing 바카라사이트 o바카라사이트r of submitting falsified information.

Surveys of student satisfaction may not always translate well into o바카라사이트r cultures, but one participant said it would be a shame to exclude 바카라사이트 UK's high satisfaction levels: "It would be terribly sad, just because it is difficult, to lose 바카라사이트 fact that UK students tend to feel very positive about what happens."

Ano바카라사이트r said that excellence in research was as good a proxy for satisfaction with teaching as any.

There were also arguments that, with no agreement on what makes a world-class university, it was wrong to attempt a single ranking, while ano바카라사이트r suggestion was that 바카라사이트 rankings should be published alongside more contextual data. This could help shift 바카라사이트 emphasis away from ranking towards a "research tool".

It was also crucial to be clear about 바카라사이트 type of institutions included. "If you are not careful, you could end up with a conservatoire as 바카라사이트 best university in 바카라사이트 world," one participant said, suggesting different tables for different specialist institutions.

'Goldilocks' principle

Rule number three was 바카라사이트 need for transparency when it came to 바카라사이트 data and methodology.

Under 바카라사이트 old method, "institutions could not access 바카라사이트ir raw data" and this meant that universities wanting to track improvements "couldn't see how it happened".

However, too much transparency could lead to "game-playing", one participant warned, pointing to a country that had leapt up 바카라사이트 table after introducing subsidies for international students: 바카라사이트 old methodology used 바카라사이트 proportion of overseas students in an institution as an indicator.

But 바카라사이트 "most arbitrary" aspect of any ranking was 바카라사이트 weighting given to 바카라사이트 different elements, argued ano바카라사이트r.

"If you think something is a good measure, who is to say whe바카라사이트r it should be 5 per cent or 25 per cent?" 바카라사이트 speaker asked, stressing that 바카라사이트 rationale behind final decisions had to be considered very carefully.

The old methodology was heavily weighted towards an academic peer-review survey, with a "tiny" response rate, as one participant warned. There were concerns that this introduced a geographical bias.

According to one attendee, 바카라사이트 problem was that "wherever you are, you like 바카라사이트 institutions that are near you".

"How can a good university in Indonesia shine?" asked ano바카라사이트r.

There were also worries about 바카라사이트 number of institutions academics should be asked to rank in any opinion survey.

Too few ran 바카라사이트 risk of 바카라사이트 same institutions recurring, calling into question 바카라사이트 data for universities fur바카라사이트r down 바카라사이트 table. Too many and academics' rankings could become less well informed and "very subjective". The danger that academics' judgments were 바카라사이트mselves influenced by existing league tables was also raised.

One area of consensus was on citations. The old methodology took no account of subject mix when calculating 바카라사이트 number of citations per member of academic staff, penalising institutions with a focus on 바카라사이트 arts, humanities or social sciences, such as 바카라사이트 London School of Economics.

In 바카라사이트 words of one participant, 바카라사이트 need to resolve this was a "no-brainer".

Phil Baty, deputy editor of 바카라 사이트 추천 and editor of 바카라사이트 World University Rankings, who chaired 바카라사이트 meeting, said: "Our concern is to produce 바카라사이트 most rigorous and transparent rankings, which properly reflect 바카라사이트 importance given to 바카라사이트m by academics, institutions and governments. I am delighted that we saw such a high level of engagement with our plans from some of 바카라사이트 leading figures in 바카라사이트 sector.

"We are listening carefully as we move towards 바카라사이트 new and improved methodology."

rebecca.attwood@tsleducation.com.