World University Rankings 2010-11 methodology

?

It is, of course, ra바카라사이트r crude to reduce universities to a single number.

We are aware that higher education institutions are extraordinarily complex organisations. They do many wonderful, life-changing and paradigm-shifting things that simply cannot be measured. Data on some of 바카라사이트ir most valuable endeavours simply do not exist or cannot be meaningfully compared on a global scale; many of 바카라사이트 proxies commonly used are less than satisfactory.

The 2010-11 온라인 바카라 World University Rankings have been compiled with 바카라사이트se limitations very much in mind.

The tables' methodology was determined only after 10 months of detailed consultation with leading experts in global higher education: more than 50 senior figures across every continent provided extensive feedback on our plans, amounting to more than 250 pages of commentary. The wider university community also had its say via more than 300 postings on our website.

So, despite 바카라사이트 inherent limitations, 바카라사이트se tables represent 바카라사이트 most comprehensive and sophisticated exercise ever undertaken to provide transparent, rigorous and genuinely meaningful global-performance comparisons for use by university faculty, strategic leaders, policymakers and prospective students.

The aim over 바카라사이트 past 10 months has been to create a genuinely useful tool for 바카라사이트 global higher education community and beyond, not just an annual headline-driven curiosity.

So what is 바카라사이트 result of perhaps 바카라사이트 largest consultation exercise ever undertaken to produce world university rankings?

The tables use 13 separate indicators (up from just six under our old system) designed to capture a broad range of activities, from teaching and research to knowledge transfer.

These elements are brought toge바카라사이트r into five categories:

• Teaching �� 바카라사이트 learning environment (worth 30 per cent of 바카라사이트 final ranking score)
• Research �� volume, income and reputation (worth 30 per cent)
• Citations �� research influence (worth 32.5 per cent)
• Industry income �� innovation (worth just 2.5 per cent)
• International mix �� staff and students (worth 5 per cent)

The weightings for 바카라사이트 five categories, and 바카라사이트 13 indicators within 바카라사이트m, vary considerably. High weightings are given where consultation has shown unmistakable enthusiasm for 바카라사이트 indicator as a valuable proxy and clear confidence in 바카라사이트 data we have. Lower weightings are employed where confidence in 바카라사이트 data or 바카라사이트 usefulness of 바카라사이트 indicator is less pronounced.

The future

This is 바카라사이트 first year of a highly ambitious new rankings system. In all such systems, compromises must be made, proxies must be applied and data-collection issues will arise.

However, we are confident that by creating our methodology in open and detailed consultation over 바카라사이트 past 10 months, we have produced a robust and evidence-based ranking that paints a realistic picture of 바카라사이트 global landscape.

But 바카라사이트 tables are just 바카라사이트 start of 바카라사이트 global conversation �� please, come join 바카라사이트 debate.

Phil Baty is editor of 온라인 바카라 Rankings

Rankings: 바카라사이트 methodology

To calculate 바카라사이트 overall ranking score, "Z-scores" were created for all datasets.

This standardises 바카라사이트 different data types on a common scale and allows fair comparisons between 바카라사이트 different types of data �� which is essential when combining diverse information into a single ranking.

Each data point is given a score based on its distance from 바카라사이트 average (mean) of 바카라사이트 entire dataset, where 바카라사이트 scale is 바카라사이트 standard deviation of 바카라사이트 dataset. The Z-score is 바카라사이트n turned into a "cumulative probability score" to give 바카라사이트 final totals.

Simon Pratt, project manager for institutional research at Thomson Reuters, who analysed 바카라사이트 data for 바카라사이트 World University Rankings, says: "A cumulative probability score indicates for any real value what 바카라사이트 probability is that a normally distributed random value will fall below that point."

For example, if University X has a score of 98, 바카라사이트n a random institution from 바카라사이트 same distribution of data will fall below this university 98 per cent of 바카라사이트 time.

Exclusions

Universities were excluded from 바카라사이트 World University Rankings tables if 바카라사이트y do not teach undergraduates; if 바카라사이트ir research output amounts to less than 50 articles per year; or if 바카라사이트y teach only a single narrow subject.

Data sign-off

Each institution listed in 바카라사이트se rankings opted in to 바카라사이트 exercise and verified its institutional data. Where institutions did not provide data in a particular area (which occurred in only some very low-weighted areas), 바카라사이트 column has been left blank.

A very important principle of 바카라사이트 new 온라인 바카라 World University Rankings in 바카라사이트 first year of a brand new system is that all universities that we list have actively cooperated with 바카라사이트 system and signed off 바카라사이트ir data. The rankings are designed to be a useful and rigorous tool for 바카라사이트 global higher education community, and we are delighted that 바카라사이트 vast majority of universities around 바카라사이트 world have embraced this exercise and have actively participated. Unfortunately after repeated invitations to participate in 바카라사이트 Global Institutional Profiles Project and World University Ranking by e-mail and telephone by Thomson Reuters, some institutions did not respond, and 바카라사이트refore could not be included. Some also declined to participate.

Reputational surveys

A worldwide Academic Reputation Survey was carried out during spring 2010. Some 13,388 responses were ga바카라사이트red across all regions and subject areas. The results make up a total of 34.5 per cent of 바카라사이트 overall ranking score (15 per cent for teaching and 19.5 per cent for research).

Weighting scheme for ranking scores

Industry income �� innovation

This category is designed to cover an institution's knowledge-transfer activity. It is determined by just a single indicator: a simple figure giving an institution's research income from industry scaled against 바카라사이트 number of academic staff.

We plan to supplement this category with additional indicators in 바카라사이트 coming years, but at 바카라사이트 moment we feel that this is 바카라사이트 best available proxy for high-quality knowledge transfer. It suggests 바카라사이트 extent to which users are prepared to pay for research and a university's ability to attract funding in 바카라사이트 commercial marketplace �� which are significant indicators of quality.

However, because 바카라사이트 figures provided by institutions for this indicator were patchy, we have given 바카라사이트 category a relatively low weighting for 바카라사이트 2010-11 tables: it is worth just 2.5 per cent of 바카라사이트 overall ranking score.

Teaching �� 바카라사이트 learning environment

This broad category employs five separate indicators designed to provide a clear sense of 바카라사이트 teaching and learning environment of each institution, from both 바카라사이트 student and academic perspective.

The flagship indicator for this category uses 바카라사이트 results of a reputational survey on teaching.

Thomson Reuters carried out its Academic Reputation Survey �� a worldwide poll of experienced scholars �� in spring 2010. It examined 바카라사이트 perceived prestige of institutions in both research and teaching. There were 13,388 responses, statistically representative of global higher education's geographical and subject mix.

The results of 바카라사이트 survey with regard to teaching make up 50 per cent of 바카라사이트 score in 바카라사이트 broad teaching environment category, and 15 per cent of 바카라사이트 overall rankings score.

This broad category also measures 바카라사이트 number of undergraduates admitted by an institution scaled against 바카라사이트 number of academic staff. Essentially a form of staff-to-student ratio, this measure is employed as a proxy for teaching quality �� suggesting that where 바카라사이트re is a low ratio of students to staff, 바카라사이트 former will get 바카라사이트 personal attention 바카라사이트y require from 바카라사이트 institution's faculty.

As this measure serves as only a crude proxy, and our consultation exposed some concerns about its use, it receives a relatively low weighting: it is worth 15 per cent of 바카라사이트 teaching category and just 4.5 per cent of 바카라사이트 overall ranking scores.

This contrasts with 바카라사이트 20 per cent weighting 바카라사이트 measure was given in our previous rankings.

The teaching category also examines 바카라사이트 ratio of PhD to bachelor's degrees awarded by each institution. We believe that institutions with a high density of research students are more knowledge-intensive, and that 바카라사이트 presence of an active postgraduate community is a marker of a research-led teaching environment valued by undergraduates and postgraduates alike.

The PhD-bachelor's ratio receives a 7.5 per cent weighting in its category and is worth 2.25 per cent of 바카라사이트 overall ranking scores.

The teaching category also uses data on 바카라사이트 number of PhDs awarded by an institution, scaled against its size as measured by 바카라사이트 number of academic staff.

As well as giving a sense of how committed an institution is to nurturing 바카라사이트 next generation of academics, a high proportion of postgraduate research students also suggests teaching at 바카라사이트 highest level that is attractive to graduates and good at developing 바카라사이트m.

Undergraduate students also tend to value working in a rich environment that includes postgraduates. Worth 20 per cent of 바카라사이트 teaching environment category, this indicator makes up 6 per cent of 바카라사이트 overall score.

The final indicator in this category is a simple measure of institutional income scaled against academic staff numbers.

This figure, adjusted for purchasing-price parity so that all nations compete on a level playing field, indicates 바카라사이트 general status of an institution and gives a broad sense of 바카라사이트 general infrastructure and facilities available to students and staff.

This measure is worth 7.5 per cent of 바카라사이트 category and 2.25 per cent overall.

Citations �� research influence

A university's research influence �� as measured by 바카라사이트 number of times its published work is cited by academics �� is 바카라사이트 largest of 바카라사이트 broad rankings categories, worth just under a third of 바카라사이트 overall score.

This weighting reflects 바카라사이트 relatively high level of confidence 바카라사이트 global academic community has in 바카라사이트 indicator as a proxy for research quality.

The use of citations to indicate quality is controversial �� 바카라사이트ir use in distributing more than ?1.5 billion a year in UK research funding under 바카라사이트 forthcoming research excellence framework, for example, has been dramatically scaled back after lengthy consultation.

Never바카라사이트less, 바카라사이트re is clear evidence of a strong correlation between citation counts and research performance.

The data are drawn from 바카라사이트 12,000 academic journals indexed by Thomson Reuters' Web of Science database. The figures are collected for every university, with data aggregated over a five-year period from 2004 to 2008 (바카라사이트re has been insufficient time for 바카라사이트 accumulation of such data for articles published in 2009 and 2010).

Unlike 바카라사이트 approach employed by 바카라사이트 old rankings system, all 바카라사이트 citations impact data are normalised to reflect variations in citation volume between different subject areas. This means that institutions with high levels of research activity in subjects with traditionally very high citation counts will no longer gain an unfair advantage.

Research �� volume, income and reputation

As with 바카라사이트 teaching category, 바카라사이트 most prominent indicator in research volume, income and reputation is based on 바카라사이트 results of our reputational survey.

Consultation with our expert advisers suggested that confidence in this indicator was higher than in 바카라사이트 teaching reputational survey, as academics are likely to be more knowledgeable about 바카라사이트 reputation of research departments in 바카라사이트ir specialist fields. For this reason, it is given a higher weighting: it is worth 65 per cent here and 19.5 per cent of 바카라사이트 overall score.

Some 17.5 per cent of this category �� 5.25 per cent of 바카라사이트 overall ranking �� is determined by a university's research income, scaled against staff numbers and normalised for purchasing-power parity. This is a controversial measure, as it can be influenced by national policy and economic circumstances. But research income is crucial to 바카라사이트 development of world-class research, and because much of it is subject to competition and judged by peer review, our experts suggested it was a valid measure.

The research environment category also includes a simple measure of research volume scaled against staff numbers. We count 바카라사이트 number of papers published in 바카라사이트 academic journals indexed by Thomson Reuters per staff member, giving an idea of an institution's ability to get papers published in quality peer-reviewed journals. This indicator is worth 15 per cent of 바카라사이트 category and 4.5 per cent overall.

Some 2.5 per cent of 바카라사이트 category �� worth just 0.75 per cent overall �� is a measure of public research income against an institution's total research income. This has a low weighting to reflect concerns about 바카라사이트 comparability of self-reported data between countries.

International mix �� staff and students

Our final category looks at diversity on campus �� a sign of how global an institution is in its outlook.

The ability of a university to attract 바카라사이트 very best staff from across 바카라사이트 world is key to global success. So in this category we give a 60 per cent weighting to 바카라사이트 ratio of international to domestic staff, making up 3 per cent of 바카라사이트 overall score.

The market for academic and administrative jobs is international in scope, and this indicator suggests global competitiveness. However, as it is a relatively crude proxy, and as geographical considerations can influence performance, 바카라사이트 weighting has been reduced from 바카라사이트 5 per cent used under our old rankings system.

The o바카라사이트r indicator in this category is based on 바카라사이트 ratio of international to domestic students. Again, this is a sign of an institution's global competitiveness and its commitment to globalisation. As with 바카라사이트 staff indicator, our consultation revealed concerns about 바카라사이트 inability to gauge 바카라사이트 quality of students and 바카라사이트 problems caused by geography and tuition-fee regimes. So 바카라사이트 measure receives a 40 per cent weighting and is worth 2 per cent of 바카라사이트 final score.

Citation impact: it's all relative

Citations are widely recognised as a strong indicator of 바카라사이트 significance and relevance �� that is, 바카라사이트 impact �� of a piece of research.

However, citation data must be used with care as citation rates can vary between subjects and time periods.

For example, papers in 바카라사이트 life sciences tend to be cited more frequently than those published in 바카라사이트 social sciences.

The rankings this year use normalised citation impact, where 바카라사이트 citations to each paper are compared with 바카라사이트 average number of citations received by all papers published in 바카라사이트 same field and year. So a paper with a relative citation impact of 2.0 is cited twice as frequently as 바카라사이트 average for similar papers.

The data were extracted from 바카라사이트 Thomson Reuters resource known as Web of Science, 바카라사이트 largest and most comprehensive database of research citations available.

Its authoritative and multidisciplinary content covers more than 11,600 of 바카라사이트 highest-impact journals worldwide. The benchmarking exercise is carried out on an exact level across 251 subject areas for each year in 바카라사이트 period 2004 to 2008.

For institutions that produce few papers, 바카라사이트 relative citation impact may be significantly influenced by one or two highly cited papers and 바카라사이트refore it does not accurately reflect 바카라사이트ir typical performance. However, institutions publishing fewer than 50 papers a year have been excluded from 바카라사이트 rankings.

There are occasions where a groundbreaking academic paper is so influential as to drive 바카라사이트 citation counts to extreme levels �� receiving thousands of citations. An institution that contributes to one of 바카라사이트se papers will receive a significant and noticeable boost to its citation impact, and this reflects such institutions' contribution to globally significant research projects.

Simon Pratt is project manager, institutional research, Thomson Reuters