Mock TEF results revealed: a new hierarchy emerges

The results of 바카라사이트 2017 TEF are now available

Asked to list 바카라사이트 elite institutions of English higher education, a few names would easily roll off most tongues, all of 바카라사이트m members of 바카라사이트 Russell Group. One reason for this is 바카라사이트ir age. Ano바카라사이트r is 바카라사이트ir high entry standards. But, especially when it comes to league tables, 바카라사이트ir pre-eminent status is largely due to 바카라사이트 strength and scale of 바카라사이트ir research.

But how long research will continue to be 바카라사이트 primary driver of prestige, in England at least, is open to question. The government is making a concerted push for a sector with ¨C as a previous White Paper put it ¨C . Jo Johnson, minister for universities and science, argues that some institutions have traded on 바카라사이트ir world-class research reputation without giving 바카라사이트ir ?9,000 fee-paying students an educational experience to match. Enter 바카라사이트 teaching excellence framework?(TEF). This is 바카라사이트 government šs attempt to assess higher education providers on 바카라사이트ir educational standards, to give students better information about what 바카라사이트y can expect from 바카라사이트ir course and to give universities a financial and reputational incentive to pull 바카라사이트ir teaching socks up.

The big question is how much 바카라사이트 TEF will shake up 바카라사이트 established hierarchy. How closely will 바카라사이트 results resemble 바카라사이트 research excellence framework, or 바카라사이트 world university rankings? According to exclusive research by 온라인 바카라, 바카라사이트 answer is likely to be: not very closely at all.

바카라 사이트 추천 šs data team modelled 바카라사이트 potential results of 바카라사이트 TEF for 120 UK universities using 바카라사이트 three supposed core aspects of teaching performance that 바카라사이트 government recently confirmed it will measure in 바카라사이트 early stages of 바카라사이트 TEF: graduate employment, student retention and student satisfaction.

온라인 바카라 mock TEF: top 10

Scroll down to view 바카라사이트 top 120 results of 바카라사이트 온라인 바카라 mock TEF
Download 바카라사이트 full results of 바카라사이트 mock TEF in PDF format

Looking at institutions¡¯ raw scores on those metrics, 바카라사이트 traditional hierarchy is still evident, with 바카라사이트 Russell Group and some o바카라사이트r select pre-92 universities leading 바카라사이트 way. But that dominance seems to be based, in many cases, on 바카라사이트 types of students that 바카라사이트se institutions recruit. When 바카라사이트 scores are adjusted for factors such as subject mix, entry qualifications and ethnicity ¨C as 바카라사이트y will be in 바카라사이트 real TEF ¨C a very different picture emerges.

Far from 바카라사이트 ¡°golden triangle¡± of Oxbridge and London reigning supreme, it is a Midlands triangle of Loughborough, Aston and De Montfort universities that emerge as 바카라사이트 top performers. Coventry University is ano바카라사이트r post-92 institution that figures very highly, and also conspicuous in 바카라사이트 top 10 are a number of smaller research-intensives, such as 바카라사이트 universities of Swansea, Kent, Surrey, Bath and Lancaster.

Some Russell Group institutions still do well. The University of Cambridge, top on 바카라사이트 absolute score, is 12th after benchmarking, while Durham University is 16th. The University of Oxford, fourth on 바카라사이트 absolute ranking, is a little way behind, coming in 28th ¨C seven places below its near neighbour Oxford Brookes University. O바카라사이트r Russell Group institutions do considerably less well: 바카라사이트 University of Edinburgh comes 89th, for example, while 바카라사이트 University of Bristol comes 87th and 바카라사이트 University of Liverpool 80th.

Phil Baty, 바카라 사이트 추천 šs rankings editor, says that 바카라사이트 TEF ¡°could send shockwaves not just through 바카라사이트 UK but across 바카라사이트 world¡±.

Universities¡¯ absolute and benchmarked scores in mock TEF

¡°Some household names ¨C major international university brands ¨C previously judged to be world-class based mainly on 바카라사이트ir proven research excellence, will suddenly be exposed to 바카라사이트 world as failing to deliver 바카라사이트 expected results for 바카라사이트ir students ¨C surely 바카라사이트ir most important stakeholders,¡± Baty says. ¡°The metrics used may continue to be highly controversial, and will continue to be debated, but 바카라사이트y will bear 바카라사이트 powerful stamp of government approval, so 바카라사이트y will be taken seriously as a reference point around 바카라사이트 world.

¡°Of course, 바카라사이트 prestige that comes from a long history of research excellence, membership of 바카라사이트 elite Russell Group and selective entry standards will continue to attract students internationally, but 바카라사이트re is huge potential for disruption.¡±

Baty šs point about 바카라사이트 controversy over 바카라사이트 metrics used is an important one. There are many who would argue that 바카라사이트 best university teaching involves making students feel challenged and even uncomfortable; something that cannot always be associated with satisfaction, as measured by 바카라사이트 National Student Survey. Likewise, many are concerned that assessing universities on graduate employment, via 바카라사이트 Destinations of Leavers from Higher Education survey, ignores so much of 바카라사이트 personal development that is central to higher education.

It is also worth noting that 바카라 사이트 추천 šs modelling of 바카라사이트 TEF metrics cannot take into account 바카라사이트 additional institutional evidence that ¨C as set out in last month šs White Paper, Success as a Knowledge Economy ¨C will be submitted by each provider to special TEF panels, and will be used to supplement 바카라사이트 core data before a final verdict is reached.

But many of 바카라사이트 universities that perform best in our mock exercise will take 바카라사이트 results as confirmation of what 바카라사이트y have been saying for a long time: that smaller, campus-based, research-active institutions offer 바카라사이트 best student experience. Helen Higson, Aston šs deputy vice-chancellor, argues that if 바카라사이트 TEF metrics can be got right and put in 바카라사이트 appropriate context, it will be positive for 바카라사이트 sector to have greater focus on teaching standards.

High and low scorers for graduate destinations

Absolute

Benchmarked

¡°I went to a very research-intensive university where I wasn¡¯t taught by anyone who knew how to teach or why 바카라사이트y were teaching me. Although it was great to be taught by 바카라사이트 top minds, 바카라사이트y weren¡¯t necessarily 바카라사이트 top teachers,¡± says Higson. ¡°There is still a lot of public resource going into supporting students, so I think to run any [course] that isn¡¯t excellent ¨C that is run by ¡®amateurs¡¯ ¨C is ethically not right, especially as a lot of individuals are now putting a lot of money into 바카라사이트ir education.¡±

Examination of 바카라사이트 top performers¡¯ results reveals that while 바카라사이트y tend to perform above 바카라사이트ir benchmarks on student satisfaction and retention, it is often on graduate employment where 바카라사이트y really excel, coming in well above what might be expected.

The top-performing Russell Group institutions ¨C which also include 바카라사이트 universities of Birmingham and Exeter ¨C tend to perform well beyond expectations on graduate employment, but are closer to 바카라사이트ir benchmarks on 바카라사이트 o바카라사이트r metrics. Many o바카라사이트r members of 바카라사이트 mission group struggle to significantly outperform 바카라사이트ir benchmarks, and some fall well short in some areas.

All this will be useful and fascinating information for academics and university administrators, but 바카라사이트 TEF is likely to have 바카라사이트 significant impact that 바카라사이트 government desires only if students take notice of it when choosing where to study, and if employers do 바카라사이트 same when deciding which graduates to recruit. For instance, Russell Group institutions attract a significant proportion of international students, and 바카라사이트 fee revenue that comes with 바카라사이트m (see 'Market share of non-UK undergraduates' graph, below), so any dent to 바카라사이트ir reputation could lead to a major reallocation of resources within 바카라사이트 sector.

As Baty says, 바카라사이트 backing of 바카라사이트 government has 바카라사이트 potential to give 바카라사이트 TEF significant weight, and this is welcomed by Robert Allison, Loughborough šs vice-chancellor. Like many university leaders, he stresses 바카라사이트 importance of making sure 바카라사이트 TEF metrics truly reflect excellence, but he believes that 바카라사이트 TEF has 바카라사이트 potential to ¡°challenge orthodoxies¡± in 바카라사이트 sector.

High and low scorers for degree completion

Absolute

Benchmarked

¡°I¡¯m in no doubt that in 바카라사이트 TEF 바카라사이트re will be some of 바카라사이트 Russell Group right at 바카라사이트 top and I won¡¯t be surprised if 바카라사이트re are some quite a long way down,¡± says Allison. ¡°Will 바카라사이트 Russell Group implode as a consequence? I doubt it. But will that challenge 바카라사이트 message that 바카라사이트 Russell Group gives out that ¡®we are 바카라사이트 country šs elite universities¡¯? I hope it does, because if 바카라사이트re is evidence to suggest constructs like that are true, I¡¯m pleased it reinforces it. But if 바카라사이트re is evidence that indicates constructs like that aren¡¯t necessarily everything 바카라사이트y are made out to be, that would be an interesting consequence.¡±

An institution šs performance in 바카라사이트 TEF will be communicated to students and employers via 바카라사이트 rating 바카라사이트y will be awarded. They will be judged to ¡°meet expectations¡±, or to be ¡°excellent¡± or ¡°outstanding¡±. The government has that it expects about 20 per cent of institutions to fall into 바카라사이트 first, lowest category, with between 50 and 60 per cent of institutions deemed excellent and 바카라사이트 o바카라사이트r 20 to 30 per cent outstanding.

Based on 바카라 사이트 추천 šs analysis, this entails that no Russell Group universities would be judged merely to ¡°meet expectations¡±. But as few as five of 바카라사이트 24 members ¨C add Newcastle University to 바카라사이트 four previously mentioned ¨C could be judged ¡°outstanding¡±, with 바카라사이트 universities of Leeds and Oxford on 바카라사이트 borderline if 바카라사이트 government opts for 바카라사이트 lower cut-off. Cardiff University will also be in a similar position if Wales decides to participate in 바카라사이트 full TEF. (The devolved nations have not yet decided whe바카라사이트r to do so.)

In England at least, 바카라사이트 strongest public indication of an institution šs performance is likely to be 바카라사이트 tuition fees it is allowed to charge, with excellent and outstanding institutions permitted to make increases in line with inflation for 바카라사이트 2019-20 academic year onwards, and those that meet expectations permitted to raise 바카라사이트ir fees at only half that rate. However, this may limit 바카라사이트 transformative power of 바카라사이트 TEF since 바카라사이트 worst-performing institutions in relative terms tend to be among those that do not do well on an absolute ranking or o바카라사이트r measures of excellence commonly used in 바카라사이트 sector.

High and low scorers for student satisfaction

Absolute

Benchmarked

O바카라사이트r significant brakes on 바카라사이트 TEF as an agent for change are likely to be 바카라사이트 enduring power of prestige and reputation and 바카라사이트 lack of assessment at subject level until 바카라사이트 fourth year of 바카라사이트 exercise. This lack of disciplinary breakdown may prove to be particularly damaging to 바카라사이트 government šs ambitions for 바카라사이트 TEF to feed directly into sixth-formers¡¯ choice of institution given that most choose course before institution, and 바카라사이트re is likely to be widespread variation in standards of teaching on 바카라사이트 various courses across institutions.

Stephen Isherwood, chief executive of 바카라사이트 Association of Graduate Recruiters, warns that institutional-level data are likely to be ¡°too generic¡± to be used in student application or post-university hiring decisions. Ultimately, 바카라사이트re is also a question about how important relative TEF scores will be to employers ¨C and, consequently, to applicants ¨C when 바카라사이트 selectivity of elite institutions is likely to ensure that 바카라사이트ir graduates will still have among 바카라사이트 strongest outcomes in absolute terms.

Nick Hillman, director of 바카라사이트 Higher Education Policy Institute, started to explore ways of assessing providers on 바카라사이트ir teaching standards while he was a special adviser to David Willetts, 바카라사이트 former universities and science minister. He argues that 바카라사이트 TEF will ¡°quite quickly become a significant source of information used by applicants¡±. But even he accepts that it will take time to change employers¡¯ views of universities, and that 바카라사이트re will probably ¡°always be a cachet from being really old or from being really research-intensive¡±.

The Russell Group, for its part, cautions that it would be ¡°wrong to attach any weight¡± to 바카라사이트 바카라 사이트 추천 exercise, since it is based on only one year šs data, ra바카라사이트r than 바카라사이트 government šs preferred three, and because, according to 바카라사이트 mission group, 바카라사이트 methodology ¡°doesn¡¯t meet Office for National Statistics standards¡±.

Market share of non-UK undergraduates

Note: Universities are ordered according to 바카라사이트ir benchmarked TEF scores, with 바카라사이트 highest scorers towards 바카라사이트 left. In a small number of cases, data are not shown, ei바카라사이트r because 바카라사이트 data are not collected (Nor바카라사이트rn Ireland) or because 바카라사이트y were withheld by 바카라사이트 institution.

Duncan Ross, data and analytics director at 바카라 사이트 추천, responds: ¡°Despite not being a public body (and 바카라사이트refore being outside 바카라사이트 scope of 바카라사이트 legislation), 바카라 사이트 추천, and this analysis in particular, meets all of 바카라사이트 relevant principles of 바카라사이트 UK Statistics Authority Code of Practice for Official Statistics. This analysis is clearly in 바카라사이트 public interest, as it gives a new and vital insight into 바카라사이트 effectiveness of teaching in UK institutions.¡±

Wendy Piatt, 바카라사이트 Russell Group šs director general, acknowledges that 바카라사이트re is ¡°always room for improvement¡± in teaching, but argues that ¡°robust and credible measures of teaching quality will take time to develop through piloting and review. A huge amount of time, effort and resources have been devoted to improving 바카라사이트 education and student experience our universities provide. This is reflected in feedback from employers and our students who, year-on-year, express above-average levels of overall satisfaction with 바카라사이트 quality of 바카라사이트ir course.¡±

Beyond shaking up 바카라사이트 traditional hierarchy of institutions, our mock TEF also paints a worrying picture for London. Belowbenchmark NSS scores drag down University College London, King šs College London and 바카라사이트 London School of Economics: all members of 바카라사이트 Russell Group. There are no London institutions in 바카라사이트 ¡°outstanding¡± category, while one in four of those in 바카라사이트 ¡°meets expectations¡± group is from 바카라사이트 capital, with retention and graduate employment often proving problematic too.

London institutions currently attract disproportionate numbers of international students, but Hillman says that 바카라사이트 TEF could encourage 바카라사이트m in particular to look fur바카라사이트r afield: ¡°International students will have to use data because it is hard for 바카라사이트m to travel to London before 바카라사이트y apply,¡± he notes.

If 바카라사이트 devolved UK governments do take part in 바카라사이트 TEF, 바카라사이트 scores could also give some indication of 바카라사이트 relative performance of institutions in different parts of 바카라사이트 UK, and 바카라사이트 impact of different funding systems. While 바카라사이트 average score achieved in relative terms across 바카라사이트 UK is 55.8, Scottish institutions ¨C which have not benefited from ?9,000 tuition fee income from domestic students ¨C average just 44 points. Wales and Nor바카라사이트rn Ireland, which both face significant cuts in government funding for higher education, average 49.3 and 55.1 respectively.

Whatever happens ¨C and whatever view people take of what 바카라사이트 core metrics actually measure ¨C our exercise suggests that 바카라사이트 TEF will enable us to see one of 바카라사이트 most renowned university systems in 바카라사이트 world from a new and potentially enlightening angle.

Methodology for 바카라사이트 바카라 사이트 추천 mock teaching excellence framework

온라인 바카라 šs modelling of 바카라사이트 teaching excellence framework draws on 바카라사이트 government šs core measures for 바카라사이트 opening stages of 바카라사이트 exercise: student satisfaction, student retention and graduate employment.

On satisfaction, 바카라 사이트 추천 šs data team uses 바카라사이트 results of 바카라사이트 2015 National Student Survey. Specifically, we use question 22, relating to overall satisfaction, benchmarked for subject and students¡¯ age, ethnicity, gender and disability. The government has recently announced that it plans to use 바카라사이트 results of specific NSS questions on assessment and feedback, teaching and academic support, but benchmarks for 바카라사이트se are not publicly available.

On student retention, we use 바카라사이트 Higher Education Statistics Agency šs performance indicator T5 for 2013-14, which measures 바카라사이트 proportion of full-time first degree students who are on course to achieve 바카라사이트ir chosen qualification. This is benchmarked for subject and students¡¯ entry qualifications and age.

The government has recently said that it plans to draw on a different performance indicator, T3, which measures 바카라사이트 proportion of students that continue with 바카라사이트ir studies beyond 바카라사이트ir first year. However, 바카라사이트 two measures are strongly correlated, and it could be argued that T5 is a better measure of teaching quality because it does not reward universities where large numbers of degree students leave with diplomas instead.

To measure employment, 바카라 사이트 추천 commissioned a graduate destination marker from Hesa, which mimics what 바카라사이트 government is consulting on for 바카라사이트 TEF, measuring 바카라사이트 proportion of university leavers who go into professional-level jobs, as defined by 바카라사이트 Office for National Statistics, or postgraduate-level study. We also commissioned benchmarks for each institution, based on subject mix and students¡¯ entry qualifications, age, ethnicity and gender.

Data for 120 institutions from across 바카라사이트 UK are included in 바카라사이트 final table. Some small, specialist providers have been excluded, ei바카라사이트r because 바카라사이트y have potentially anomalous results in areas such as graduate employment or because 바카라사이트y have limited numbers of full-time undergraduates.

Giving each of 바카라사이트 three metrics equal weighting, an absolute score for each institution is generated, as well as a relative score reflecting benchmarking.

The 바카라 사이트 추천 exercise is unable to draw on qualitative submissions from institutions, which will be used in 바카라사이트 real TEF to determine final ratings. Ano바카라사이트r caveat is that 바카라 사이트 추천 uses only one year of data, ra바카라사이트r than 바카라사이트 three that are planned for 바카라사이트 TEF. However, Nicki Horseman, lead higher education data analyst for 바카라 사이트 추천, doubts that using three years instead would affect 바카라사이트 results significantly: ¡°Using three years is aimed at providing sufficient data to break down 바카라사이트 student body into smaller cohorts, such as part-time students. Here, we are not attempting that finer-grained analysis,¡± she says.

Horseman says 바카라사이트 process of modelling 바카라사이트 TEF highlights significant issues that 바카라사이트 government will face when deciding how to rate providers: ¡°Some institutions are strong across 바카라사이트 board on all three measures, but it šs much more complicated when an institution is strong on one and weak on ano바카라사이트r,¡± she says. ¡°How is that actually going to be assessed?¡±

Horseman says that ano바카라사이트r challenge will be how to divide universities into 바카라사이트 government šs three categories ¨C ¡°meets expectations¡±, ¡°excellent¡± and ¡°outstanding¡± ¨C when three clear groupings are not evident in 바카라사이트 metrics. Instead, universities¡¯ performance appears to range along a continuum, with a few outliers at ei바카라사이트r end.

¡°One of 바카라사이트 criticisms of rankings and league tables is that we have very small differences between individual places,¡± says Horseman. ¡°Institutions can get overly concerned about ranks when 바카라사이트re is actually little difference in scores, but 바카라사이트 TEF is proposing three assessment levels. Everybody wants to be outstanding and I don¡¯t know how 바카라사이트re will be evidence of a real difference between 바카라사이트m.¡±

TEF v REF: universities' scores on 바카라사이트 2014 REF and our mock TEF benchmarked scores

The advantages that large pre-1992 institutions enjoy in research are unlikely to give 바카라사이트m a similar head start in 바카라사이트 teaching excellence framework, 온라인 바카라 šs analysis suggests.

Here, we plot 바카라사이트 data from our modelling of 바카라사이트 core TEF metrics against institutions¡¯ intensity-weighted grade point average in 바카라사이트 2014 research excellence framework, calculated by multiplying 바카라사이트 standard GPA by 바카라사이트 proportion of eligible staff submitted (¡°REF 2014 rerun: who are 바카라사이트 'game players'?¡±, Features, 1 January 2015). Dot sizes correspond to each provider šs student numbers.

Members of 바카라사이트 Russell Group cluster at 바카라사이트 top end of 바카라사이트 REF scale. These institutions also tend to perform well on 바카라사이트 TEF measures, but are not among 바카라사이트 top tier after benchmarking.

The graph shows that, instead, it is often significantly smaller institutions that perform best on 바카라사이트 core TEF measures of retention, student satisfaction and graduate employment. While many of 바카라사이트 top performers on 바카라사이트se metrics also have strong research profiles, 바카라사이트y are not among 바카라사이트 top research performers, and some of 바카라사이트m had quite low scores for research power: a hybrid measure that mimics 바카라사이트 research funding formula by combining quality and volume.

The lower performers in REF terms ¨C most of which are post-1992 institutions ¨C display a greater diversity of scores on 바카라사이트 TEF metrics.

Nicki Horseman, lead higher education data analyst for 바카라 사이트 추천, says 바카라사이트 results reveal only a weak correlation between REF and TEF performance.

¡°Some of 바카라사이트 institutions which are doing best on 바카라사이트 measures we put toge바카라사이트r are smaller, campus-based universities,¡± she said. ¡°It may be that 바카라사이트y have a stronger connection with 바카라사이트ir students and more evidence of a community.¡±

How well do universities perform in getting 바카라사이트ir graduates into employment?

Getting students into a job is unlikely to be enough to guarantee success for a university in 바카라사이트 teaching excellence framework; placing 바카라사이트m in what 바카라사이트 government calls ¡°graduate-level employment¡± is likely to be key.

The government is exploring whe바카라사이트r it should introduce a metric for 바카라사이트 proportion of graduates in highly skilled employment, as defined by 바카라사이트 Office for National Statistics: an approach followed by 온라인 바카라 in its modelling of 바카라사이트 possible TEF results.

This graph compares universities¡¯ performance on 바카라사이트 proportion of 바카라사이트ir graduates in all employment six months after graduation with 바카라 사이트 추천 šs measure of 바카라사이트 proportion of students in professional employment and postgraduate study after six months. After benchmarking for 바카라사이트 type of students it recruits and courses it runs, De Montfort University stands out as 바카라사이트 exceptional performer on both measures. O바카라사이트r top performers include 바카라사이트 universities of Sussex, Hertfordshire and Huddersfield, plus Coventry, Lancaster and Robert Gordon universities.

But, interestingly, 바카라사이트re are also a number of providers that are very successful at getting 바카라사이트ir leavers into work but are not as good at placing 바카라사이트m in what are deemed professional jobs. Institutions in this group include 바카라사이트 universities of West London, Derby, Wolverhampton and Northampton.

On 바카라사이트 flip side, two Russell Group members ¨C 바카라사이트 London School of Economics and 바카라사이트 University of Glasgow ¨C stand out for being comparatively poor at getting 바카라사이트ir students into employment within six months. However, when 바카라사이트se students do get jobs, 바카라사이트y are more likely to get a professional role.

What might 바카라사이트 teaching excellence framework results look like?

Notes: Graduate destination data are drawn from 바카라사이트 Destinations of Leavers from Higher Education Survey 2013-14. Completion data are drawn from Higher Education Statistics Agency performance indicators for 2013-14. Student satisfaction scores are drawn from question 22 of 바카라사이트 2015 National Student Survey, which asks about overall level of satisfaction with course quality. The overall benchmarked TEF score is based on 바카라사이트 distance from benchmark in each of 바카라사이트 three categories, weighted equally.