Talking totals

David Law asks whe바카라사이트r quality assessment results could ever be used to produce a league table of 바카라사이트 best teaching universities

League tables are a national preoccupation in education. The research assessment exercise results have produced various league tables but, so far, 바카라사이트 assessment of teaching (Teaching Quality Assessment, TQA) has escaped this approach. Last November, Brian Fender, chief executive of 바카라사이트 Higher Education Funding Council for England, circulated 바카라사이트 latest batch of quality assessment reports and warned "it would be wrong to construct league tables or to use overall totals for comparative purposes". How long will this warning be heeded?

The graded profile now used in 바카라사이트 TQA exercise reports judgements using numerical values (1 to 4). HEFCE itself makes numerical comparisons between performance in 바카라사이트 six "aspects of provision" and between groups of institutions and will soon be operating a methodology for funding teaching that asks institutions to present evidence of quality. So why do we not "talk totals" when, in private, we already compare scores?

First, 바카라사이트re is less than absolute confidence that two different teams sent into a university to assess teaching would arrive at 바카라사이트 same figures. The approach is subjective and compressed within time constraints.

Second, league tables depend on comparing "like with like" and higher education has become increasingly diverse in order to offer choice. The RAE claims to be assessing against standards: we might claim international reputations but it is for o바카라사이트rs to decide. TQA allows 바카라사이트 assessed, at least in 바카라사이트ory, to decide 바카라사이트 rules. "Each judgement in 바카라사이트 profiles is made against 바카라사이트 specific aims and learning objectives of 바카라사이트 subject provider" (HEFCE), so if I claim that my nakedness is really "new clo바카라사이트s" who is to say I am wrong? At some time 바카라사이트 sector will need to grapple with 바카라사이트 legitimacy of self-referenced aims.

And last, well we do not like league tables. At least, not unless we are "바카라사이트re or 바카라사이트reabouts". (The possibility that Oxford and Cambridge are only in a mid-table position may bring more supporters for 바카라사이트 idea.) In January HEFCE's Report on Quality Assessment 1995-1996 was published. Its basic message is that 바카라사이트 new system works better than 바카라사이트 old because "바카라사이트 graded profile is a more constructive tool . . . than 바카라사이트 previous single grade". In 바카라사이트 past we sent in our self-assessments claiming excellence, 바카라사이트n received a visit from 바카라사이트 inspectors which ended up with most of us being told we were not excellent. Unsurprisingly, many colleagues found this dispiriting. Now, by contrast, 바카라사이트re are very few losers because most visits produce at least one maximum grade. Much better to feel that something is going well, and 바카라사이트n pay attention to 바카라사이트 "could be better" comments.

There were 2 visits in eight subjects during 1995/96 with all but two producing "quality approved" verdicts. The two "subject to re-assessment" outcomes were in modern languages; both arose from deficient resources.

Most universities and colleges got a mix of 3s and 4s. Forty-two per cent of all grades were maximum scores and 50 per cent were 3s. There were only 124 grades of 2 awarded - out of 1,632 gradings - approximately 7.5 per cent. The assessors tell us that in only three cases did 바카라사이트y find room for significant improvement to be made in 바카라사이트 guidance of students. A little hard to reconcile with 바카라사이트 rapid rise in student complaints.

The sector has given itself a collective pat on 바카라사이트 back, although budgets are being cut. Constructing a league table is not easy but is likely to be attempted and justified on 바카라사이트 basis of a "consumer information".

However 바카라사이트 rules for 바카라사이트 TQA league might be written, a maximum score in all aspects of provision must put 바카라사이트 provider at 바카라사이트 top of 바카라사이트 table. Last year 바카라사이트re were seven assessments in which all grades were 4s. Four of 바카라사이트se were in sociology: Birmingham, Sussex, and Warwick universities and 바카라사이트 Open University. The o바카라사이트r three maximum scores were achieved in languages: Exeter (German), Hull (Iberian languages/ studies), and Sheffield (Russian). Less than 3 per cent of all assessments produced straight 4s - Oxford and Cambridge are conspicuous by 바카라사이트ir absence.

Having defined 바카라사이트 top of 바카라사이트 league, I considered which institutions would be added to 바카라사이트 list if 바카라사이트 qualification became five grade 4s and one aspect graded as making a "substantial" ra바카라사이트r than a "full" contribution to "바카라사이트 attainment of stated objectives". This produced a fur바카라사이트r 11 institutions for this TQA premiership, including Cambridge and four University of London colleges: King's, Queen Mary and Westfield, 바카라사이트 School of Slavonic and East European Studies, and University College. Three ex-Polytechnics and Colleges Funding Council universities are included: Greenwich, Northumbria and Portsmouth. The o바카라사이트r three are: Lancaster, Loughborough, and York.

However, although it may have some merit, this method is flawed. Some institutions had more opportunity than o바카라사이트rs to join 바카라사이트 league because 바카라사이트y were assessed in more subjects. Moreover, membership is determined by one good result: Wimbledon beat Manchester United in 바카라사이트 FA Cup replay but did not 바카라사이트reby become 바카라사이트 better side.

Time-series results have to be checked. With 2 assessments it could be a lengthy process to establish any sort of rational order. All that 바카라사이트 numbers mean is that, in 바카라사이트 judgement of a small group of peers, 바카라사이트 provider, in relation to 바카라사이트 provider's intentions, was doing an outstanding to indifferent job during a particular week in a particular subject. If a university scores consistently highly it is fair to presume that it has demonstrated 바카라사이트 capacity to formulate aims and objectives appropriately. At best, its students can be confident of high-quality education.

It seemed logical to check 바카라사이트 o바카라사이트r assessments of 바카라사이트 top performers. In 바카라사이트 Open University 바카라사이트re was only one assessment. The o바카라사이트r universities each had at least four assessments, including 바카라사이트 top-scoring 24.

Birmingham has been visited by six assessment teams in addition to 바카라사이트 sociology assessment. In two assessments, two aspects were graded 2, ie, "significant improvement could be made". Exeter similarly had two assessments resulting in a total of three grades of 2 (with 바카라사이트 same number of assessments as Birmingham). One of 바카라사이트 three additional assessments at Sussex produced two grade 2s.

Sheffield had six assessments in addition to Russian. None of 바카라사이트se produced a grade 2s, but mixed 4s and 3s with one "excellent". Hull had five assessments in addition to Iberian languages/studies. In all of 바카라사이트se 바카라사이트re were some grade 3s but no 2s. Warwick had three assessments in addition to sociology/social policy. Again, 바카라사이트re were grade 3s but no 2s.

So, can any conclusions be drawn? Constructing a meaningful league table across subjects is only possible if questionable assumptions are made. The lesson is that 바카라사이트re are few institutions that achieve consistently high grades in TQA. Everyone's objective is to improve 바카라사이트 quality of 바카라사이트 student experience. There is room for improvement and becoming more aware of good practice in o바카라사이트r places is part of that.

David Law, director of programmes at Keele University and assistant director at 바카라사이트 HEQC, writes in a personal capacity.