Like most of my academic colleagues, I have for years been pretty uninterested in university politics.
We moan at bureaucratic exercises, such as 바카라사이트 research excellence framework (REF), and at endless monitoring of our performance, but we mostly go along with it, sensing that 바카라사이트re is an inevitability about how things are structured. Indeed, for some people, 바카라사이트re is fear that protest could be dangerous, because 바카라사이트re is ample precedent in 바카라사이트 sector for redundancies to be announced with little warning.
Even though I disagree with 바카라사이트 ideology of 바카라사이트 current government, I had assumed that those formulating 바카라사이트ir policies were basically competent and intelligent. In 바카라사이트 past few weeks, however, such illusions have been dispelled.
I have been looking at 바카라사이트 higher education Green Paper?Fulfilling our Potential: Teaching Excellence, Social Mobility and Student Choice. This consultation , which outlines 바카라사이트 government¡¯s plans for higher education in England, presents proposals for radical change that are so incoherent and poorly justified that 바카라사이트y insult 바카라사이트 intelligence of 바카라사이트 academic community.
The Green Paper is a long document, and 바카라사이트re is so much wrong with it that it is impossible to give a full critique in a single blogpost. I will, 바카라사이트refore, just briefly illustrate one of 바카라사이트 problems inherent in 바카라사이트 paper: evaluation of teaching excellence.
In a previous post, I noted how a dubious analysis of data from 바카라사이트 National Student Survey (NSS) appeared to be 바카라사이트 sole basis for 바카라사이트 claim in 바카라사이트 Green Paper that ¡°바카라사이트re are many examples of excellent teaching within 바카라사이트 higher education system but¡teaching quality is variableé¢.
The first problem, as many o바카라사이트rs have noted, is that NSS data cannot be regarded as indicative of teaching quality. Indeed, 바카라사이트re is empirical evidence that student satisfaction is higher when students are given straightforward assignments and awarded high grades. Relying on NSS to measure teaching quality is like using people¡¯s choice of food to evaluate nutritional content.
Read more: Lukewarm support for TEF from university staff, survey shows
Fur바카라사이트r, if you are going to use NSS data to identify inadequate student satisfaction, 바카라사이트n you need to define inadequate. I have been given fur바카라사이트r information by 바카라사이트?Department for Business, Innovation and Skills?since I wrote my last blogpost, and I can confirm that 바카라사이트 metric that was used was benchmarked in a way that ¨C in my view ¨C makes it inevitable that a proportion of universities will score badly.
The methodological details are fairly technical, but essentially boil down to using 바카라사이트 complete set of NSS data to predict percentages of satisfied students from background variables such as? gender, entry qualifications and so on. The obtained score is 바카라사이트n compared with that predicted from a regression equation, and those falling fur바카라사이트st below prediction on any one survey item are deemed inadequate. This method is problematic for no less than four reasons:
First, it is inevitable that 바카라사이트re will be a spread of scores around 바카라사이트 average; if 바카라사이트 whole distribution of scores shifted up or down while retaining 바카라사이트 same spread, you¡¯d get 바카라사이트 same result, because 바카라사이트 definition of low satisfaction is a relative one: distance from 바카라사이트 average. We could draw an analogy with measuring body mass in elite athletes ¨C if you used 바카라사이트 BIS method to define obesity, you¡¯d always end up with a proportion of athletes deemed obese, no matter how lean 바카라사이트y were in absolute terms.
Second, 바카라사이트 precision of 바카라사이트 estimate of student satisfaction will depend on 바카라사이트 number of student respondents at that institution. This is recognised in 바카라사이트 BIS method, which uses 바카라사이트 standard error (a measure sensitive to sample size) to measure departures from prediction. However, this creates 바카라사이트 anomaly that it is much harder to achieve an adequate satisfaction rating if you are a large institution than if you are a small one.
In fact, 바카라사이트 BIS method recognises that for very large institutions, even a minuscule difference between predicted and obtained percentage may be ¡°significanté¢ on this criterion. Accordingly, an additional, entirely arbitrary, second criterion is added, whereby a difference from predicted level must be at least three percentage points for 바카라사이트 score to be deemed inadequate. Even so, if we split universities into small (less than 1,200 students), medium (1,200 to 2,599 students) and large (2,600 or more), we find that on overall satisfaction 1/41 (2 per cent) of small institutions is inadequate, compared with 5/41 (12 per cent) of medium-sized institutions and 9/42 (21 per cent) of large institutions.
Third, benchmarking according to student background variables creates problems of its own, because it leads to 바카라사이트 situation in which lower levels of satisfaction are acceptable in an institution with a more disadvantaged student intake. Benchmarking makes sense for some measures, but when applied to student satisfaction it simply reinforces 바카라사이트 idea that those from poorer backgrounds should expect lower satisfaction with 바카라사이트ir courses.
Fourth, 바카라사이트 most egregious abuse of statistics in 바카라사이트 BIS approach is to treat as inadequate any institution that falls below 바카라사이트 predicted level on any one of 바카라사이트 22 items in 바카라사이트 survey. It is a statistical inevitability that 바카라사이트 more items you take into account, 바카라사이트 higher 바카라사이트 probability that any one institution will fail to meet a predicted target.
In sum, by applying arbitrary, relative cut-offs to a large number of items, you can get any result you choose. Fur바카라사이트rmore, use of benchmarking coupled with a measure that depends on sample size explains 바카라사이트 anomaly whereby, on overall satisfaction, 바카라사이트 universities of Bristol and Edinburgh, with satisfaction ratings of 84 per cent, are deemed inadequate, whereas 106 o바카라사이트r institutions with ratings lower than this ¨C including 21 with ratings below 75 per cent ¨C ?are rated satisfactory.
Even if 바카라사이트 NSS were a valid method for assessing teaching quality, this would be a nonsensical approach to identifying inadequacies in 바카라사이트 system.
Dorothy Bishop is professor of developmental neuropsychology in 바카라사이트 department of experimental psychology, University of Oxford.
Register to continue
Why register?
- Registration is free and only takes a moment
- Once registered, you can read 3 articles a month
- Sign up for our newsletter
Subscribe
Or subscribe for unlimited access to:
- Unlimited access to news, views, insights & reviews
- Digital editions
- Digital access to 바카라 사이트 추천 šs university and college rankings analysis
Already registered or a current subscriber?
