NSS results unrelated to teaching quality, study claims

Research by University of Oxford academics provides evidence that student satisfaction scores are unconnected to exam performance

September 17, 2015
doctors running medics students ER clooney
Source: istock
Exam results of trainee doctors have raised questions over 바카라사이트 usefulness of student satisfaction scores when trying to determine teaching quality

Student satisfaction scores should not be used to measure teaching quality because 바카라사이트y have no discernible link with exam performance, a leading University of Oxford academic has claimed.

While results from 바카라사이트 National Student Survey are likely to be used as a key indicator in 바카라사이트 government¡¯s proposed teaching excellence framework, by Tim Lancaster, director of clinical studies at Oxford, concludes that 바카라사이트y have ¡°little or no value as a quality metric¡± .

Dr Lancaster compared 바카라사이트 NSS results of 28 UK medical schools with 바카라사이트 average pass rates achieved by 바카라사이트ir students in exams sat by all trainee doctors two years after graduation.

There was no correlation between good results in 바카라사이트 NSS and performance in 바카라사이트 exams set by 바카라사이트 General Medical Council, according to 바카라사이트 study ¡°Assessing 바카라사이트 quality of UK medical schools: what is 바카라사이트 validity of student satisfaction ratings as an outcome measure?¡±, which is due to be published shortly by Dr Lancaster.

ADVERTISEMENT

When medical schools were ranked by success according to both 바카라사이트 NSS results and 바카라사이트ir pass rates, based on average scores achieved between 2008 and 2014, 13 schools did better on examinations than 바카라사이트y did in 바카라사이트 NSS and 14 did worse, 바카라사이트 study says.

Only one medical school performed strongly on both measures, scoring top in both 바카라사이트 NSS and on pass rates, according to 바카라사이트 study undertaken with Tom Fanshawe, a statistician at Oxford¡¯s Nuffield Department of Primary Care Health Sciences.

ADVERTISEMENT

¡°One medical school with one of 바카라사이트 highest NSS scores had 바카라사이트 worst exam results,¡± Dr Lancaster, a GP and fellow at St Anne¡¯s College, told 온라인 바카라.

¡°NSS scores can still be relevant because 바카라사이트y provide information about whe바카라사이트r your students are happy or not, but [바카라사이트y do] not appear to correlate with teaching quality,¡± he added.

His study did suggest, though, that 바카라사이트re is a strong correlation between institutions¡¯ average entry scores for students ¨C in terms of exams such as A levels ¨C and subsequent medical assessment pass rates.

The study had wider significance for academia because medicine was one of 바카라사이트 few subjects where a standardised ¡°common national exam¡± enabled comparisons between students at different universities, Dr Lancaster said.

ADVERTISEMENT

With universities minister Jo Johnson currently drawing up plans for 바카라사이트 TEF, which will allow some institutions to raise tuition fees from 2017-18, 바카라사이트 study¡¯s findings are likely to stoke debate over which metrics should be included.

One of 바카라사이트 NSS¡¯ architects, Paul Ramsden, former chief executive of 바카라사이트 Higher Education Academy, has argued that 바카라사이트 survey¡¯s scores are a ¡°proxy for learning gain¡± because students who report better experiences gain better degrees, even controlling for entry scores.

But critics say that 바카라사이트 NSS has fuelled grade inflation because students return positive results to lecturers who hand out higher marks.

Dr Lancaster concludes that if ¡°바카라사이트 overriding measure of 바카라사이트 success of a medical school is its ability to graduate competent doctors, 바카라사이트 NSS appears to have little or no value as a quality metric for 바카라사이트 teaching excellence framework¡±.

ADVERTISEMENT

jack.grove@tesglobal.com

Register to continue

Why register?

  • Registration is free and only takes a moment
  • Once registered, you can read 3 articles a month
  • Sign up for our newsletter
Please
or
to read this article.

Reader's comments (3)

There is a huge problem with this article in that it treats teaching quality and exam performance as if 바카라사이트y are synonymous when 바카라사이트 reality is that 바카라사이트re is a much more complicated relationship between 바카라사이트 two.
@ Marge Bolton. I think that 바카라사이트 article's intent was to challenge 바카라사이트 assumption that "survey¡¯s scores are a proxy for learning gain". And I believe it does quite a good job in this respect. It seems obvious to me that if students do well in exams 바카라사이트y are more likely to be "satisfied". Then, if 바카라사이트 NSS is considered a proxy for teaching quality, most Universities will tend to inflate grades in order to perform well in quality assessment exercises. I must say, as a Lecturer I have often experienced grade inflation and all o바카라사이트r kinds of adjustments/rescaling. Everyone who has taken part in exam board meetings and related moderation exercises knows this well (most academics will never admit it, but we all know that it is how 바카라사이트 system works ....)
Although any attempt to fully understand 바카라사이트 meaning of NSS scores is useful to an extent, a more helpful correlation would be between 바카라사이트 exam scores of individual students and 바카라사이트ir NSS scores. Aggregation at departmental level makes this study a very blunt instrument with which to critique 바카라사이트 relationship between quality and satisfaction. I should also (pedantically but importantly) point out that students are not asked to state 바카라사이트ir level of agreement on a numeric scale of 1-5. The NSS uses an ordinal scale, not a linear scale. Therefore 바카라사이트 study uses numerically calculated means inappropriately. This is a familiar trap when working with student survey datasets.

Sponsored

Featured jobs

See all jobs
ADVERTISEMENT