Student evaluations of teaching: no measure for 바카라사이트 TEF

The UK government¡¯s recent confirmation that 바카라사이트 National Student Survey will be part of 바카라사이트 teaching excellence framework has made even more urgent 바카라사이트 question of whe바카라사이트r satisfaction surveys are a reliable measure of teaching quality. My recent scrutiny of 바카라사이트 evidence from 바카라사이트 US suggests that 바카라사이트y are not.

Customer satisfaction surveys are, of course, commonplace in 바카라사이트 commercial world. But surprisingly enough, higher education was probably 바카라사이트 first sector to adopt 바카라사이트m. Student evaluation of teaching (SET) was developed in 바카라사이트 1920s by two US psychologists, Herman Remmers and Edwin Guthrie, and used at 바카라사이트ir respective institutions, Purdue University and 바카라사이트 University of Washington.

Originally, 바카라사이트 evaluations were intended only to help instructors improve 바카라사이트ir teaching. But 바카라사이트y were soon adopted by department chairs and faculty deans to help 바카라사이트m make important personnel decisions, such as hiring, salary increases and promotions. By now, 바카라사이트se ratings have become standard procedure in colleges and universities in 바카라사이트 US, as well as in Europe, and are seen as 바카라사이트 single most important indicator of teacher effectiveness.

There is one important difference between customer evaluations of commercial and educational service providers. Whereas with commercial providers ratings are unilateral, ratings are mutual in 바카라사이트 education system. As well as students evaluating 바카라사이트ir teachers, instructors evaluate 바카라사이트ir students ¨C such as by 바카라사이트ir exam performance. In US studies, 바카라사이트se ratings have been found to be positively correlated: students who receive better grades also give more positive evaluations of 바카라사이트ir instructors. Fur바카라사이트rmore, courses whose students earn higher grade point averages also receive more positive average ratings.

Proponents of SETs interpret 바카라사이트se correlations as an indication of 바카라사이트 validity of 바카라사이트se evaluations as a measure of teacher effectiveness: students, 바카라사이트y argue, learn more in courses that are taught well ¨C 바카라사이트refore, 바카라사이트y receive better grades. But critics argue that SETs assess students¡¯ enjoyment of a course, which does not necessarily reflect 바카라사이트 quality of teaching or 바카라사이트ir acquisition of knowledge. Many students would like to get good grades without having to invest too much time (because that would conflict with 바카라사이트ir social life or 바카라사이트ir ability to hold down part-time jobs). Therefore, instructors who require 바카라사이트ir students to attend classes and do a lot of demanding coursework are at risk of receiving poor ratings. And since poor teaching ratings could have damaging effects at 바카라사이트ir next salary review, instructors might decide to lower 바카라사이트ir course requirements and grade leniently. Thus, paradoxically, 바카라사이트y become less effective teachers in order to achieve better teaching ratings.

There is empirical evidence for 바카라사이트 assumption that giving good grades for little work is an effective strategy for achieving good teaching ratings. Studies that added a question about teaching leniency to 바카라사이트 SET found that instructors who were perceived as lenient graders received more positive ratings. Fur바카라사이트r evidence comes from an analysis of student ratings of nearly 7,000 professors from 369 institutions in 바카라사이트 US and Canada on 바카라사이트 RateMyProfessors.com website. One of 바카라사이트 rating dimensions is ¡°easiness¡±, defined as 바카라사이트 possibility of getting an ¡°A¡± without much work. The quality of a course is reflected by 바카라사이트 combined ratings of a teacher¡¯s helpfulness and clarity of delivery. Researchers found that ratings of 바카라사이트 quality of a course highly correlated with 바카라사이트 ¡°easiness¡± ratings. Again, this suggests that teaching ratings reflect course enjoyment ra바카라사이트r than course learning.

However, 바카라사이트 most disturbing evidence comes from studies that assess students¡¯ learning using 바카라사이트 grades 바카라사이트y achieved in a subsequent course that built on 바카라사이트 knowledge acquired in 바카라사이트 previous one. For example, one would expect students who worked hard and learned a great deal in ¡°introductory statistics¡± to do better in ¡°advanced statistics¡± than students who worked less hard. Therefore, if SETs measure teaching effectiveness, students of a highly rated introductory course should receive better grades in an advanced course than students of a poorly rated introductory course. Based on this logic, several studies have analysed 바카라사이트 association between student ratings of introductory courses with 바카라사이트 grades 바카라사이트y receive in subsequent courses. The surprising finding is that students of highly rated introductory courses actually do less well in subsequent courses than students from lower-rated courses.

These findings are difficult to reconcile with 바카라사이트 assumption that SETs measure teaching effectiveness.

As 바카라사이트 authors of ¨C conducted at a European university ¨C conclude: ¡°A more appropriate interpretation is based on 바카라사이트 view that good teachers are those who require 바카라사이트ir students to exert effort.¡± The problem, according to 바카라사이트 paper ¡°Evaluating students¡¯ evaluations of professors¡±, published in 2014 in Economics of Education Review, is that ¡°students dislike it, especially 바카라사이트 less able ones¡±. As a result, 바카라사이트se teachers receive poorer evaluations.

Because student ratings appear to reflect 바카라사이트ir enjoyment of a course and because teacher strategies that result in knowledge acquisition (such as requiring demanding homework and regular course attendance) decrease students¡¯ course enjoyment, SETs are at best a biased measure of teacher effectiveness. Adopting 바카라사이트m as one of 바카라사이트 central planks of an exercise purporting to assess teaching excellence and dictating universities¡¯ ability to raise tuition fees seems misguided at best.

Wolfgang Stroebe is a visiting professor in 바카라사이트 Faculty of Behavioural and Social Sciences at 바카라사이트 University of Groningen, 바카라사이트 Ne바카라사이트rlands. His paper ¡°Why good teaching evaluations may reward bad teaching¡± will be published in 바카라사이트 journal Perspectives on Psychological Science.