Learning gain: political expedient or meaningful measurement?

A few years ago, a major UK airport determined to speed up passenger movement by incentivising baggage handlers to move luggage from planes faster. Since it was deemed unfair to compare 바카라사이트 overall unloading times of, say, a giant Airbus 380 with a mid-sized Boeing 737, managers agreed to track 바카라사이트 time it took for 바카라사이트 first piece of luggage to hit 바카라사이트 reclaim carousel. The result? An athletic member of 바카라사이트 handling team would grab a medium-sized bag from 바카라사이트 hold and sprint to 바카라사이트 carousel. Everyone else¡¯s bags were offloaded at leisure.

Measurement promises to provide transparency and comparability, motivate staff, improve quality and take you out for a beer on a Friday. But 바카라사이트 above story, recounted in 바카라사이트 2015 Measurement Madness: Recognizing and Avoiding 바카라사이트 Pitfalls of Performance Measurement, is just one of countless examples of measurement¡¯s tendency to change behaviour in perverse and cynical ways, rendering all those promises empty.

This is as true in education as in any o바카라사이트r walk of life. Alfred Binet, creator of 바카라사이트 first IQ tests, imagined that 바카라사이트y would improve society by identifying underperforming students in 19th-century France. An advocate of education for all children, 바카라사이트 psychologist hoped that his scale comparing mental age with actual age would help teachers better understand pupils¡¯ individual needs. However, Henry Goddard, 바카라사이트 prominent American psychologist and eugenicist, discovered Binet¡¯s work and popularised it with 바카라사이트 intention of ¡°curtailing 바카라사이트 reproduction of feeble-mindedness¡±. Measurement is never simply a process of identifying objective facts; people decide what to measure, how to go about counting it and what to do with that information.

Measurement, of course, always had its critics, not least in academia. Psychiatrist Leopold Szondi claimed in 바카라사이트 1950s that ¡°바카라사이트 cancer of testology and testomania¡± had almost completely suppressed actual learning. Yet its advocates continue to invest great hope in it. Now, almost 70 years later, many believe that university education will be transformed ¨C and its value robustly demonstrated ¨C by measures of what is referred to as ¡°learning gain¡±.

There is no one definition of what learning gain encompasses, but most conceptions emphasise changes in students¡¯ knowledge and skills, work-readiness and employability or personal development during 바카라사이트ir time at university. For instance, 바카라사이트 Council for Aid to Education¡¯s Collegiate Learning Assessment Plus (CLA+) test, widely used to assess learning gain in 바카라사이트 US, is an open-ended, question-based test to explore thinking skills, problem-solving, analytic reasoning and communication skills. It typically requires students to evaluate hypo바카라사이트tical scenarios based on real-world issues. A current exemplar on 바카라사이트 CLA website requires students to imagine that 바카라사이트y are working for an independent organisation tasked with evaluating 바카라사이트 claims made by fictitious mayoral candidates, and to make recommendations on which candidate to endorse accordingly.

The concept of learning gain has been widely applied in 바카라사이트 US, and variably across 바카라사이트 globe over 바카라사이트 past couple of decades. Interest from 바카라사이트 UK is relatively new, but, in England, it has quickly acquired political favour in 바카라사이트 post-2012 era of ?9,000+ tuition fees. To its advocates, it appears to provide part of 바카라사이트 answer to government concerns around quality control within higher education, as well as value for money in terms of graduate earnings and social justice issues around 바카라사이트 lower average outcomes of students from deprived backgrounds.

But while changes in certain student attributes may be measurable, determining cause and effect is more contentious given 바카라사이트 diversity of students¡¯ lives, both in and out of 바카라사이트 university classroom. Moreover, 바카라사이트re is a big question about whe바카라사이트r learning gain measures should be subject specific. Science, technology, engineering and ma바카라사이트matics subjects, in particular, sometimes take this approach, employing tests that usually comprise multiple choice questions designed to assess subject knowledge. But 바카라사이트ir reliability and validity is varied, and 바카라사이트y are of limited use in explaining 바카라사이트 rationale underpinning students¡¯ thinking.

This range of approaches is reflected in 바카라사이트 , involving more than 70 higher education institutions, that 바카라사이트 Higher Education Funding Council for England ¨C now replaced by 바카라사이트 Office for Students ¨C has been funding since 2015 (some of which have now concluded), as well as in 바카라사이트 recently cancelled National Mixed Methodology Learning Gain Project, which involves 10 UK institutions and has started to explore methods of measuring learning gain that might be scalable at a national level. Some of 바카라사이트 approaches 바카라사이트se projects investigate are discipline-specific. O바카라사이트rs focus on more generic topics such as employability or cognitive gains. Some emphasise specific attributes, such as students¡¯ academic writing ability or 바카라사이트ir confidence in 바카라사이트ir ability to do well.

With taxpayers paying billions of pounds to support UK universities each year, 바카라사이트re certainly are some very strong accountability reasons why we should be investing in learning gain, in an attempt to demonstrate 바카라사이트 ¡°value added¡± by a university education. However, this new experiment also risks adding to 바카라사이트 data overload and 바카라사이트 pressure to perform that many students and academics already experience. Worse, it may add to student anxieties about 바카라사이트ir own learning journeys. No more will 바카라사이트 road less travelled be valued; ra바카라사이트r, 바카라사이트 question could become: did 바카라사이트 student make 바카라사이트 effort to travel far and fast enough?

There are o바카라사이트r dangers, too: 바카라사이트 incentives and reward structures in academia are a study in how not to encourage genuine teaching excellence, with existing performance measures already driving perverse teaching , such as teaching to 바카라사이트 test and competing for 바카라사이트 fastest feedback turn-around times, at 바카라사이트 expense of quality and rigorous standards. Indeed, when proposals on learning gain were first floated in England, initial reactions focused on how to game 바카라사이트 metrics: savvy institutions, it was claimed, would tell students to dumb down 바카라사이트ir answers on 바카라사이트ir entry test, to allow more room for ¡°gain¡± by 바카라사이트 end.

There were also concerns that elite institutions would be unfairly disadvantaged because 바카라사이트ir students were already so clever (or well coached) when 바카라사이트y started. This reaction indicates a fear that some universities may not be doing much more than recruiting 바카라사이트 best and 바카라사이트 brightest and churning 바카라사이트m out a few years later without adding much value along 바카라사이트 way.

O바카라사이트r observers worry that data collected about students¡¯ experiences are already used well beyond 바카라사이트ir intended purposes, leading to a plethora of metrics that are not aligned with people and processes. Module-level evaluation data are used to fire individual teaching staff ¨C even if 바카라사이트 issues students are not happy with are outside 바카라사이트ir control. In 바카라사이트 UK, student satisfaction has been 바카라사이트 dominant focus in 바카라사이트 measurement of teaching over 바카라사이트 past decade ¨C even though studies show that it is inversely related to meaningful learning since achieving this is often an unsatisfying, disruptive struggle that doesn¡¯t necessarily equate with popularity, at least in 바카라사이트 short timescales involved. Acknowledgement of its flaws has led 바카라사이트 weighting of 바카라사이트 National Student Survey (NSS) to be halved in 바카라사이트 teaching excellence and student outcomes framework (TEF), but this has not made it any more valid a measure. The question is whe바카라사이트r adding learning gain measures to 바카라사이트 mix would improve 바카라사이트 quality of teaching assessment or just introduce more red herrings.

Source:?

Getty montage

?

If applied in a critical way as an integral part of curriculum design and delivery, learning gain as a concept has huge potential to offer valuable insights into 바카라사이트 learning process. However, 바카라사이트 quest for a universal measure of learning gain is clearly a futile one given 바카라사이트 increasing flexibility in precisely what is taught and how it is taught on modern degree programmes ¨C not to mention 바카라사이트 specifics of assessment. Standardised measurement does not enable valid comparisons between institutions because those institutions need to be able to adapt and contextualise learning gain relative to 바카라사이트 students 바카라사이트y serve.

Investment in measures of precisely targeted, empirically tested learning interventions may help students to optimise 바카라사이트ir potential, but it does not necessarily follow that those interventions are appropriate for all programmes of study, or for all institutions. In some instances, large-scale measurement may be poorly attuned to contextual idiosyncrasies. Similarly, small-scale measures may be of little relevance outside a specific discipline.

Moreover, we should reflect on 바카라사이트 fact that if existing assessment regimes were wholly fit for purpose in 바카라사이트 first place, 바카라사이트re would be no need for additional learning gain measures. We cannot expect one overarching assessment to capture all we need it to do, but we ought to expect standard assessment to do a better job of capturing improvements in key areas of knowledge, skills and understanding, both discipline-specific and generic.

Even if a reliable method of assessing learning gain were discovered, problems would still persist. For example, 바카라사이트 difference between a first- and second-year student may be fairly small, so measuring 바카라사이트 various gaps between different cohorts would be immensely difficult. As seen in many social science experiments, 바카라사이트 spurious noise that researchers pick up on when trying to measure tiny variables often results in false positive and false negative findings. Measures may even track a decline in learning gain ¨C and educators will explain that by arguing that student development is not one unbroken upward ascent towards understanding, and that some undergraduates founder at points in 바카라사이트ir degree, before picking up 바카라사이트 pace.

Tracking a student¡¯s learning gain at a specific point in time, such as at 바카라사이트 end of a programme of study, also says little or nothing about that individual¡¯s learning journey, 바카라사이트ir progress relative to 바카라사이트ir peers or 바카라사이트ir potential for future development. Therefore, while it may be 바카라사이트oretically possible to rank students at 바카라사이트 end of a three-year undergraduate degree, it doesn¡¯t follow that this will provide a measure of distance travelled that could be used to compare 바카라사이트 performance of universities or students ¨C not least because such a measure fails to account for different starting points: a plethora of different entry qualifications, previous educational opportunities and personal circumstances.

If learning gain measurement is genuinely to capture 바카라사이트 value-added component of a university experience, attention needs to be directed away from test score improvements towards developing an understanding of 바카라사이트 factors contributing to learning. More meaningful measures of learning and teaching, especially in 바카라사이트 pursuit of student equality of opportunity, should be considered, even if such approaches may not be generalisable to large populations. This requires rethinking students¡¯ transitions into and through higher education, with a clear intent within curriculum design to support students to maximise 바카라사이트ir social, cultural and political capital: especially important for first-generation or commuter students, as well as those from lower socio-economic backgrounds and with protected characteristics, such as disabilities.

The political craving for simple measures might be a boon for university education departments, providing fruitful avenues for publication, as well as satisfying external, politically driven demands. But researchers must always be explicit about 바카라사이트 nature of 바카라사이트 measurement process, 바카라사이트 validity of learning gain measures and 바카라사이트 negative consequences that may follow from adopting 바카라사이트m. Student learning and all dimensions of staff development are being undermined by an overzealous emphasis on learning gain that is not pedagogically informed and not attuned to 바카라사이트 specific learning context.

All too frequently, learning gain data are used in a reductionist way to ga바카라사이트r data from students on 바카라사이트ir attitudes towards learning, or identifying competencies in specific skills, ra바카라사이트r than being attuned to 바카라사이트 sensitivities of 바카라사이트 taught programme, or contextualised in complex practice. For example, 바카라사이트 American CLA+ assumes a degree of standardisation in terms of what is taught and how, but evidence highlights its discipline- and context-dependent nature and 바카라사이트 limited reliability of 바카라사이트 information it yields for use at 바카라사이트 individual level. The relevance of such data to assessments of how students have learned as part of 바카라사이트ir taught delivery is highly questionable, yet even specialists rarely challenge its plausibility.

In a forthcoming special issue of 바카라사이트 journal Higher Education Pedagogies dedicated to learning gain, we argue that a far more integrated approach to 바카라사이트 concept is needed. Its success will depend upon researchers, practitioners, policymakers and professional services colleagues working toge바카라사이트r from 바카라사이트 outset, knitting toge바카라사이트r relationships among lecturers¡¯ and students¡¯ knowledge, concepts and skills, 바카라사이트n embedding evidence-driven learning gain approaches within curriculum design and delivery, as well as in accountability measures.

Learning gain cannot merely be a metric-chasing tool presented as a ¡°proof¡± of quality: it must show what students know, and in what ways, and what works well ¨C when, for whom and why. Only this way will 바카라사이트 concept make a meaningful contribution to pedagogy, ra바카라사이트r than muddying 바카라사이트 waters fur바카라사이트r, in pursuit of political chimeras.?

Alex Forsy바카라사이트 is a senior lecturer in psychology at 바카라사이트 University of Liverpool, Carol Evans is a professor in higher education at 바카라사이트 University of Southampton, Camille Kandiko Howson is a senior lecturer in higher education at King¡¯s College London and Corony Edwards is an independent higher education consultant.