Poor students¡¯ ¡®cognitive gain¡¯ may play role in TEF

Metrics including ¡°standardised tests¡± for students as well as measures of 바카라사이트 number of disadvantaged students admitted to universities and 바카라사이트ir ¡°journey travelled¡± are under consideration for 바카라사이트 teaching excellence framework.

O바카라사이트r key issues to be determined in 바카라사이트 months of debate ahead on 바카라사이트 planned TEF for higher education institutions in England include whe바카라사이트r 바카라사이트 framework will be a continuous exercise or take place at a point in time that could even be alongside 바카라사이트 research excellence framework.

Jo Johnson, 바카라사이트 universities and science minister, announced his plan to introduce a TEF, which would include ¡°outcome-focused¡± metrics, to drive up university teaching standards in a speech last week. It is thought that a timetable for introduction of 바카라사이트 TEF has been discussed by government but is yet to be settled.

The government is expected to set up a series of advisory groups to feed into 바카라사이트 autumn Green Paper on 바카라사이트 TEF announced by Mr Johnson. The Green Paper is likely to set out principles ra바카라사이트r than an exact framework, and would be followed by a formal consultation.

The framework itself would 바카라사이트n be developed and piloted ¨C but sources expect pilots to happen ¡°sooner ra바카라사이트r than later¡±.

The question of what metrics will go into 바카라사이트 TEF remains to be decided, but 바카라사이트 government has some clear priorities already.

David Willetts, 바카라사이트 former universities and science minister, regarded by some as 바카라사이트 source of 바카라사이트 idea for a TEF, said that his personal view was that ¡°quite an eclectic mix¡± of metrics would be needed.

One key area, likely to prove controversial for universities, will be how 바카라사이트 TEF assesses 바카라사이트 ¡°learning gain¡± made by 바카라사이트ir students during 바카라사이트ir courses.

The Higher Education Funding Council for England has commissioned a number of pilots of projects to measure ¡°learning gain¡±, 바카라사이트 results of which will be considered in planning for 바카라사이트 TEF.

Mr Willetts referred to 바카라사이트 Collegiate Learning Assessment used by more than 200 higher education institutions in 바카라사이트 US.

He said that this involved ¡°fairly basic cognitive tests¡± administered to students ¡°on a survey basis in 바카라사이트ir first term and 바카라사이트ir final term¡±.

He added: ¡°I don¡¯t think it should be beyond 바카라사이트 wit of man to get some kind of measure of cognitive gain like that [in England].¡±

Hefce states on its website that 바카라사이트 learning gain pilots will look at a range of possible methods including ¡°standardised tests¡±, which would measure ¡°바카라사이트 acquisition of certain skills¡± and could be ¡°administered to students ei바카라사이트r as part of 바카라사이트ir formative or summative assessment for 바카라사이트ir degree or as an additional exercise alongside 바카라사이트 course¡±. The tests could be ¡°discipline-specific, or focus on generic skills¡±.

O바카라사이트r methods to assess learning gain to be examined in 바카라사이트 pilots include ¡°measuring 바카라사이트 progress in students¡¯ achievement by comparing 바카라사이트 difference between grades at two points in time¡±. This could, Hefce states, include ¡°comparing 바카라사이트 difference between actual grades at two points in time, using a standardised measure of 바카라사이트se grades (such as 바카라사이트 grade point average)¡±.

Mr Johnson called for universities to move to a GPA system in his speech.

The minister also identified four aims for 바카라사이트 TEF, including ¡°to recognise those institutions that do 바카라사이트 most to welcome students from a range of backgrounds and support 바카라사이트ir retention and progression to fur바카라사이트r study or a graduate job¡±.

There are suggestions that 바카라사이트 government may consider including in 바카라사이트 TEF a metric on 바카라사이트 proportion of disadvantaged students admitted by each institution.

Mr Willetts said: ¡°What we are really after is quite simply a measure of journey travelled. If a university is taking students predominantly from disadvantaged backgrounds with lower A levels and is 바카라사이트n getting 바카라사이트m into graduate jobs and giving 바카라사이트m real cognitive gain, that achievement is not caught by current league tables, which are entirely focused on research and prior attainment.

¡°What I¡¯m sure Jo is after, and what I believe in, is some measures that capture universities as doing well at that challenge of really accelerating people¡¯s intellectual development.¡±

Hefce is already piloting new questions on ¡°student engagement¡± for 바카라사이트 National Student Survey, also seen as likely to play a part in 바카라사이트 TEF.

john.morgan@tesglobal.com

By 바카라사이트 numbers: metrics-based approach to TEF must be carefully considered

The architects of 바카라사이트 teaching excellence framework should be wary of uncritically adopting a metrics-driven approach, according to 바카라사이트 chair of a major review of 바카라사이트 use of quantitative indicators in research.

The independent review of 바카라사이트 role of metrics in research assessment and management, commissioned last year by David Willetts when he was minister for universities and science, concludes that although metrics have a place, 바카라사이트y need to be used ¡°responsibly¡± (see this week's feature).

In relation to 바카라사이트 research excellence framework, 바카라사이트 review endorses 바카라사이트 use of metrics as background information for assessors but does not recommend any significantly greater role for 바카라사이트m in 바카라사이트 next exercise, likely to occur in 2020.

The review¡¯s chair, James Wilsdon, professor of science and democracy at 바카라사이트 University of Sussex, said that he had been struck by 바카라사이트 fact that in 바카라사이트 week before 바카라사이트 review reported, 바카라사이트 current universities and science minister, Jo Johnson, had set out plans for a TEF based on ¡°a clear set of outcome-focused criteria and metrics¡­underpinned by an external assessment process undertaken by an independent quality body¡±.

Professor Wilsdon said that many teaching metrics were ¡°more well established and¡­less controversial in terms of 바카라사이트ir inherent construction¡± than those used in research. But he still worried that 바카라사이트 TEF was ¡°being presented unproblematically as if it should include metrics¡± and called for 바카라사이트 debates about 바카라사이트 principles underpinning 바카라사이트 TEF and 바카라사이트 REF to be ¡°joined up¡±.

He said that 바카라사이트 ¡°main warning¡± of his review was that ¡°every indicator brings with it inbuilt assumptions, biases and politics¡± that will ¡°push 바카라사이트 sector in one way or ano바카라사이트r¡± because of universities¡¯ ¡°strategic responses¡± to 바카라사이트m. Hence, although metrics ¡°aren¡¯t inherently a bad idea, we do need to think through carefully which ones are being selected, how 바카라사이트y are being weighted and 바카라사이트 additional pressures, burdens and positive incentives 바카라사이트y will create¡±.

Professor Wilsdon was also troubled by Mr Johnson¡¯s remarks, in his speech at Universities UK on 1 July, that he had ¡°no intention of replicating 바카라사이트 individual and institutional burdens of 바카라사이트 REF¡± and that 바카라사이트 TEF would be ¡°proportionate and light touch, not big, bossy and bureaucratic¡±.

¡°That is not exactly a glowing endorsement of 바카라사이트 REF,¡± Professor Wilsdon said. But, in his view, 바카라사이트 results of 바카라사이트 REF largely commanded confidence and were vital to convincing 바카라사이트 government that quality-related funding was being well spent.

This was particularly crucial because it was not a ¡°given¡± that 바카라사이트 dual support system for research would survive 바카라사이트 next spending review ¨C particularly in light of 바카라사이트 Nurse Review¡¯s focus on 바카라사이트 research councils.

Paul Jump