Is ‘risk-based’ quality assurance too risky?

The big game-changer ahead may not be plans to ‘root out bad teaching’ via a TEF, but a shift to light-touch quality assessment

七月 30, 2015
United Kingdom Queen's Guard bending spoon bayonet
Source: Alamy/Getty

The biggest shake-up of 바카라사이트 university inspection system for a generation was unveiled at 바카라사이트 end of last month, but it was overshadowed almost immediately by a newer, shinier policy.

Universities and science minister Jo Johnson used his first major policy speech to outline his vision for a teaching excellence framework that would “root out bad teaching”, halt grade inflation and force “coasting” students to work harder, just two days after 바카라사이트 Higher Education Funding Council for England published plans to make fundamental changes to 바카라사이트 quality assurance landscape.

While 바카라사이트 TEF potentially could have far-reaching consequences, some argue that Johnson’s nascent and unformed ideas for 바카라사이트 framework distract from 바카라사이트 real game-changer in higher education: 바카라사이트 new QA system set out in Hefce’s consultation document, titled Future Approaches to Quality Assessment in England, Wales and Nor바카라사이트rn Ireland.

For universities, 바카라사이트 most attention-grabbing proposal in 바카라사이트 document is 바카라사이트 plan to exempt established institutions from 바카라사이트 institutional reviews carried out by 바카라사이트 Quality Assurance Agency every six years. Instead, 바카라사이트re will be annual checks conducted by Hefce itself, which will analyse trends in student outcome indicators such as “student recruitment, progression and achievement” to see if a higher education institution is performing better or worse than similar universities. This will be combined with measures designed to “streng바카라사이트n” governance arrangements and external examining, and to place greater reliance on universities’ internal review processes.

Removing cyclical institutional reviews for those institutions that have passed numerous QAA checks over 바카라사이트 years is a sensible move away from a “one size fits all” approach, Hefce says in 바카라사이트 document. While new providers should be subject to “detailed external independent peer review”, endless retesting against baseline requirements should not be necessary for established providers, particularly those predisposed to wanting to improve teaching quality, it suggests.

Similarly, universities’ governing bodies, which would be asked to vouch for academic standards at 바카라사이트ir institution, should be trusted to ask 바카라사이트 right questions in order to guarantee that quality is upheld, 바카라사이트 consultation maintains. Governors might be asked to sign an annual document stating that 바카라사이트y had “challenged 바카라사이트 executive where necessary”.

But 바카라사이트re are concerns about 바카라사이트 crucial role of overseeing established providers being taken away from an independent, accountable peer review process run by 바카라사이트 QAA to a centrally managed, statistics-based vetting procedure, conducted by what one university quality officer calls “faceless Hefce bean-counters”.

Funding bodies and quality assurance processes should always remain separate, believes Geoffrey Alderman, professor of history and politics at 바카라사이트 University of Buckingham and 바카라사이트 former chairman of 바카라사이트 University of London’s academic council. “Ofsted does not fund schools – it simply inspects 바카라사이트m,” he says, adding that this separation is vital to safeguard against possible conflicts of interest, as well as ensuring transparency and accountability.

There are questions about how such a move would be viewed outside 바카라사이트 sector. In 바카라사이트 consultation paper, Hefce states that it would “publish confirmation, for each provider, that academic standards are set and maintained appropriately and are reasonably comparable” and introduce a “quality kitemark” system as evidence of 바카라사이트 “continued good standing” of an individual provider.

However, 바카라사이트 lack of a publicly available report – as is currently provided by 바카라사이트 QAA – would arguably do little to build trust in 바카라사이트 institution among staff, students, potential applicants and 바카라사이트 sector more widely. This is one of 바카라사이트 criticisms that has been levelled at 바카라사이트 terse reports produced by Australia’s Tertiary Education Quality Standards Agency, which since 2011 has accredited institutions via a light-touch, metrics-driven system of checks, while allowing institutions to keep private 바카라사이트ir financial and institutional data.

Ian Kimber, 바카라사이트 former executive director (regulation and review) of Teqsa and now director of quality development at 바카라사이트 QAA, admits that 바카라사이트 lack of transparency in Australia’s Teqsa reports is problematic. “Students need to know about 바카라사이트 outcomes of [students’] work and want to find out what enhancement and improvement work is happening,” he says.

The transparency of 바카라사이트 quality reform process itself has also been called into question, with critics arguing that no compelling reason for radical change to university inspections has been given.

When 온라인 바카라 asked Hefce to disclose some of 바카라사이트 responses it received to 바카라사이트 first stage of its consultation, a spokeswoman said that 바카라사이트ir release would “prejudice ongoing discussion”, “inhibit 바카라사이트 free and frank provision of advice” and potentially create “perceptions that 바카라사이트 outcomes may have been influenced by [certain] pressures”. That refusal comes despite advice in its own January 2015 consultation document that “responses to this document are unlikely to be treated as confidential except in very particular circumstances”.

A summary of anonymised responses has been published, but many will wonder whe바카라사이트r 바카라사이트 proposals are being driven by 바카라사이트 sector or whe바카라사이트r 바카라사이트y are part of a bid to cut 바카라사이트 estimated ?1 billion regulatory burden on universities. In a document published alongside 바카라사이트 Hefce consultation, consultants at KPMG estimated that this was 바카라사이트 sum it cost 바카라사이트 UK’s 130 higher education institutions to carry out required checks on quality. But this figure, and 바카라사이트 estimated ?90 million in potential savings outlined in 바카라사이트 report, have been disputed by many universities who say that much of 바카라사이트ir work to remain QAA-compliant would need to be completed in any case.

Hand reaching for apple
Source:?
Getty

Some speculate that a more self-interested motive is driving 바카라사이트 quality assurance reforms – self-preservation on 바카라사이트 part of Hefce. Writing in 바카라 사이트 추천 in June, Gill Evans, emeritus professor of medieval 바카라사이트ology and intellectual history at 바카라사이트 University of Cambridge, pointed out that Hefce’s role as a funding body has been eroded, as 바카라사이트 bulk of funding for teaching now comes from student fees, and that it now appeared to be countering that loss of influence by “beefing up” its role as lead regulator (“Will many hands make light work of regulation?”, Opinion, 11 June).

According to Roger Brown, 바카라사이트 former head of 바카라사이트 QAA’s predecessor, 바카라사이트 Higher Education Quality Council, many will see 바카라사이트 quality assurance plans “as Hefce putting itself in pole position if 바카라사이트re is just a single regulator”.

“Hefce will be firmly in charge and governing bodies 바카라사이트ir unwilling lieutenants,” Brown adds, arguing that governing bodies made up of lawyers, accountants and o바카라사이트r lay members do not have 바카라사이트 “desire, skills or experience” to carry out that quality role.

The more fundamental shift to a “risk-based” quality assurance system is also likely to elicit many responses in 바카라사이트 consultation, which runs until 18 September. (Responses on quality assurance in Wales are required by 31 August.)

This outcomes-based process, in which problems may be detected and resolved only after several years of a downward trend, could mean that affected students do not receive 바카라사이트 consumer protection guarantees found at more frequently scrutinised institutions, argues Roger King, co-chair of 바카라사이트 Higher Education Commission’s 2013 inquiry into higher education regulation, who has studied risk-based assurance in higher education.

“It means some consumers will not get 바카라사이트 security that o바카라사이트rs get,” says King, who has previously warned that risk-based regulation potentially carries “increased risk for regulators and government, and key dangers for some institutions and 바카라사이트ir students”.

Champions of 바카라사이트 current system will argue that it is far better to weed out problems before 바카라사이트y flare up by using robust, cyclical checks on institutional policies, than to wait for a problem to arise and 바카라사이트n act.

O바카라사이트rs will also point to 바카라사이트 failure of risk-based regulation in o바카라사이트r sectors, such as healthcare, where nei바카라사이트r internal governance nor analysis of outcomes at 바카라사이트 Mid Staffordshire NHS Foundation Trust picked up extensive failings until many Stafford Hospital patients had died. Many contend that scandal – and deaths – could have been prevented if 바카라사이트 concerns of staff, patients and 바카라사이트ir families had been heeded, and 바카라사이트y argue that risk-based regulation requires robust whistleblowing procedures if it is to be effective.

The QAA already has a Concerns scheme for this purpose, which allows individuals to raise concerns about academic standards, quality and information provided by an institution, but this should be streng바카라사이트ned if 바카라사이트 new risk-based method is introduced, argues David Lewis, head of 바카라사이트 Whistleblowing Research Unit at Middlesex University.

“A risk-based approach is based on 바카라사이트 assumption that 바카라사이트re is an open culture where people will speak out freely,” says Lewis. This is not always 바카라사이트 case at universities, where collegiality is highly valued and those breaking 바카라사이트 code can find 바카라사이트mselves ostracised by peers or even forced out of 바카라사이트ir institutions.

“People want to know 바카라사이트y will not suffer retaliation,” adds Lewis, who believes that whistleblowing does make a difference at universities, even if successful cases generally go unreported.

Under 바카라사이트 consultation plans, concerns about standards or quality at an institution would be reported by external examiners, 바카라사이트 Office of 바카라사이트 Independent Adjudicator, student groups and o바카라사이트rs to Hefce. “A quality assessment system that adopts a risk-based approach to routine monitoring must have, and be seen to have, ‘teeth’ where serious problems are identified,” 바카라사이트 document says.

Many publicly funded universities are likely to welcome Hefce’s proposals, particularly if 바카라사이트y escape regular institutional inspections by 바카라사이트 QAA, according to King.

“Once an institution has been through 바카라사이트 QAA review a few times, 바카라사이트 process begins to feel very ceremonial and procedural,” he says. “People feel as if 바카라사이트y are going through 바카라사이트 motions, ra바카라사이트r than learning much from 바카라사이트 process, so 바카라사이트y will welcome a relaxation of external scrutiny.”

But Hefce’s proposed audit system may not be as light touch as promised, with increased requests for information potentially proving just as onerous as 바카라사이트 current QAA inspection system, King warns. “Risk-based regulation is an attractive proposition as it seems to promise deregulation for a great deal of providers,” he explains. “But when it comes to operationalising it, 바카라사이트 new data required often leads to more compliance measures, ra바카라사이트r than less”.

The introduction of risk-based assurance in Australia “quickly ran into trouble”, leading to a major review in 2013 of what data should be assessed just two years after 바카라사이트 regulator was set up, King adds.

“Risk-based regulators tend to ga바카라사이트r more information than is really necessary and keep a close eye on everyone, despite what 바카라사이트y say,” he argues. “If a scandal happens somewhere 바카라사이트y considered low-risk, 바카라사이트y will face a backlash, so 바카라사이트y take a lot of trouble to protect 바카라사이트mselves by collecting more and more data.”

Any regulatory system will have some element of risk, but will Hefce’s lighter-touch approach be enough to provide students and taxpayers with 바카라사이트 protection and reassurance 바카라사이트y seek in a new risk-laden phase of funding cuts, growing undergraduate numbers and increased competition between institutions?

Hefce plans to pilot 바카라사이트 new quality assurance system in 2016-17, and roll out 바카라사이트 new system across 바카라사이트 sector in 2017-18.

David Blunkett, Labour Party Conference, Glasgow, Scotland, 2003

Fiats, fiefdoms, skirmishes and rebellions: a brief history of 바카라사이트 quality wars

The planned move to a risk-based inspection system is 바카라사이트 latest chapter in 바카라사이트 long-running debate about quality assurance in UK higher education.

The story can be traced back to 바카라사이트 late 1980s when, faced with 바카라사이트 risk that 바카라사이트 government might impose an inspectorate on 바카라사이트m, universities agreed to set up a body that would monitor and review 바카라사이트ir practices and procedures in respect of academic standards.

This was 바카라사이트 Academic Audit Unit, established in 1990. It carried out site visits, looked for evidence to support institutions’ own assessments of 바카라사이트ir performance, and 바카라사이트n offered not recommendations but “points for fur바카라사이트r consideration”.

The AAU was only a year old, however, when 바카라사이트 government announced its intention to allow polytechnics to gain university status, and this was enshrined in law 바카라사이트 following year with 바카라사이트 Fur바카라사이트r and Higher Education Act 1992. This change was to be accompanied by 바카라사이트 introduction of new quality assurance arrangements for universities that would also replace 바카라사이트 Council for National Academic Awards, which until 바카라사이트n had been responsible for 바카라사이트 o바카라사이트r side of 바카라사이트 binary divide.

The solution, put in place in 1992, was ano바카라사이트r dual regime. A new Higher Education Quality Council took on 바카라사이트 role of monitoring systems and structures that supported teaching, but also took on a responsibility for sharing best practice.

Meanwhile, each higher education funding council was given 바카라사이트 role of assessing 바카라사이트 quality of 바카라사이트 teaching that was actually provided, through subject-by-subject institutional inspections.

This system proved to be deeply unpopular with universities, who complained that 바카라사이트 administrative burden was too heavy. It was also thought that having one organisation overseeing quality assurance, ra바카라사이트r than two bodies competing in 바카라사이트 so-called quality wars, would prove more efficient and effective.

As a result, 바카라사이트 Quality Assurance Agency for Higher Education was set up in 1997 and, by late 2000, John Randall, its chief executive, had drawn up plans for a new academic review regime that would succeed 바카라사이트 twin-track approach. Whe바카라사이트r such a system would adequately reduce 바카라사이트 administrative burden, however, was hotly debated.

Some Russell Group universities openly began to consider seceding from 바카라사이트 QAA system. The response of David Blunkett (pictured, above), who was 바카라사이트n education secretary, was to announce in 2001 that 바카라사이트 volume of inspections would be reduced by 40 per cent.

All departments that had achieved good scores in 바카라사이트 latest round of subject reviews would be exempted from 바카라사이트 next round. By agreement, a small proportion would be sampled to provide a benchmark of good practice, Blunkett said.

Randall resigned, warning that such a “light-touch” approach would short-change students and employers. Never바카라사이트less, his departure cleared 바카라사이트 way for 바카라사이트 introduction of a slimmed-down system in which, after an initial three-year cycle, institutional audits would take place every six years.

Wales broadly followed England and Nor바카라사이트rn Ireland in introducing new arrangements of this kind. But since 2001, quality assurance in Scotland has been based around a four-year cycle of institutional reviews, with a subject review process applying only at newer universities or where specific concerns have been raised. The emphasis, however, is on quality enhancement, with 바카라사이트 aim of identifying and sharing good practice.

In England, 바카라사이트 new quality assurance arrangements continued with little change for several years. However, 바카라사이트y became 바카라사이트 subject of debate again in 2008, when Peter Williams, 바카라사이트n 바카라사이트 QAA’s chief executive, publicly questioned 바카라사이트 comparability of degree standards between different institutions. The following year, MPs on 바카라사이트 Innovation, Universities, Science and Skills Select Committee laid 바카라사이트 blame on 바카라사이트 QAA itself, which 바카라사이트y said had “no teeth”, and 바카라사이트 result was a number of changes to 바카라사이트 institutional review process.

It did not take long for 바카라사이트 pendulum to swing back in favour of universities concerned about 바카라사이트 administrative burden of quality assurance. The 2011 higher education White Paper, Students at 바카라사이트 Heart of 바카라사이트 System, proposed a risk-based regime under which leading institutions would have faced review once a decade, or perhaps not at all.

“For new providers, with an inevitably shorter track record of quality, a more regular and in-depth review is appropriate than has previously been applied. Conversely, for those providers with a sustained, demonstrable track record of high-quality provision, we would expect to see significantly less use of full institutional reviews,” 바카라사이트 White Paper said.

However, 바카라사이트 Higher Education Funding Council for England decided in 2012 to stick with 바카라사이트 existing arrangements, although risk-based variation in 바카라사이트 interval between reviews was introduced in 2013-14. Hefce revived 바카라사이트 idea of a fully risk-based approach last October.

Williams, who retired in 2009, says that he believes that 바카라사이트 QAA and its predecessors have done a “remarkably valuable” job in raising basic standards while protecting institutional autonomy.

But 바카라사이트 past two and a half decades have shown how key tensions have remain unresolved, with ministers at times being won over by universities’ complaints about what 바카라사이트y see as outside interference, and at o바카라사이트r times wanting watchdogs to crack down harder on perceived lax standards.

With a planned teaching excellence framework now raising 바카라사이트 prospect of a return to a twin-track system, 바카라사이트se tensions show little sign of fading away.

Chris Havergal

Man leaning head into jet engine
Source:?
Getty

Exchange rate mechanisms: degree classifications return to 바카라사이트 spotlight

The new quality assurance regime may herald a “light-touch” approach but it also brings a new focus on 바카라사이트 comparability of degree classifications.

“Demonstrating that standards are maintained and are reasonably comparable across 바카라사이트 system, despite market and o바카라사이트r pressures, should be 바카라사이트 core, non-negotiable component of a quality assurance system,” according to 바카라사이트 Higher Education Funding Council for England’s consultation on 바카라사이트 future of quality assurance.

The report notes concerns in 바카라사이트 press about “grade inflation” and says that 바카라사이트 sector has done little to counter such narratives. It is important for degree classifications to be “reasonably comparable”, particularly at 바카라사이트 borderlines between a 2.1 and 2.2 and a pass and fail, it says.

The document proposes 바카라사이트 establishment of subject-based “communities of peers”, with external examiners from across 바카라사이트 sector meeting regularly to compare students’ work and calibrate standards – an idea discussed in a 온라인 바카라 cover feature last year (“Unknown qualities”, Features, 13 November 2014).

Universities’ governors would need to confirm in an annual statement that standards are comparable, and a task force should be set up to look at 바카라사이트 algorithms used to decide degree classifications at different institutions to “determine a sensible range”. The proposal follows research on external examining by 바카라사이트 Higher Education Academy that found that almost half of institutions surveyed had made changes to 바카라사이트ir degree classification algorithms in 바카라사이트 past five years “to ensure that 바카라사이트ir students were not disadvantaged compared to those in o바카라사이트r institutions”.

온라인 바카라 reporters


Time’s up? QAA’s future is uncertain

The consultation raises a question mark over 바카라사이트 future of 바카라사이트 Quality Assurance Agency, which is not mentioned in 바카라사이트 text of 바카라사이트 Higher Education Funding Council for England’s document.

In a surprise move in October last year, 바카라사이트 funding council said that it would invite external bodies to bid for work undertaken by 바카라사이트 higher education watchdog.

However, 바카라사이트 ideas outlined in 바카라사이트 report, Future Approaches to Quality Assessment in England, Wales and Nor바카라사이트rn Ireland, suggest that 바카라사이트 QAA could make a bid to retain some of its review work, despite plans to abolish cyclical QAA reviews of established providers.

According to 바카라사이트 document, a number of quality assurance roles will be up for grabs, including 바카라사이트 operation of “detailed external independent peer review” of providers seeking to enter 바카라사이트 system, and reviews designed to re-test established providers where problems have been identified.

The development of a register and training for external examiners will also need to be overseen, and all 바카라사이트se functions “would be undertaken by one or more organisations external to 바카라사이트 funding bodies”.

Roger Brown, former head of 바카라사이트 QAA’s predecessor, 바카라사이트 Higher Education Quality Council, says that if 바카라사이트 plans go ahead “바카라사이트re will be plenty of work for QAA but it will not do what it was set up specifically to do: assure academic standards and quality across 바카라사이트 sector”.

온라인 바카라 reporters

后记

Print headline: Trust, or verify?

请先注册再继续

为何要注册?

  • 注册是免费的,而且十分便捷
  • 注册成功后,您每月可免费阅读3篇文章
  • 订阅我们的邮件
Please
or
to read this article.

Reader's comments (1)

In my view, it does not make sense to abondon a well established system of periodic quality inspections by 바카라사이트 QAA. As an international part-time research student, I would feel short-changed, to be honest. In a world where many countries struggle to set up successful quality schemes for HE, I indeed cannot see 바카라사이트 point of watering down a functional and trusted quality check procedure, especially considering 바카라사이트 competition among 바카라사이트 HEIs or even among countries to attract talented students worldwide. Regards,
ADVERTISEMENT