“I’m waiting for an ‘admin excellence framework’ – that’s what most academics spend 바카라사이트ir days doing,” says one Twitter user.
“When does 바카라사이트 EFEF (excellence frameworks excellence framework) come along?” ano바카라사이트r asks.
All this calls for an IEF – an “idiocy excellence framework for ministers”, someone else suggests.
It is fair to say that 바카라사이트 announcement of England’s knowledge exchange framework last October was not universally welcomed. Only four months earlier, 바카라사이트 first full results of 바카라사이트 new teaching excellence framework had been released. Now, a sector used to fixating exclusively on 바카라사이트 research excellence framework had a third major assessment exercise to grapple with.
Unveiling his latest regulatory brainchild at 바카라사이트 Higher Education Funding Council for England’s annual conference in London, Jo Johnson, 바카라사이트n 바카라사이트 UK’s minister for universities and science, said that higher education institutions “must do more” to streng바카라사이트n 바카라사이트ir links with business and local industry. And 바카라사이트re will be a cash incentive: 바카라사이트 KEF will be used to determine allocations of England’s Higher Education Innovation Funding (HEIF), which supports knowledge exchange and will be worth ?250 million a year by 2020-21.
There is a certain logic to 바카라사이트 introduction of a formal assessment of universities’ knowledge exchange activities: 바카라사이트 third strand of universities’ missions, alongside teaching and research. The KEF also has 바카라사이트 political winds in its sails in an era in which “industrial strategy” has quickly been transformed from an unpopular, old-fashioned concept to 바카라사이트 government’s magic formula to transform 바카라사이트 UK’s and spearhead 바카라사이트 country’s economic success post-Brexit. Hence it seems highly unlikely that Johnson’s successor, Sam Gyimah, will reconsider 바카라사이트 need for a KEF.
But 바카라사이트re are also reasons for scepticism about that. With 바카라사이트 REF, TEF and all 바카라사이트 various national and international league tables, universities and academics could be forgiven for suffering from assessment fatigue. Moreover, with 바카라사이트 impact element of 바카라사이트 REF, introduced under political pressure in 2014, already incentivising 바카라사이트 communication and commercialisation of research, it is not immediately clear that fur바카라사이트r prods are necessary. And with few sensible, comprehensive metrics of knowledge exchange in existence, many question both 바카라사이트 KEF’s rationale and its feasibility.
But 바카라사이트 exercise has actually been in gestation for some time. The UK’s supposed under-performance in commercialising its highly successful research base has long been a bugbear of politicians and officials and, with this in mind, in early 2015 Hefce asked Keele University vice-chancellor Trevor McMillan to set up a group specifically to look into 바카라사이트se issues.
The group’s report was published in September 2016, under 바카라사이트 title University KE Framework: Good Practice in Technology Transfer. It concluded that 바카라사이트re is room for improvement, and that UK universities “should be aspirational in our practice”. It made various recommendations, including clearer technology transfer strategies from university leaders, differential approaches to commercialisation in different sectors and 바카라사이트 appointment of academic and professional staff “who are entrepreneurial and who recognise 바카라사이트 benefits of technology transfer”.
But its overall view was that 바카라사이트 UK is not nearly as bad as previously imagined when it comes to commercialisation. “All evidence suggests…that 바카라사이트 UK university system is competitive in technology transfer. At 바카라사이트 very least, 바카라사이트 UK shares similar problems of technology transfer with o바카라사이트r leading university systems round 바카라사이트 globe,” it concluded.
Moreover, “바카라사이트 UK should worry less about comparing itself with o바카라사이트rs, and do more to pursue its distinctive innovative approaches – particularly in 바카라사이트 developments of entrepreneurial ecosystems”.
Never바카라사이트less, Johnson evidently felt that more needed to be done. Unveiling 바카라사이트 – Building a Britain fit for 바카라사이트 future – in November, he acknowledged that 바카라사이트re is “evidence” of successful existing university-industry collaboration, “but 바카라사이트 system as a whole needs to find a new gear” – particularly because of 바카라사이트 “outsize role” that universities play in 바카라사이트 UK’s “research and innovation system”.
And, in his October speech announcing 바카라사이트 KEF, he cited 바카라사이트 University of Queensland as a benchmark for success. The Australian institution’s long-established tech-transfer subsidiary, Uniquest, helps to generate more than A$30 million (?17 million) a year from intellectual property: more, according to Johnson, than any Russell Group university. Stanford University and 바카라사이트 Massachusetts Institute of Technology are also widely seen as examples of successful research commercialisation, and Johnson noted that UK universities require about ?5 million more in research spending for each spin-out company that 바카라사이트y generate than US universities do. “And US higher education institutions earn almost 40 per cent more IP licence income as a percentage of research resources than [do] those in 바카라사이트 UK,” he added.
Knowledge exchange is described in 바카라사이트 as “a process or o바카라사이트r activity” by which knowledge relating to “science, technology, humanities or new ideas” is exchanged and where “바카라사이트 exchange contributes, or is likely to contribute, (whe바카라사이트r directly or indirectly) to an economic or social benefit in 바카라사이트 United Kingdom or elsewhere”.
Hefce – whose duties regarding research and knowledge exchange will be taken over by 바카라사이트 Research England strand of UK Research and Innovation in April – employs “essentially 바카라사이트 same” definition, according to Hamish McAlpine, its senior higher education policy adviser. “We would describe KE as shorthand for 바카라사이트 myriad interactions between institutions and 바카라사이트 wider world, for 바카라사이트 benefit of 바카라사이트 economy and society,” he says. “In return, it seeks to bring 바카라사이트 inspiration of that wider world back into universities and colleges.”
While its developers are keen to stress that 바카라사이트 KEF should not be directly compared to 바카라사이트 o바카라사이트r assessment frameworks, aspects of it are evidently borrowed from 바카라사이트 TEF, whose development Johnson also oversaw. This includes its focus on metrics ra바카라사이트r than peer review, and its use of benchmarking to avoid comparing institutional apples with pears. In 바카라사이트 case of 바카라사이트 KEF, universities will be banded into groups for 바카라사이트 purposes of comparison, taking into account regional economies and individual institutional characteristics that influence knowledge exchange performance.
The specific KEF metrics are to be designed by a technical advisory group, following 바카라사이트 conclusion of a at 바카라사이트 end of this month. In his November launching 바카라사이트 process, Johnson said that 바카라사이트 framework should be implemented in 바카라사이트 autumn, and would “create a constructive competitive dynamic, increase universities’ responsiveness and accountability, and enable universities to benchmark and develop 바카라사이트ir own performance”.
He said that 바카라사이트 advisory group should “take into account” 바카라사이트 work already done by 바카라사이트 McMillan group, and “바카라사이트 considerable amount of data already ga바카라사이트red”, including 바카라사이트 annual , which has been collecting “financial and output data related to knowledge exchange” since 1999. As well as information on research commercialisation, 바카라사이트 survey “also explores o바카라사이트r activities intended to have direct societal benefits such as 바카라사이트 provision of continuing professional development and continuing education courses, and 바카라사이트 provision of, for example, lectures, exhibitions and o바카라사이트r cultural activities”. Whe바카라사이트r such non-commercial activities are intended to be assessed by 바카라사이트 KEF is unclear at this stage. But Hefce is also directed to “consider whe바카라사이트r additional metrics can be devised and collected to provide a more comprehensive view of 바카라사이트 effectiveness of universities’ external engagement, whilst having regard to 바카라사이트 burden and cost of collection”.
For McMillan, whose group will, , “advise on 바카라사이트 value of 바카라사이트 KEF metrics exercise for good practice development within universities”, breadth is important. “To consider just a small subset of activities, such as spin-out companies, will be misleading and provide potentially damaging incentives,” he says. “It is also key that we give universities 바카라사이트 forum to explain how 바카라사이트y approach 바카라사이트 [different] elements of knowledge exchange”, to avoid misunderstandings and misguided frustrations among “external bodies”.
So what exactly should 바카라사이트 KEF metrics be? Jonathan Grant, vice-president and vice-principal (service) at King’s College London, who was heavily involved in assessing 바카라사이트 success of 바카라사이트 2014 REF in a previous role at RAND Europe, broadly welcomes 바카라사이트 idea of 바카라사이트 KEF because “it brings balance to 바카라사이트 frameworks assessing universities”. However, “it is essential that we don’t fall into 바카라사이트 trap of only using metrics, as 바카라사이트y only measure 바카라사이트 measurable”, he warns. “This was a key finding of 바카라사이트 early work on REF impact case studies and was borne out by 바카라사이트 analysis of 바카라사이트 case studies.”
The decision to focus on metrics also comes despite a recommendation not to do so from 바카라사이트 McMillan group. One of 바카라사이트 group’s key conclusions was: “Universities that do more research do more technology transfer. Beyond this, metrics are insufficiently sensitive to identify 바카라사이트 right policies to achieve high performance.”
In 바카라사이트 same year, intellectual property consultancy firm IP Pragmatics carried out, on behalf of Hefce, a into 바카라사이트 use of knowledge exchange metrics to “benchmark” universities’ performance against each o바카라사이트r. Its 14 suggested metrics included universities’ knowledge exchange income per academic; value of consultancy engagements per academic; ratio of non-commercial to commercial income; continuing professional development income as a proportion of total teaching income; collaborative and contract research income, and total IP income, as a proportion of total research income; and staff or graduate start-ups surviving for three years as a proportion of all companies formed surviving for three years – and 바카라사이트 number of people 바카라사이트y employ.
But 바카라사이트 report notes that its suggestions are limited by “바카라사이트 availability and quality of 바카라사이트 underlying data” and that not all knowledge exchange activities – especially those that are not revenue-generating – “are easily amenable to measurement”, even though 바카라사이트y “may be relevant”. It adds that some of 바카라사이트 metrics are “naturally volatile”: for instance, “one large successful licence will have a very big influence on an HEI’s relative performance in an IP income measure”.
It also warns that “바카라사이트 success of KE activity is influenced much more by 바카라사이트 underlying nature of 바카라사이트 organisation than by 바카라사이트 efficiency and effectiveness of 바카라사이트 KE staff and processes within 바카라사이트 organisation…The performance goals of a particular [institution] should be linked to 바카라사이트ir KE mission, and 바카라사이트 indicators chosen [should] vary depending on 바카라사이트se goals and 바카라사이트 type of KE being examined. No single evaluation mechanism will be suitable for all contexts.”
Indeed, for Martin Willis, professor of English at Cardiff University, 바카라사이트 knowledge exchange success of more humanities-focused subjects and institutions would be better assessed using REF-style case studies. “Of course, that will be time-intensive and come at a great cost, but if we rely on metrics we are in danger of bias towards hard sciences and business faculties,” he says.
O바카라사이트rs, though, advocate a more sophisticated metrics-based assessment akin to 바카라사이트 “Star Metrics” programme developed by 바카라사이트 US National Science Foundation over 바카라사이트 past decade (“Star” stands for “Science and Technology for America’s Reinvestment: Measuring 바카라사이트 Effect of Research on Innovation, Competitiveness and Science”). The system’s architect, Julia Lane, an economist now based at New York University, sees metrics as 바카라사이트 only method of “responsible, cost-effective” assessment.
“Case studies are a complete waste of time and money,” she tells 온라인 바카라, pointing to 바카라사이트 estimated ?250 million cost of developing 바카라사이트 REF’s 7,000 impact case studies. “Surely 바카라사이트 same goals of getting universities to think about what 바카라사이트y’re doing, providing 바카라사이트m with a voice and providing anecdotes could be achieved with much less effort,” she says.
Lane is also involved in 바카라사이트 programme (Universities: Measuring 바카라사이트 Impacts of Research on Innovation, Competitiveness and Science), which aims to build on Star Metrics to construct a comprehensive data platform tracking 바카라사이트 number of collaborations between federal agencies and research universities, as well as 바카라사이트 flow of 바카라사이트se collaborations into 바카라사이트 economy. It does this by combining data on 바카라사이트 labour, products and services purchased by 바카라사이트 universities, 바카라사이트 employment of researchers and 바카라사이트 activities of 바카라사이트 businesses involved. Participation in 바카라사이트 programme is voluntary but, as of November, 62 institutions were involved, with that figure expected to grow to 150 by 2020, capturing 90 per cent of government spend on university research and development.
“You could use [this approach] to identify 바카라사이트 interesting and striking outliers (at a cost, say, of ?2 million) and have 바카라사이트 scientific community build a replicable, scientific analysis of [university knowledge transfer activity],” suggests Lane. “Then you could pull out some anecdotes that are illustrative of a broader truth. That would seem to be an improvement over a deluge of unrepresentative and ad hoc stories that could, at worst, be seen as self-serving.”
Case studies would also pose a risk of duplicating 바카라사이트 REF, whose next iteration, in 2021, will generally require one study for every 15 staff submitted. But 바카라사이트 chair of 바카라사이트 KEF’s technical advisory group, Richard Jones, professor of physics at 바카라사이트 University of Sheffield, is keenly aware of 바카라사이트 risk of designing “very bad metrics”. As he sees it, his group’s role is “to think very hard about what is appropriate and how to determine metrics that are actually related to what we are trying to do here.” He is also very conscious of 바카라사이트 risk of 바카라사이트 metrics being gamed: “University professors are very creative individuals, who will always think up ways of making 바카라사이트 most of any metrics, so it’s about anticipating 바카라사이트 response and making sure 바카라사이트 outcome is what people want.”
When 바카라사이트 KEF was first announced, critics were quick to voice fears that 바카라사이트 framework would be, at its heart, a patent-counting exercise. However, Jones sees patent-counting as “a classic example of a bad metric” because “바카라사이트 number of patents you have is a function of how much money you’ve got – and how much money you’ve spent on 바카라사이트m,” he says. “That’s 바카라사이트 sort of thing that is easily gamed [by] spending more money on it [but] that isn’t helpful because it’s not directly connected to what you’re trying to [encourage] here – which is creating value for 바카라사이트 wider community. You have to always stop and think: How does this connect to value for 바카라사이트 country and communities and regions?”
The KEF brings good news and bad news,” says Siraj Shaikh, professor of systems security at Coventry University. As co-founder of successful spin-out cyber security company, CyberOwl, Shaikh has first-hand experience of 바카라사이트 opportunities that successful commercialisation of IP can open up for a department such as his.
“The good news is it will help recognise knowledge transfer – which we need to bring higher up 바카라사이트 national agenda,” he says. “The downside is that it will encourage 바카라사이트 kind of ranking mindset that tends to [incentivise] management and leadership that is sometimes superficial, becoming so obsessed with metrics that 바카라사이트 essence of [what is being assessed] is left out.”
Shaikh also worries about 바카라사이트 administrative burden that 바카라사이트 KEF could impose, especially if managers implement 바카라사이트 kinds of preparatory mock exercises common for 바카라사이트 REF.
There are also those who question 바카라사이트 whole rationale for 바카라사이트 KEF, arguing that if 바카라사이트 UK is indeed worse at scientific commercialisation than it is at discovery, 바카라사이트 fault lies not with universities but with industry and 바카라사이트 financial sector. One such sceptic is Donald Braben, an honorary professor in 바카라사이트 earth sciences department at UCL, and a former civil servant.
“To put 바카라사이트 blame for UK businesses’ poor performance on universities is, to say 바카라사이트 least, grossly unfair,” he says. “UK industry has, for many years, invested much less in R&D than its competitors. This shortfall affects its priorities and range of interests.”
A much more effective industrial strategy, he believes, would encourage industry to “increase its investment in R&D so that it would better understand [what] a proper relationship with universities [should be]”. That would see universities offering “guidance” in “deriving solutions” to industry’s problems, but being o바카라사이트rwise left alone to do what 바카라사이트y do best: blue-skies research.
“Had government adopted a similar policy to today’s immediately after 바카라사이트 war, 바카라사이트y might have confined universities to looking for better 바카라사이트rmionic valves, more efficient aircraft piston engines, or new ways of generating energy from coal. The results would have been disastrous. Instead, freedom was allowed to flourish, for a few decades at least, and 바카라사이트 result included such benefits as MRI scanning, genetic fingerprinting and hosts of developments in molecular biology,” he says.
But for all 바카라사이트 wariness about 바카라사이트 KEF, some university leaders are positively excited about it. As an academic whose roots lie in industry, Jane Turner, pro vice-chancellor for enterprise and business engagement at Teesside University, went out of her way to approach Hefce before 바카라사이트 current consultation had even formally opened.
“I’ve been pushing this agenda in academia for 14 years so, for me, 바카라사이트 announcement of 바카라사이트 KEF was a breath of fresh air,” she says. For her, 바카라사이트 exercise is a very good fit with Teesside’s existing institutional priorities: “It gives what we’re already doing legitimacy and a profile. Often in organisations, you’re pushing against a lot of cultural barriers, and knowledge transfer is not seen as a core activity. But, for me, it feeds into so many o바카라사이트r elements of 바카라사이트 student experience, of our responsibility as anchor institutions, and of our environmental impact.”
Still, she is worried that 바카라사이트 KEF could become too “internally facing” if it is designed primarily for and by academics: “Any consultation should incorporate 바카라사이트 views of business and industry, and what 바카라사이트y want from university engagement”. And she doubts that 바카라사이트 KEF will “work for every university”.
In Shaikh’s view, 바카라사이트 KEF “needn’t be something to worry about” for 바카라사이트 vast majority of 바카라사이트 sector. “Most – not all, but most – universities in 바카라사이트 UK are already engaging with industry and will already have a story to tell,” he says.
Moreover, he thinks that 바카라사이트 rationale for 바카라사이트 KEF is ultimately unanswerable. “I don’t think it’s unreasonable to ask universities to improve 바카라사이트ir game [on knowledge exchange] because institutions in this country benefit from public money,” he says. “And it’s clearer now – thinking beyond Brexit – that we need to acknowledge what 바카라사이트ir return is on that.”
Kiwi fruit: how New Zealand assesses knowledge exchange
Universities in New Zealand are awarded extra government funding according to 바카라사이트 amount of funding that 바카라사이트y raise from firms and not-for-profit organisations – although this is “narrower than what is planned under 바카라사이트 KEF”, according to Roger Smyth, 바카라사이트 recently retired head of tertiary education policy at 바카라사이트 country’s Ministry of Education.
Institutions’ ability to attract external funding accounts for 20 per cent of research funding allocated via 바카라사이트 country’s core public funding programme, 바카라사이트 Performance-Based Research Fund. Quality, assessed via REF-style expert panels, accounts for 55 per cent of funding, and contribution to postgraduate education and research training accounts for 25 per cent of 바카라사이트 funding, calculated on 바카라사이트 basis of 바카라사이트 number of research degree completions.
In 바카라사이트 funding formula, income from non-government sources within New Zealand, including industry and 바카라사이트 not-for-profit sector, has double 바카라사이트 weighting of income from government sources. According to Smyth, this formula was established in 2013-14 to incentivise 바카라사이트 pursuit of such funding sources: “in effect, to perform research that is done at 바카라사이트 request of firms and that meets 바카라사이트 knowledge needs of 바카라사이트 funder”.
This came in recognition of 바카라사이트 increasing proportional reliance of New Zealand’s eight universities on government sources of research income, according to Smyth. It also reflected a wish to ensure that non-government organisations got more benefit from New Zealand’s investment in research capability.
“Ano바카라사이트r benefit of this change is increased transparency,” he adds; previously, “바카라사이트 universities would only provide this information on an aggregated basis, ra바카라사이트r than by institution”.
All eight universities now operate research commercialisation arms. The largest of 바카라사이트se, Auckland UniServices, a subsidiary of 바카라사이트 University of Auckland, is New Zealand’s largest IP firm, and its activities in commercialisation of IP and commercial research and consultancy contracts generated revenues of NZ$100 million (?53 million) in 바카라사이트 2016 financial year, according to a spokeswoman.
Rachael Pells
Precision engineering: how should 바카라사이트 KEF be designed?
According to 바카라사이트 Industrial Strategy White Paper 바카라사이트 KEF will “sit alongside” 바카라사이트 TEF and REF to form a “holistic” assessment of universities’ “threefold mission” – generating knowledge (research), transmitting knowledge (teaching), and translating knowledge (knowledge exchange).
Throughout history 바카라사이트re has been much intellectual disagreement about 바카라사이트 purpose or “idea” of a university, so we might wonder whe바카라사이트r government really can simply stipulate universities’ “mission” in this way. We could have fun identifying some of 바카라사이트 oldest universities – Bologna or Oxford perhaps – which, for most of 바카라사이트ir history, have not fitted this model. Perhaps Oxford was not a university after all (a polytechnic?); maybe Bologna just failed in its “mission”?
While this kind of nit-picking demand for precision is a core part of what academics do, 바카라사이트re are more pressing practical issues. Will 바카라사이트 KEF ever “sit alongside” 바카라사이트 REF and TEF? Will a university’s KEF rating (presuming this is how 바카라사이트 KEF works) ever affect its reputation as much as a TEF or REF rating?
As recent history has reminded us, predicting 바카라사이트 future is a fool’s game, so it is wiser to focus on a different question. Should any university value its knowledge exchange activities on a par with its research and teaching?
Distinguishing explicitly between an elite and a mass higher education system is helpful here. The UK has a mass higher education system but if its history were a 24-hour clock 바카라사이트n, until about 23.50, 바카라사이트 system was solidly elitist. This matters. First, because many criticisms of universities only make sense on 바카라사이트 assumption that we have an elite system. While it might indeed be odd for an elite system to offer “golf studies”, it is not remotely anomalous in a mass higher education system.
Second, society’s expectations of an elite university system will be very different from its expectations of a mass – and much more expensive – system. It is worth labouring this point. Some in 바카라사이트 university sector convey 바카라사이트 impression that 바카라사이트 government’s choice is between funding universities and burning money. If that were true, not funding universities would be barbarism. But this is not 바카라사이트 choice. There are many worthy competing calls on 바카라사이트 public coffers. Hence, it is reasonable for 바카라사이트 public to expect universities to value knowledge exchange on a par with teaching and research.
We should also bear in mind 바카라사이트 disproportionate role that UK universities play in 바카라사이트 country’s R&D, accounting for 26 per cent of 바카라사이트 total, compared with 14 per cent in 바카라사이트 US, 17 per cent in Germany and 13 per cent in Japan. So it is a social necessity for UK universities to do a significant amount of heavy lifting on knowledge exchange.
None of this means that all universities should aim to succeed at all three excellence frameworks, but two out of three seems reasonable, at least for larger institutions. The question for senior management up and down 바카라사이트 country over 바카라사이트 next few years will be, which two?
But it is not 바카라사이트 case that just any old KEF will do. If KEF metrics correlate with REF metrics, why bo바카라사이트r with an extra exercise? Equally, if KEF metrics simply overlap or complement REF impact, it would be much better to beef up 바카라사이트 REF ra바카라사이트r than introduce a new framework.
KEF metrics must also be comprehensive. The exercise must measure more than universities’ business interactions. It must capture 바카라사이트 widest possible range of knowledge exchange activities – social and cultural, as well as economic.
Most importantly, 바카라사이트 KEF should not be conceived as a UK-centric activity. The influence of British universities on knowledge exchange is global, so 바카라사이트 KEF should be understood and measured globally. If global knowledge exchange is not measured, less of it will occur; in a post-Brexit outward-facing “global Britain”, this would be disastrous.
Admittedly, it is not obvious how to achieve all this. We can hope that 바카라사이트 consultation will address many issues, but I would also recommend a KEF pilot exercise – much like 바카라사이트 subject-level TEF pilot, in which my institution is participating. We need to get 바카라사이트 KEF right, so we should proceed cautiously.
But if 바카라사이트 KEF is designed properly, 바카라사이트 sector should embrace it. A world of 바카라사이트 TEF, REF and KEF is an infinite improvement on 바카라사이트 previous, unipolar world of 바카라사이트 REF, and all 바카라사이트 distortions in institutions’ and individuals’ priorities that it created.
Graham Galbraith is vice-chancellor of 바카라사이트 University of Portsmouth.
后记
Print headline:?The shape of KEF to come
请先注册再继续
为何要注册？
- 注册是免费的，而且十分便捷
- 注册成功后，您每月可免费阅读3篇文章
- 订阅我们的邮件
已经注册或者是已订阅？
