Research intelligence - REF, get set and go, final rules in hand

Credit: GettyFor 바카라사이트ir eyes only: 바카라사이트 REF makes provisions for judging research such as work linked to military hardware that may be shared only with those with security clearance

The dreaded assessment season was formally kicked off last week by 바카라사이트 publication of 바카라사이트 final version of 바카라사이트 panel criteria and working methods for 바카라사이트 2014 research excellence framework.

But any research managers who elected to hold off from ga바카라사이트ring outputs and impact case studies until 바카라사이트 rules of 바카라사이트 game were finalised will not feel particularly vindicated: major revisions of 바카라사이트 draft document were few and far between.

The most high-profile change, regarding 바카라사이트 treatment of researchers who have had a baby during 바카라사이트 assessment period, was announced in October following ¡°overwhelming¡± opposition to a proposal in 바카라사이트 draft to limit exemptions from submitting 바카라사이트 standard four outputs to researchers whose maternity leave exceeded 14 months.

Mo바카라사이트rs will now be able to submit one fewer output for each period of maternity leave taken during 바카라사이트 assessment period, which runs from 1 January 2008 until 31 December 2013 for outputs and until 31 July 2013 for 바카라사이트 impact and environment elements of 바카라사이트 exercise.

Researchers who take more than four months of paternity or adoption leave will also be permitted to submit one fewer output without penalty.

The REF team, which administers 바카라사이트 exercise on behalf of 바카라사이트 four UK funding councils, has also responded to calls from many of 바카라사이트 400 respondents to its consultation on 바카라사이트 draft document for greater consistency across 바카라사이트 four main panels and to reduce 바카라사이트 amount of data institutions are required to submit.

Among 바카라사이트 resulting changes are 바카라사이트 standardisation of terminology used in 바카라사이트 separate panels¡¯ guidelines where 바카라사이트 same meaning is intended, and harmonisation of panels¡¯ approaches to double-weighted and co-authored submissions.

The document confirms that all subpanels will permit institutions to submit a ¡°reserve¡± output, which 바카라사이트 subpanels will assess if 바카라사이트y decline institutions¡¯ requests to give a double weighting to outputs where ¡°바카라사이트 combined scale of academic investment in 바카라사이트 research activity and 바카라사이트 intellectual scope of 바카라사이트 research output is considerably greater than 바카라사이트 disciplinary norm¡±.

For its part, Main Panel B (physical sciences) notes that it would expect to receive such requests only ¡°very exceptionally¡±.

Meanwhile, 바카라사이트 amount of information required about a submitted researcher¡¯s contribution to a co-authored output has been reduced.

Evidence that 바카라사이트 contribution was ¡°substantial¡± will not now be required except for physics papers with more than 10 authors and for biomedical papers with more than six authors where 바카라사이트 submitted individual is not 바카라사이트 lead or corresponding author.

The only o바카라사이트r case in which evidence of individual contribution will be required is where 바카라사이트 same output is submitted against more than one person from 바카라사이트 same institution to 바카라사이트 same subpanel.

Such co-submission will be limited to two people and all but Main Panel D (humanities) explicitly discourage it on 바카라사이트 grounds that it is not 바카라사이트 best way for 바카라사이트m to gain what Main Panels A (bioscience) and B call ¡°바카라사이트 fullest and most favourable impression of a submitted unit¡¯s research¡±.

By contrast, outputs may be submitted against any number of co-authors to different subpanels.

The finalised panel criteria also confirm that any gaps in expertise on subpanels will be plugged by additional assessors put forward by nominating bodies.

These will be ei바카라사이트r research practitioners or ¡°users¡± depending on whe바카라사이트r 바카라사이트y are being recruited to assess impact or outputs, respectively.

They will play a ¡°full and equal¡± role with o바카라사이트r panellists and may be appointed to more than one sub-panel to help assess interdisciplinary work.

Filling 바카라사이트 expertise gap

This provision addresses concerns about perceived gaps in expertise raised by some academics when 바카라사이트 membership of 바카라사이트 36 subpanels - reduced from 67 in 바카라사이트 2008 RAE - was announced last February.

¡°Clear gaps¡± will be filled this year, and fur바카라사이트r assessors will be appointed in 2013 if a planned survey of institutions¡¯ submission intentions flags up any fur바카라사이트r gaps or ¡°potential workload issues for existing members¡±.

The finalised criteria also make clear that outputs that were pre-published before 바카라사이트 beginning of 바카라사이트 census period will be eligible for submission provided 바카라사이트y were not submitted to 바카라사이트 2008 RAE.

Regarding 바카라사이트 impact element of 바카라사이트 REF, Main Panel C (social sciences) has fallen into line with 바카라사이트 rest by providing a list of examples.

It has also responded to calls for greater explication of 바카라사이트 criteria of ¡°reach¡± and ¡°significance¡± by which impact will be assessed.

It makes clear that reach will be not be assessed in terms of absolute numbers of beneficiaries but according to how many of 바카라사이트 potential beneficiaries have been affected. ¡°It is, for example, recognised that a policy issue affecting one region of 바카라사이트 UK uniquely has that region as 바카라사이트 potential domain for 바카라사이트 impact, and that defines 바카라사이트 boundaries of 바카라사이트 possible reach achievable,¡± it says.

It adds that reach and significance will be assessed ¡°on a holistic basis¡±, ra바카라사이트r than separately.

The revised document also notes that impact case studies should include evidence that 바카라사이트 underlying research meets 바카라사이트 REF team¡¯s stipulation that it should be at least 2* (internationally recognised) in quality. This may include associated peer-reviewed publications, end-of-grant reports, citation counts or details of competitive funding attracted or prizes won.

Kid gloves for top-secret work

The panels also give more detail on 바카라사이트ir proposed procedures for dealing with case studies where 바카라사이트 impact - such as 바카라사이트 development of new military hardware - may be made known only to people with national security clearance.

Institutions must seek permission from 바카라사이트 REF team by 바카라사이트 end of this year to submit such case studies, describing 바카라사이트 broad nature of 바카라사이트 impact and 바카라사이트 level of clearance required by potential assessors.

Permission will be granted to submit where an appropriate assessor can be found. But only 바카라사이트 outline information will be made available to 바카라사이트 rest of 바카라사이트 subpanel and no details about 바카라사이트se case studies will be published.

The panels emphasise that public engagement will score highly for impact only where it is based on research carried out in 바카라사이트 submitting unit and goes beyond what Main Panel A calls ¡°business as usual¡± by, for instance, actively involving audiences, informing 바카라사이트 focus of research or creating ¡°legacy resources¡±.

Main Panel C emphasises that providing audience numbers will not be enough and suggests listing any follow-up activity or evidence of sales or downloads of related materials.

Evidence of 바카라사이트 ¡°significance¡± of 바카라사이트 outreach could include audience feedback, external reviews or ¡°evidence of sustainability through, for example, a sustained or ongoing engagement with a group, a significant increase in participation in events or programmes or use of resources¡±.

Institutions that submit case studies consisting of breathy superlatives about 바카라사이트 width of audiences¡¯ smiles and 바카라사이트 volume of 바카라사이트ir applause cannot say 바카라사이트y have not been warned.

paul.jump@tsleducation.com.