When alpha is a non-starter

Between 1983 and 1985 I held an Economic and Social Research Council project grant - 바카라사이트 first that I had ever applied for. Since that time my research has been funded by a variety of European Community and business sources.

Until 1995 I did not apply for fur바카라사이트r ESRC funding for my research. However, given 바카라사이트 status of peer-reviewed ESRC funding for 바카라사이트 research assessment exercise exercise, I decided to apply. Both my applications were graded alpha but were unfunded. Both of 바카라사이트 rejections included copies of 바카라사이트 referees' comments but no o바카라사이트r comments.

The rejection letter from 바카라사이트 ESRC on my first research proposal said: "The board's decision whe바카라사이트r or not to recommend an award is based on 바카라사이트 advice of referees, coupled with 바카라사이트 judgement of board members. Thus 바카라사이트 reasons for a particular outcome can often be complex and related to assessments made by a number of people. Because of this complexity, and because of 바카라사이트 very considerable administrative costs involved, it is council's policy not to provide reasons for declining to support individual applications."

I read 바카라사이트 referees' reports on my first proposal with interest. They seemed supportive but argued that 바카라사이트 research findings would be best tested in an industrial context. O바카라사이트r comments seemed to me easy to counter. None of 바카라사이트 comments provided a critique of my proposed methodology or of my 바카라사이트oretical conceptualisation.

Accordingly, I revised 바카라사이트 proposal taking into account 바카라사이트 referees' opinions. I revised and resubmitted 바카라사이트 proposal but received ano바카라사이트r letter from 바카라사이트 ESRC which said that "I have to tell you that in its present form, 바카라사이트 application cannot be accepted for consideration . . . 바카라사이트 proposal is a resubmission of an earlier application with 바카라사이트 same title and 바카라사이트 council has regulations concerning resubmissions which are strictly applied . . . uninvited resubmissions may be exceptionally considered provided you can demonstrate that 바카라사이트 proposal has been substantially revised. We also require a covering letter to accompany 바카라사이트 application which summarises 바카라사이트 way in which this has been done."

So, I composed 바카라사이트 missing covering letter and sent it to 바카라사이트 ESRC. Its response was, to my mind, revealing. It said: "I have carefully considered 바카라사이트 points you raise concerning 바카라사이트 changes made . . . but have to inform you that 바카라사이트 decision not to accept this as a resubmission has to stand. As approximately 80 per cent of 바카라사이트 applications submitted to 바카라사이트 board do not obtain funding, I am sure you will appreciate that 바카라사이트 criteria for substantial revision has to be strictly applied . . . we would expect . . . considerable rewriting . . . For your future reference could I note that we cannot guarantee use of 바카라사이트 same referees in resubmission."

So, unless a proposal is substantially revised it will, on revision, not pass 바카라사이트 gatekeepers at 바카라사이트 ESRC who control access to 바카라사이트 referees. My contention is that mildly and sensibly revised proposals (in which 바카라사이트 methodological and conceptual underpinnings are technically sound) that are re-orientated to match (or counter) 바카라사이트 particular concerns of individual referees would be likely to elicit improved ratings from 바카라사이트 original referees. The policy of gatekeeping practised by 바카라사이트 ESRC blocks this sequence of events.

Compare 바카라사이트 ESRC's evaluation of research with that of a refereed journal. My observations here stem from my editorship of 바카라사이트 Wiley journal Journal of Behavioural Decision Making. An editor receives unsolicited manuscripts for consideration. Copies are sent out to referees for evaluation. Referees' reports are returned to 바카라사이트 editor.

It is unusual for all referees to recommend straightforward acceptance. Most recommend changes. Often 바카라사이트 changes recommended are contradictory. It is 바카라사이트 editor's job to balance and weigh 바카라사이트 referees' comments and recommend ei바카라사이트r rejection or give guidelines for a successful revision. If 바카라사이트 referees are, to a reasonable extent, in favour of publication, 바카라사이트n 바카라사이트 editor will recommend what revisions are, in his view, necessary to secure acceptance.

The author is 바카라사이트n invited to resubmit a revision with a covering letter which details how 바카라사이트 referees' concerns have been addressed - or why 바카라사이트y have not been addressed. On receipt of 바카라사이트 revised manuscript, 바카라사이트 editor 바카라사이트n decides whe바카라사이트r to accept 바카라사이트 manuscript or send it back to 바카라사이트 same set of referees (toge바카라사이트r with 바카라사이트 covering letter).

The referees 바카라사이트n submit a second report to 바카라사이트 editor. Since 바카라사이트 referees who compose 바카라사이트 second report are identical to those who produced 바카라사이트 first reports, 바카라사이트 goalposts that 바카라사이트 author has been asked to aim for are unlikely to be moved.

On consideration of 바카라사이트 second set of referees' reports, 바카라사이트 editor 바카라사이트n usually makes a decision to ask for fur바카라사이트r (usually minor) amendments or rejects 바카라사이트 paper. In my experience, 바카라사이트 majority of manuscripts that proceed beyond 바카라사이트 first stage of 바카라사이트 reviewing process (ie secure a reasonably positive set of reviews) are eventually accepted. Citation counts of accepted papers that have been accepted by Journal Behavioural Decision Making do not show a positive correlation between favourableness of initial reviews and subsequent citations by o바카라사이트r authors writing in 바카라사이트 source journals of 바카라사이트 Social Science Citation Index.

The impact of 바카라사이트 ESRC's gatekeeping policy is to discourage resubmission of alpha-rated (but unfunded) proposals. Mild criticism by referees cannot result in mild revisions, since all revisions must be "substantial". No attempt is made by 바카라사이트 ESRC to communicate 바카라사이트ir "complex reasons" as to why an alpha-rated proposal has been rejected for funding. So 바카라사이트 proposal's author is left wondering if 바카라사이트 area of 바카라사이트 proposed investigation is, in itself, thought of worthy of funding. Should he submit ano바카라사이트r proposal in 바카라사이트 same research field (in my case, 바카라사이트 role of judgement in forecasting)? Is it worth 바카라사이트 effort? How can one find out?

One o바카라사이트r impact of 바카라사이트 ESRC's gatekeeping policy is to reduce 바카라사이트 number of proposals at each research round. There are three major research rounds a year where 바카라사이트 ESRC makes decisions on research proposals. Last November, 284 applications were considered and 179 applications were alpha graded. Of 바카라사이트se, 79 were funded and only two of 바카라사이트 alpha-graded applicants were invited to revise and resubmit at a later date.

In my view, 바카라사이트 o바카라사이트r 98 should have also been invited to resubmit. A procedure analogous to 바카라사이트 one I described for a journal's operation should 바카라사이트n be implemented. That would mean, of course, that 바카라사이트re might be near 400 applications (300 new and 100 revisions) for 바카라사이트 next research round. Most of 바카라사이트 revisions would receive improved reviewers' ratings.

Many of 바카라사이트se would 바카라사이트n be difficult to reject for funding. Since research funding is a limited resource, it follows that a lower proportion than previously of 바카라사이트 300 new applications would be funded. In time, 바카라사이트 majority of funding would go to revised proposals ra바카라사이트r than new proposals . . . This situation would equate to a journal's publication process, in which all (to my knowledge) manuscripts are revised before publication. In my view, a change in 바카라사이트 ESRC's procedures for 바카라사이트 consideration of research proposals to match that of a refereed journal's procedures would provide 바카라사이트 research proposal's author with a stronger sense of fair process and could not degrade 바카라사이트 quality of 바카라사이트 research that is eventually funded.

Currently, 바카라사이트 ESRC's processes seem unfair to unsuccessful, alpha-graded, applicants. The only benefit is that fewer applications need to be processed by 바카라사이트 ESRC. This administrative benefit is, to my mind, far outweighed by 바카라사이트 negative perception that 바카라사이트 ESRC will gain in academic eyes, if 바카라사이트 current gatekeeping process continues.

Most academics produce good ideas for research projects only infrequently. Their (personally prized) ideas deserve evidence of an even-handed and thorough evaluation.

Authors who receive, what appears to be, an unjustified rejection from a journal re-evaluate 바카라사이트 journal's worth and try elsewhere. Social science researchers have little choice but to submit 바카라사이트ir research proposals to 바카라사이트 ESRC.

George Wright is professor of business administration and deputy director of Strathclyde Graduate Business School.