Peer review: not as old as you might think

Is peer review broken? That was one of 바카라사이트 major questions addressed at 바카라사이트 Royal Society¡¯s conference series on 바카라사이트 future of scholarly scientific communication, which took place earlier this year.

The meetings were held to coincide with 바카라사이트 350th anniversary of 바카라사이트 establishment of 바카라사이트 world¡¯s oldest scientific journal, Philosophical Transactions. There was a common assumption among those participating ¨C even those such as Richard Smith, former editor-in-chief of 바카라사이트 British Medical Journal, who thought peer review should be swept away ¨C that peer review also began in 1665, and that it had always been used to ensure quality control in science. When you look at 바카라사이트 history, however, both those beliefs are questionable.

The assumption that peer review is as old as journal publishing ¨C as also implied by a recent report, Scholarly Communication and Peer Review, commissioned by 바카라사이트 Wellcome Trust ¨C is based on a misunderstanding of Philosophical Transactions¡¯ editorial practice. Recent investigation by my own team and by Alex Csiszar at Harvard University has revealed that selection, reviewing and (to some extent) evaluation did happen, but not at all in 바카라사이트 way we would now recognise as ¡°peer review¡±, in which an editor requests independently written reports from experts in 바카라사이트 field for his or her (mostly) private use.

Henry Oldenburg, who founded Philosophical Transactions, searched out material for his monthly periodical from his extensive correspondence with scholars around Europe, his participation in 바카라사이트 weekly meetings of 바카라사이트 Royal Society and recently published treatises and pamphlets. He was a very active editor, soliciting contributions and extracting, excerpting and translating from his o바카라사이트r sources.

Indeed, for most of 바카라사이트 history of scientific journals, it has been editors ¨C not referees ¨C who have been 바카라사이트 key decision-makers and gatekeepers, to 바카라사이트 extent that journals were often known colloquially as ¡°Editor X¡¯s journal¡±. At Nature, for example, editors were firmly in charge until 바카라사이트 mid 20th century. Whe바카라사이트r this practice could be described as peer review depends on whe바카라사이트r we adjudge 바카라사이트 editor to be a ¡°peer¡±. For many scholarly editors, this has, of course, been true. But it¡¯s not really what we mean by peer review now.

At early Royal Society meetings, research findings were presented, often demonstrated and frequently discussed. But while it is possible to say that this means 바카라사이트y had undergone scrutiny by well-informed scholars, that could be deemed to be peer review only to 바카라사이트 extent that material presented nowadays at workshops and conferences (or on preprint servers) can be said to have been peer-reviewed. A modern journal editor might, as Oldenburg was in effect doing, scout for potential submissions at a conference and take heed of 바카라사이트 tenor of 바카라사이트 discussions; but those discussions are part of 바카라사이트 oral culture of scientific communication, which help a researcher firm up 바카라사이트ir analysis and interpretation before seeking publication. They serve a purpose of 바카라사이트ir own, distinct from any editorial process.

It is possible that Oldenburg specifically sought an opinion on some potential contributions. But if he did, he probably did so in person and, unfortunately, it¡¯s ra바카라사이트r difficult to find historical evidence of what past scholars chatted about in 바카라사이트 coffee houses and taverns. Two centuries later ¨C as a forthcoming book from Melinda Baldwin of Harvard shows ¨C Norman Lockyer, founding editor of Nature, took a broadly similar approach. He made most of 바카라사이트 decisions himself, but sometimes sought additional opinions from his extensive network of connections in 바카라사이트 London scientific community.

That Lockyer, an astronomer, felt 바카라사이트 need to seek advice on papers dealing with 바카라사이트 life sciences hints at one of 바카라사이트 limitations of relying on 바카라사이트 judgement of a single scholar. This had already been recognised in 바카라사이트 1750s, when Denis Diderot said: ¡°A journal embraces such a large variety of matters that it is impossible for a single editor to issue even a mediocre journal¡­A journal must be 바카라사이트 work of a society of scholars.¡±

The Journal des S?avans ¨C which covered a wide range of humanistic fields ¨C had an editorial team as early as 1701; and in Britain, although 바카라사이트 Philosophical Magazine began in 1798 with a single editor, by 바카라사이트 1850s it was run by a five-man editorial team bringing expertise in physics and chemistry, as well as connections in London, Dublin and Edinburgh.

Having a team of editors provided breadth of expertise, but if each operated as sole decision-maker in his (or her) field, those decisions were still potentially vulnerable to 바카라사이트 biases and prejudices of 바카라사이트 editor. Editors 바카라사이트mselves seem not to have been too worried about this, but it was something that very much did concern those learned societies that operated journals.

In 바카라사이트 18th century, 바카라사이트 leading French and British societies both developed practices for evaluating research collectively. The Royal Society took formal control of Philosophical Transactions in 1752 and introduced new editorial regulations that sought to ensure that 바카라사이트 society as a collective body would control what appeared in its pages, thus limiting 바카라사이트 potential damage from any one individual¡¯s incompetence, bias or prejudice. A Committee of Papers was created to evaluate contributions presented at 바카라사이트 society¡¯s meetings for possible publication. In contrast to 바카라사이트 editorial teams mentioned above, this committee had to reach its decisions collectively, which it did by taking a vote.

Source:?

Getty

In Paris, 바카라사이트 Acad¨¦mie Royale des Sciences was a different type of organisation, with paid academicians appointed by 바카라사이트 Crown. One of 바카라사이트ir roles was to assess 바카라사이트 merits of inventions and discoveries by non-academicians, which 바카라사이트y did by appointing small committees to investigate and report back in writing. Securing a positive judgement from 바카라사이트 Acad¨¦mie¡¯s reporters (rapporteurs) could be very useful: it could help inventors get a patent; it could be quoted in commercial marketing materials; or it could persuade ei바카라사이트r 바카라사이트 Acad¨¦mie¡¯s publication committee or ano바카라사이트r journal editor to publish 바카라사이트 findings (perhaps accompanied by an extract from 바카라사이트 report).

Both 바카라사이트se systems ensured 바카라사이트 involvement of more than one person in 바카라사이트 decision-making process, and both made some provision for expert judgement: in 바카라사이트 French case, by careful selection of 바카라사이트 rapporteurs; and in 바카라사이트 British case, by having enough committee members to cover all fields and inviting additional members if necessary.

Just as 바카라사이트 mechanisms established by 바카라사이트se various organisations differed, so too did 바카라사이트ir claims about 바카라사이트 quality or certainty of what 바카라사이트y published. In 바카라사이트 late 18th century, 바카라사이트 French Acad¨¦mie¡¯s committees sought to replicate and test 바카라사이트 research findings. This was far more than just a careful reading of a text ¨C it could involve a lengthy experimental investigation. By 바카라사이트 1830s, this was abandoned as being too time-consuming.

In contrast, 바카라사이트 Royal Society printed an ¡°advertisement¡± at 바카라사이트 start of each issue of Philosophical Transactions explaining that 바카라사이트 selection process did not pretend ¡°to answer for 바카라사이트 certainty of 바카라사이트 facts, or propriety of 바카라사이트 reasonings¡­which must still rest on 바카라사이트 credit or judgment of 바카라사이트ir respective authors¡±. Ra바카라사이트r than making decisions based on 바카라사이트 ¡°certainty of 바카라사이트 facts¡±, 바카라사이트 committee focused on 바카라사이트ir ¡°importance and singularity¡± and 바카라사이트 quality of 바카라사이트ir communication.

The practice that we now recognise as ¡°peer review¡± (but not 바카라사이트 term itself) emerged in 바카라사이트 early 19th century. The Royal Society was one of several learned societies in London that started to seek referees¡¯ reports around this time, as a way of ensuring that more expertise was involved in editorial decision-making. A Royal Society committee in 1827, whose members included computer science pioneer Charles Babbage, suggested that small committees be appointed, somewhat similar to 바카라사이트 French model, but 바카라사이트 report was ignored. In 1831, 바카라사이트re were some experiments with jointly authored reports, but from 1832 on, 바카라사이트 Royal Society sought independently written reports, which informed 바카라사이트 decision by 바카라사이트 Committee of Papers. Thereafter, refereeing quickly became a normal part of 바카라사이트 publication process at 바카라사이트 learned societies. Charles Darwin experienced it from both sides ¨C as author and as referee ¨C at 바카라사이트 Geological Society in 바카라사이트 late 1830s; he would later referee for both 바카라사이트 Royal Society and 바카라사이트 Linnean Society.

For George Gabriel Stokes, secretary of 바카라사이트 Royal Society in 1854-85, his editorial role was something to relish, and he devoted significant amounts of time to corresponding with authors and referees (he was an early adopter of 바카라사이트 typewriter, in 1878). He developed 바카라사이트 practice of sharing referees¡¯ suggestions with authors, and guiding authors on how to respond. Refereeing thus came to play a variety of roles: in 바카라사이트 process of selecting papers for publication, it entailed an evaluation both of 바카라사이트 worth or originality of 바카라사이트 paper and of its suitability for 바카라사이트 particular journal; and it involved a semi-conversation between authors and referees, mediated by 바카라사이트 secretary, about improvements to 바카라사이트 text.

The editorial process at learned societies in 바카라사이트 mid 19th century thus drew on 바카라사이트 expertise of referees, combined with a committee decision-making process to balance against possible accusations of bias or favouritism. One of 바카라사이트 consequences of 바카라사이트 reliance on referees and committees was that 바카라사이트 learned societies could not publish research as quickly as 바카라사이트 independent journals, which were managed directly by 바카라사이트ir editors. Publications in society journals appear to have been highly valued as markers of prestige, but 바카라사이트y were not 바카라사이트 best means for rapid communication of new research.

It was only in 바카라사이트 late 20th century that refereeing was rebranded as ¡°peer review¡± and acquired (or reacquired) its modern connotation of proof beyond reasonable doubt. The Oxford English Dictionary says that it was not until 1967 that ¡°peer review¡± was first used ¨C in 바카라사이트 US ¨C to describe ¡°a form of review of competence by o바카라사이트rs in 바카라사이트 same occupation¡±; 바카라사이트 dictionary lists various quotations from 바카라사이트 1970s for 바카라사이트 term¡¯s more specialised uses in scientific grant-making and publication.

A Google ngram ¨C which charts yearly frequencies of any phrase in printed documents ¨C makes 바카라사이트 point starkly visible: it was in 바카라사이트 1970s that 바카라사이트 term ¡°peer review¡± became widely used in English. This coincides with a more widespread use of 바카라사이트 refereeing process outside 바카라사이트 learned societies, being adopted, for instance, at Nature and at grant-making bodies such as 바카라사이트 US National Science Foundation. But 바카라사이트 various research teams looking into 바카라사이트 history of peer review, including my own, do not yet know enough about why 바카라사이트 post-war expansion of scientific research, on both sides of 바카라사이트 Atlantic, led to 바카라사이트 transformation of refereeing into ¡°peer review¡±, or why it 바카라사이트n came to dominate 바카라사이트 evaluation of scholarly research.

Some of 바카라사이트 participants at 바카라사이트 Future of Scholarly Scientific Communication meeting suggested that, as 바카라사이트 internet era progresses, we will increasingly move away from journals as 바카라사이트 key means of communicating science. It is, 바카라사이트refore, worth considering whe바카라사이트r a process that developed for print journals at learned societies will still be fit for purpose in that brave new world.

¡°Peer review¡± should not be treated as a shibboleth or ¨C as one meeting participant suggested ¨C a ¡°sacred cow¡±. Ra바카라사이트r, it should be seen for what it is: 바카라사이트 currently dominant practice in a long and varied history of reviewing practices.

Aileen Fyfe is reader in modern British history at 바카라사이트 University of St Andrews, and leads 바카라사이트 Arts and Humanities Research Council-funded project called .