Studying Peer Review… scientifically

Recent Pseudo-scientific peer review research as Gatekeepingology: Post-publication review of Siler, Lee and Bero’s (2014) ‘Measuring the effectiveness of scientific gatekeeping’

Recent  ‘Who is Afraid of Science? Exploring why the study of peer review evades scientific scrutiny’.

The retraction of 60 papers at one time (and resignation of the education minister), ‘predatory’ journals, journal hijackings, purported low inter-rater reliability, bias and conservatism and (obvious) difficulty in readily detecting fraud or misconduct are dynamics at science and medical journals that highlight a need to study these dynamics, starting with journal peer review.

The site is dedicated to the investigation of peer review as a scientific object of study – not a pre-constructed, self-evident process that is based on common experience with peer review. Of the hundreds of papers and studies on peer review that I have come across (starting in 1830 with Granville’s study), I have found only a handful that have attempted to carefully tend to the object of study prior to applying scientific methods of analysis. By this I mean that peer review was not laden with assumptions, i.e. that peer review must be pre-publication, that referees must be anonymous, or that peer review was borne of a need for rationality at the first journals in 1665.

One of the goals of the site therefore is to help steer research of peer review away from a common experience object to a decidedly scientific object. Currently, this site focuses on journal peer review at scientific and medical journals.

I invite you to browse through several preprints on journal peer review as a scientific object of study with historical (Part I, Part II, abridged) and contemporary shaping of peer review, on the changing meanings of scientific knowledge, and on the purported resistance of editorial readers to new scientific ideas. In addition, see my commentary on University World News.

Finally, I invite you to join the conversation with post-publication review of scholarship on peer review!

Joanne Gaudet