Progress on obtaining journal-level data on the peer review system Feb 18. 2011 | Comments (0)
Just an update on the latest progress on PubCreds. Owen Petchey and I have formally requested detailed data on the peer review system from a whole bunch of ecology journals. Basically, we've asked for anonymized data that would let us conduct the analyses we suggested in a previous post, to test for the 'journal-level symptoms' of a 'tragedy of the reviewer commons.'
These data are available from online ms handling systems, although they aren't trivial to extract. We're very gratified that the British Ecological Society has agreed to provide data for all four of their long-running journals, for all the years for which the data exist (2003-2010). Several other journal owners also have agreed to try to provide data, or else have provided qualitative or semi-quantitative summaries of the data. We've only had one hesitant response so far (I won't say from who). We're still waiting to hear from most of the journals we've contacted; it's very early days.
Once we have as much data as we can get, our plan is to conduct the analyses we've previously described, and any other analyses suggested to us by EiCs, Managing Editors, and journal owners. We will write up our results for publication and provide the ms to the data providers to get their feedback before submitting.
Our purpose in this exercise (which we are undertaking voluntarily, for free) is descriptive. In order for discussion about peer review in ecology to move forward, there has to be an agreed set of facts about the past and current state of the system. That's what we aim to provide.