What about those who aren't asked to review very much? Jul 22. 2010 | Comments (9)
The most common concern voiced to us so far about PubCreds is that it might negatively impact students, postdocs, and others who aren't asked to review very much and so have little opportunity to earn PubCreds. So I thought I'd offer some further thoughts on what's clearly a very important issue. Believe me, the last thing Owen and I would want would be for PubCreds to hurt students and postdocs!
My most important comment is that I think that this issue can only fully be addressed after a period of data collection before the PubCred system goes live, as suggested in our article. It's essential to be able to put numbers on the problem. How often do students and postdocs submit, what fraction of those submissions have more senior co-authors (who are likely to be able to pay the submission fee), and how often are they asked to review?
Having good data would help choose among, and refine the details of, the many possible solutions to the problem. Besides the overdraft system we suggested in our article, we've heard many other suggestions, including:
-charging students and postdocs reduced submission fees
-giving new entrants to the system some free PubCreds to start them off
-allowing students and postdocs (or maybe everyone?) a small number of free submissions/year
-allowing submission for free if none of the authors has been asked to review in a while
It's also been suggested that the PubCred system could provide the means to reduce the scale of the problem. The PubCred system could be used as an ecology-wide reviewer database, which could include information on an individual's expertise and even a status button individuals could use to indicate their willingness to review right now. Editors at all journals would then have a powerful tool for steering review requests to reviewers who might not otherwise be asked, and who are sure to accept, rather than repeatedly asking for reviews from the 'usual suspects'.
Besides highlighting the importance of this issue, and summarizing the many suggestions we've received, I also want to make a couple of other points which either haven't been raised, or haven't been sufficiently emphasized.
One is that any proposed solution to the problem of students and postdocs not being asked for reviews has to avoid undermining the entire system or creating perverse incentives. Obviously, any system that's too generous (e.g., a huge overdraft limit) will undermine the system. But there are more subtle possibilities. For instance, imagine a lab in which students typically co-author papers with their supervisor. A supervisor could decline to do any reviews himself, and forbid his students from doing any reviews. The lab could continue to publish as usual by using the students' overdrafts (or the free PubCreds issued to new entrants to the PubCred system). Depending on the lab's submission rate, the overdraft limit, and the rate at which new students join the lab to replace graduating students, the lab could continue to operate this way indefinitely. This specific situation is merely one example of a broader point: relieving *any* author of the burden of earning enough PubCreds to support their writing activities undermines the entire system to *some* extent. Students and postdocs *absolutely* need to be treated fairly--but this needs to be done in such a way that the whole system isn't undermined to a *significant* extent.
Second is that it's not necessarily straightforward to distinguish those who lack PubCreds through no fault of their own from those who lack PubCreds due to their own actions. Obviously, someone who's never been asked to review at all has had no opportunity to earn PubCreds. And someone who receives but declines many requests to review is choosing not to earn many PubCreds. But less clear-cut cases are, I suspect, quite common. For instance, imagine someone who receives few requests to review, and agrees to those few requests, but who also 'wastes' the few PubCreds they earn by submitting least publishable units, submitting minor work to highly-selective journals in hopes of getting lucky, resubmits rejected mss without bothering to revise, etc. The point is that lack of PubCreds can be 'overdetermined'; the same individual can lack PubCreds for both good and bad (or less-good) reasons.
In summary, the ideal system will not prevent submissions from people who lack PubCreds through no fault of their own, and will prevent submissions from people who lack PubCreds due to their own actions. But there's not always a clear-cut difference between those two kinds of people, which makes the 'ideal' system a little tricky to design. I don't think the problem is insoluble (in fact, I'm confident there's at least one solution that's good enough), but it clearly will require data, careful thought, and discussion in order to solve it.
-Jeremy Fox
If you had few pubcreds to spend, you would want to make sure you had a very good chance of your paper getting accepted on the first try. This might help improve the quality of submitted manuscripts, but it might also keep authors from taking risks that could get their papers in more selective journals with wider readerships.
I could also foresee a decrease in the quality of reviews, since editors would now have a fairly big incentive to farm out reviews to members of their lab groups, whether or not they were the most appropriate people for the job. My biggest concern, however, is the one you've already addressed; I can't see how this system will be fair to Early-Career researchers who must publish as many high quality papers as possible, but who are not well known enough to attract many reviews. I don't think one or two freebie publications will solve this. For such a system to work, we need to ensure that postdocs and grad students (who are less well known in their fields) get offered a fair number of review opportunities. Perhaps we can think of a system where anonymous abstracts are placed online and individual researchers can 'bid' on the ones they'd like to review? Editors could then select among the bids, only resorting to cold-calling in the event a paper receives no bids. I obviously haven’t thought this through completely, but such a system would at least give everyone the opportunity to review papers if they wanted.
Good on you for raising this issue..I think it's a very important one. Thanks you for getting a conversation started!
Tanya: I believe you're incorrect about editors having an incentive to farm out reviews to their lab groups. Even if editors are only paid 0.5 PubCreds per ms (and several people have suggested they be paid 1 PubCred, as with referees), most editors handle more than enough mss per year to pay for all their submissions. Indeed, it's been suggested to me that editing would be a sufficiently lucrative gig that people would do it for a few years and then resign, taking with them a lifetime's supply of accumulated PubCred and never needing to review again!
In response to the points about editors (and as an editor myself I realize I might be shooting myself in the foot here), I think a strong case could actually be made for giving editors of manuscripts zero pubcreds. The ‘tragedy of commons’ problem applies to finding good reviewers, not to finding good editors; most folk that are asked to join as an editor for a journal (particularly a good one) will willingly agree to do so. This is because serving as an editor already brings a number of rewards – they can include it prominently in their CV, gain greater name recognition (and the benefits that come with that), and have the opportunity to make real decisions that influence the direction and profile of the journal. The poor anonymous reviewer gets none of these benefits and s/he is the real unsung hero of the peer review process. If the purpose of pubcreds is to correct the ‘commons’ problem then I cannot see how also giving editors pubcreds will help given that they are already amply supplied with other incentives to do their job.
David:
Wow, you are shooting yourself in the foot! ;-)
In seriousness, I see your point. In response, I would argue that editors ought to be paid to compensate them for the refereeing they would otherwise do. I would also suggest that, in a world in which one needs to earn PubCreds in order to submit, the other benefits of serving as an editor might come to seem less worthwhile. This might make it harder to find willing editors unless they are paid. In all honesty, I personally would have to think hard about whether to continue as an editor at Oikos under the PubCred system if I wasn't paid for it.
I found your article thought-provoking, but I'm not convinced that the rise in 'return without review' is all that bad. In my experience (which involves seeing several thousand editorial decisions per year) it is normally clear when a paper is good enough to get reviewed and similarly clear when a paper is not at the journal's standard and hence not worth sending out for an inevitable set of rejection recommendations. The 'stochastic' papers are typically those right at the lower edge of the journal's standards and hence make it to peer review only if the editor can see some redeeming feature, which is naturally a subjective choice. Furthermore, peer review only clarifies a proportion of these cases, as often the editor has to pick whether to go with the 'reject' or the 'accept' recommendations.
Of course, everyone can point to quirky decisions they or their or their colleagues have received, but I would need to see hard data before being convinced that editors returning papers without review leads to an inconsistent decision more often than full peer review.
That said, the PubCred system could encourage a number of positive changes (as you point out), so the idea is worth exploring. I recommend contacting Tiffany Coker at Thomson Reuters (tiffany.coker@thomsonreuters.com) to ask whether ScholarOne Manuscripts (aka Manuscript Central) could accommodate such a system. The biggest practical problem I can see is duplicate accounts- these accumulate very fast and would mean an author account does not get credited for a review performed under another.
Tim Vines
Managing Editor, Molecular Ecology
nArKvI I really liked your article post.Much thanks again. Really Great.
It's a shame you don't have a donate button! I'd certainly donate to this superb blog! I suppose for now i'll settle for book-marking and adding your RSS feed to my Google account. I look forward to new updates and will talk about this blog with my Facebook group. Talk soon!
Today, I went to the beach front with my kids. I found a sea shell and gave it to my 4 year old daughter and said "You can hear the ocean if you put this to your ear." She put the shell to her ear and screamed. There was a hermit crab inside and it pinched her ear. She never wants to go back! LoL I know this is completely off topic but I had to tell someone!