Play around with your marking scheme until you find the configuration that will give the best results in a realistic amount of time.Ģ. Compare the minimum info you need before fairly accepting (or rejecting) a submission, to the ideal, highly-detailed review. Find a happy medium between detail and speed. Tips for creating a reviewer-friendly processġ. But if your reviewers are working through a long list of papers, any time they spend recording their reviews is time they may not give to completing more of them.Īn example of one-click reviewing on Ex Ordo 5. This second part might not be a big deal if people are reviewing one or two submissions. On top of this, consider how long it may take reviewers to formulate individual reviews, as well as how long it will take them to physically enter each one into your online peer review software or Google form. How many submissions will you assign to each reviewer? How much time will you have between giving out assignments and sending acceptance letters? Is this a fair ask of your reviewers? To prevent this sorry state of affairs, take a few steps back when you’re designing your process and consider it in its entirety. A poorly-built one can create unhappy reviewers who fail to deliver quality reviews, withdraw their offers to take part, or go AWOL entirely… So, regardless of whether you’re using single-blind or not, you need to think about how you’re constructing your peer review process. Conference reviewing usually happens over a few busy weeks, and most of your reviewers will be giving a substantial voluntary time commitment on top of an already busy workload. Having said that, peer review at a conference is about much more than selecting a method of review. Consider your review process in its entirety Or any sort of bias.Īnd reviewers may depend too heavily on an author’s or an organisation’s reputation, allowing it to blind them to the quality of the research in front of them. So single-blind peer review doesn’t protect your authors against gender, racial or geographic bias. While reviewers are anonymous, they can see who authors are. The ethics of single-blind peer review are still a topic of discussion. It’s worrying to think that almost half of the people who are supposed to benefit from this system have little confidence in it. In fact, only 52% of researchers assessed would label single-blind reviewing as effective (whereas 71% chose double-blind). While single-blind peer review was the most predominant form of reviewing identified in the study (with 85% of respondents claiming to have used this system), researchers preferred other methods. Disadvantages of single-blind reviewĪ study carried out by the Publishing Research Consortium found that researchers rate the effectiveness of single-blind significantly below double-blind peer review. Single-blind peer review eliminates this as a problem and enables the reviewer to be more honest without fear of public criticism. But most people don’t want to put themselves in the firing line. And sometimes, this means shining a bright light on a work’s limitations so all can see them. The best peer review is honest and unflinching. And, because they know who the author is, they can use their knowledge of the author’s previous research to aid in their assessment. This takes a considerable amount of pressure off your reviewers and allows them to judge research more objectively. Authors can’t contact the reviewer since they don’t know who they are. Keeping your reviewers anonymous allows them to critique work without being influenced by your authors. Like any other form of peer review, there are advantages and disadvantages to single-blind. ) If you’re planning to use a robust conference management system, it’ll let you hide reviewers identities easily and allow them to leave anonymous comments for authors. And in open peer review, authors and reviewers are both visible to each other. (In double-blind review, neither reviewers nor authors know who the other party is. In it, reviewers know the identity of authors, but authors don’t know the identity of reviewers. Single-blind peer review is the traditional method of review. But you should read up on double-blind and open review too. I’ll be covering single-blind peer review in this post. When you’re planning your conference’s review process, it’s a good idea to have an understanding of the benefits and drawbacks of each before you commit to one method. There are three common forms of reviewing : single-blind peer review, double-blind peer review, and open review. And, despite its flaws, it’s still the best option out there for validating scientific and technical research. Peer review has been a foundation of the scientific method since the 1600s. Thinking about using single-blind peer review for your next conference? Here’s the low-down.
0 Comments
Leave a Reply. |