Scientific consensus, part 2: Role of the peer review process
By Jeff Bradshaw, Associate Professor of Entomology and Extension Specialist
Panhandle Research and Extension Center
In my last column, I talked about the scientific method, how science weighs evidence objectively, and how that evidence is published through a peer-review process. In this article, I will describe the role of the peer-review process in scientific consensus.
Before research results are published in a scientific journal, the work of scientists is reviewed by peers (for example, subject experts). This process is referred to as “peer review.” It is initiated by the journal editor and involves the editor’s selection of third-party reviewers who are usually anonymous to the author. Sometimes the process is double-blind (the reviewers don’t know who they are reviewing).
Usually three reviewers critique the validity of a submitted manuscript, looking for errors in logic, experimentation, analysis, and conclusion. They recommend to an editor whether or not a manuscript should be accepted or rejected for publication, and why. The author has an opportunity to respond or make corrections to the manuscript, but the editor then has the final say based on the evidence presented.
The notion of peer review is a relatively young process in science, having been in place formally since around 1831 with the Royal Society of London. It is a challenging process that can take from three to six months and having a manuscript rejected is not at all uncommon. It is a rigorous process.
As an Associate Professor of Entomology at the University of Nebraska, I am required to publish my research in a journal that uses peer review. In fact, my promotion is tied to peer-reviewed publications, as I am evaluated annually by both my Department Head (Dr. Gary Brewer, Entomology) and Research and Extension Director (Dr. Jack Whittier, Panhandle R&E Center) as to whether I’ve published in sufficient quantity and quality. So, it is just as important to me that I publish high-quality science as it is to you to receive high-quality information.
However, scientists are certainly not perfect and that is, in part, why multiple reviewers are chosen and moderated through an editor. For example, a reviewer for a manuscript that I recently submitted suggested that I should be “disappointed in my results” even though the review found no error in analysis, experimental design, or conclusion. The manuscript concerned a somewhat sensitive topic in agriculture – seed-applied insecticides.
My student and I evaluated the impact of insecticidal seed treatments on sugar beet on a group of beneficial insects (for example, ground beetles). Our previous work has shown that these beetles feed on pest insects and weed seeds. These ground beetles offer up inexpensive pest control, albeit with some limitations. However, recent research in other cropping systems has found possible negative impacts of seed treatments on pollinators and other beneficial insects.
Nearly all sugar beets are planted with a seed-applied insecticide to help control early-season pests. Therefore, it seemed reasonable to evaluate the potential for these insecticides to reduce the beneficial insects in our systems as well. In summary, our findings from a two-year study showed minimal or no effect of seed treatments on beneficial ground beetles. This is the result in which one reviewer stated I should be “disappointed.”
This comment suggested that I should have wanted a different result. I argued my case to the editor that it would be unethical for me to be disappointed in the results, that being disappointed in a test result would suggest that I should engage in some bias in my work. We made the case to the editor that if all that were published on beneficial insect-insecticide interactions was a reduction in beneficial insects, then we would have a fairly skewed perspective of a complex situation. In other words, our scientific consensus would be incorrect. Our argument won and the manuscript will be published later this year.
This last example is the kind of back and forth that happens daily in the scientific publication process. The ability to defend one’s work, particularly with potentially contentious topics, is central to this process and it is partly why all specialists at the University of Nebraska have doctoral degrees in their field of study. There is no other forum that offers as much rigor to the passage of information as the process of peer review in our scientific journals.
In a future Panhandle Perspective article, I'll discuss how you know the information you are receiving is valid.