My last post described our 450-page response to the DOE Climate Working Group report. This DOE report seems designed to muddy the waters about climate science — it’s a new iteration of the Merchants of Doubt. We found the report used selective misquoting of the scientific literature (cherry picking), omission of contrary results from the scientific literature, and simple errors due to a lack of understanding of the science to reach its conclusions. Further commentary of the process is in this post.
A reporter asked me for a comment on a post on Dr. Judy Curry’s blog about our review of the DOE Climate Working Group report. In her post, she said:
Let me first say, wow, this is generous and far better than I expected. Only referring to me as “unhinged” once is a W and I’ll take it. Near the end of her post, she wrote:
When I read the underlined portion (added by me), I said to myself, “so that’s how they’re going to do it.” Yes, the fix is in — everyone should understand that this is a show trial for climate science and the inevitable outcome of this “debate” will be that climate science is found to be too uncertain to justify climate policy. And now I think I know how they’ll do it.
Let me walk you through it, starting with an explanation of how peer review works.
how does peer review work?
Peer review is a system of quality control designed to ensure the rigor of research and it’s one of the bedrocks of science. This process begins after individual research groups conduct their analyses and write up their findings in a manuscript.
The authors submit their manuscript to a scholarly journal. The manuscript is then assigned to one of the journal’s editors, who are typically established experts in the field. The editor first assesses the paper for its suitability for the journal. If it meets the basic criteria, the editor then sends the paper out to a few independent experts in the same field. These are the “peers” in peer review.
These peer reviewers critically evaluate the manuscript, scrutinizing its methodology, the validity of its data, the soundness of its conclusions, and its overall contribution to the field. They then send detailed comments and recommendations back to the editor.
The editor evaluates these comments and sends anonymized versions to the authors. The authors are then required to revise their paper, addressing the concerns and suggestions raised by the peer reviewers.
A crucial aspect of this process is that the ultimate authority to publish rests with the editor. Therefore, the authors must convince the editor that they have made the appropriate and necessary changes to their work. The paper can go back and forth over several revision cycles between the authors, the editor, and the peer reviewers.
This rigorous back-and-forth is a robust process. I can’t tell you how many times, when working with a student, I’ll tell them “That argument will never make it through peer review” as an indicator that we need to shore up that part of the work.
Eventually, the editor is satisfied that the authors have fixed the issues and the manuscript is accepted for publication. Or the editor decides that the paper does not meet the standard for publication and the paper is rejected.
While peer review does not eliminate all subpar work from being published, it absolutely compels authors to write better, clearer, and more scientifically defensible papers than would occur in its absence.
you need an editor to handle the process
The editor’s role as a neutral arbiter is essential to the peer-review system. A process where reviews were sent directly back to authors, with instructions to “incorporate the good comments” would be disastrously ineffective.
Peer review is absolutely brutal. Receiving negative peer-review feedback after you’ve invested months of work into a manuscript can be a heart-rending and upsetting experience. The natural human tendency is to dismiss negative critiques (“the reviewers are idiots”) and ignore comments that require substantial revision.
This is why the editor is so important. To get the paper published, the authors understand that they have to address these criticisms head on. They cannot just blow them off because they’re upset — they must make cogent arguments that convince the editor that they have responded reasonably to the reviewers, regardless of how distasteful that is.
To demonstrate this, journals typically require authors to submit a point-by-point response that explains how each reviewer’s comment was addressed, often supplemented by a revised manuscript with tracked changes so the editor can see exactly where the changes were made.
scientific assessments are no different
The need for an impartial editor to ensure peer review is properly handled is also a requirement for scientific assessments. The Intergovernmental Panel on Climate Change (IPCC) has formalized this by appointing a “Review Editor” to each chapter. These editors play the same role as journal editors: they are auditors of the review process.
Their main responsibility is to take the complete record of comments from expert and government reviewers and verify that the author teams have responded to every single one, either by revising the text or by providing a transparent and scientifically valid reason for not doing so. This structured process guarantees that criticism is not simply dismissed by angry authors.
There are comprehensive descriptions of the duties of review editors (REs). Here’s the guidance for REs from the last IPCC report:
the DOE report
When I saw Dr. Curry’s comment that “I wouldn’t change any of the conclusions of the DOE report in response” to our comments, I suddenly realized that the DOE is not going to appoint a review editor.
They’re going to simply send the comments to the DOE authors and leave it up those authors whether and how to address comments. This is a recipe for a review process that is an unscientific sham and it would demonstrate in no uncertain terms that the DOE is uninterested in legitimate scientific debate.
The good news is that the solution is simple: they just need to appoint an unbiased, expert review editor for each chapter and design a process like that followed by the IPCC. That review editor would track every comment and ensure that a reasonable effort has been made to respond and publicly post that.
Luckily, there are thousands of climate scientists out there, so it should not be a problem to find a half dozen or so who are unbiased.
Perhaps I’m reading too much into that Curry quote. Maybe in a few days the DOE will announce the details of a credible review process, including the appointment of expert and unbiased review editors. Maybe pigs will fly. Maybe I’ll win the 2025 AL Cy Young Award. Who knows which of these equally likely things will occur.
I’d be grateful if you could hit the like button ❤️ below! It helps more people discover these ideas and lets me know what's connecting with readers.
Thank you for writing this so that those of us who are barely keeping up with the rapid fire onslaught of nonsense have an intelligent and inside look at what is actually being done. This is so helpful and simultaneously infuriating.
Great explanation. Also, my guess is that the goal of the DOE report does not align with the hunt for an unbiased editor.