Peer Review

Peer review  is a processed used to improve the quality of articles published in scientific journals as well as the talks given at scientific conferences.  Peer review can best be thought of as a filter that filters out papers with identifiable issues that impact their reliability, accuracy or correctness.  Peer review also guards against overlooked mistakes (say a mislabeled graph), improves the article’s readability and helps the authors create a better paper.  Not all journals and conferences use the peer review process.   Journals and conference proceedings usually state if the contents were peer-reviewed.  If such a statement is lacking, assume that the journal of conference is not peer-reviewed.

Each journal or conference defines their own process, but most are pretty similar.  Typically the author (or authors) sends the paper to either the editor (in the case of journal) or the selection committee (in the case of a conference).   Let assume we are dealing with a journal article.  The editor  reviews the paper and decides if the subject of the article fits the subject of the journal.  Submitting an article on  geese  migration to a journal that specializes in condensed matter physics, will likely get rejected as inappropriate for the journal.   If the editor notes an obvious  problem with the article (say it has already been published in another journal) the editor may reject the paper for that reason.

If the editor decides that the article is a good fit for the journal, and there are no obvious problems with the article, the editor will select reviewers (often 2) who are knowledgeable in the disciple of the article.   In a double-blind review process, the  reviewers do not know the authors names and addresses or the authors and the authors do not know the names and addresses of the reviewers.  The reviewers read the article.   If the reviewers find something seriously wrong with the research, they can recommend that the article be rejected outright.  In my experience this is rare and reserved for the most egregious cases.   If the reviewer finds a mistake (say a reference to an equation is miss-numbered) the reviewer would pass along that information with the requirement that the mistake be fixed before publishing the article.  Perhaps the reviewer has an issue with a statistical analysis in the article, the reviewer would write-up the reasons for their concern and suggest or require that author address the issues before publication.  Perhaps the wording of part of the article is confusing and difficult to follow, the reviewer might suggest that the article be edited to make that section clearer.  The reviewer might point out other work, that is not referenced in the article, that the author might want, or needs, to consider addressing before publication.

The two reviews are returned to the editor.  Lets assume that the reviewers basically agree that the article should be published but have some corrections and suggested improvements.  The reviews are sent to the author of the article.  The author is then asked to either make the corrections, or to reply to the reviewers.   The reviewers might get a second look at the article and might make additional comments to the author.  When the process is over the article is published.

It the two reviewers evaluations are split, say one reviewer says don’t publish and the other says publish with changes, the editor either casts the deciding vote,  sends the article to a third reviewer, or invites the authors to respond (or a mixture).

If both reviewers give a negative review and suggest that the article not be published, most likely the article will be rejected.

Is peer review perfect, no.  Peer review is a process that involves human beings, with all their failures and biases.  Does the peer review process result in higher quality articles being published? Absolutely.  Does peer review guarantee that the article is correct? No, but it does increase the odds significantly.   If an article is not peer-reviewed is it necessarily wrong?  No.   When reading an article that is not peer-reviewed, you need to assume the role of a peer reviewer.

One last note.  Not all reports about peer-reviewed articles correctly state the conclusions or findings of the article.  Sometimes the reports deliberately miss states, or exaggerates the findings, perhaps to get a good headline.  If you find a report about the article interesting, or you would like to use the results, I strongly suggest getting a copy of the article and read it for yourself.  If you don’t understand the article, find someone who does.

A dialog between Science, economics and Religion