Peer review and the sin of pride

Last updated on: August 4, 2015 at 5:02 pm by Tim Muldoon

Peer review and the sin of pride 2015-08-04T17:02:40-04:00 Tim Muldoon

LOGO Patheos Explore the world's faith through different perspectives on religion and spirituality! Patheos has the views of the prevalent religions and spiritualities of the world.

Dufflepuds, from CS Lewis’s “Voyage of the Dawn Treader.”

There’s a recent fascinating story of a rising high-school freshman whose work exposed a professor’s myth. Ben Collins explains [link added]:

In 2002, University of Illinois-Chicago history professor Richard J. Jensen printed “No Irish Need Apply: A Myth of Victimization,” a peer-reviewed article in the Journal of Social History. His abstract begins:

“Irish Catholics in America have a vibrant memory of humiliating job discrimination, which featured omnipresent signs proclaiming ‘Help Wanted—No Irish Need Apply!’ No one has ever seen one of these NINA signs because they were extremely rare or nonexistent.”

Eighth-grade student Rebecca Fried did a little googling and followed her curiosity to discover a great deal of evidence to the contrary. Eventually she contacted Kerby Miller, Curators’ Professor at the University of Missouri, who gave her some advice on how to turn her research into a publishable article. She did that, and it was published a month ago in the Oxford Journal of Social History. From her abstract:

This article surveys additional evidence from the nineteenth and early twentieth centuries documenting the publication of NINA-restricted solicitations directed to men. It shows that there were many such advertisements and signs, and argues that a variety of lines of evidence support the conclusion that such publications were sometimes common in some places during the nineteenth century.

Read the whole of Ben Collins’ article about Fried, who some day (I hope) will be appointed a professor at Boston College. It’s a great story in itself, but it highlights one of the fragilities at the heart of one of academia’s many sacred cows: namely, the centrality of the peer-reviewed article. Entire careers are built upon them; tenure decisions rest heavily on them; prestige and notoriety rise and fall upon them.

On one hand, it is entirely fitting that academics be held to standards of scholarly discourse in order to remain conversant in their fields of study. Tenure expectations regarding publication in one’s field amount to standards of professional competency: if one can publish, one is regarded as up-to-date in one’s field. All well and good.

On the other hand, there are a number of problems with peer review. One commentator likened peer review to Wittgenstein’s critique that it’s like proving the headline from a newspaper by checking it against the headline of another paper from the same press. Peers are, by definition, like-minded thinkers, and so the process of peer review may sound a little like Dufflepuds constantly stroking their leader’s ego.

According to Heather Morrison, who has researched scholarly communications, there are 26,746 active, academic, peer-reviewed journals published today, and another 21,099 publications which are academic in nature. It is increasingly easy to find narrowly like-minded interlocutors who can rubber stamp thinking that is sufficiently close to their own. Even in the sciences, peer-review– a relatively recent innovation in academia– is coming under criticism. From The New Atlantis:

[P]eer review is not simply synonymous with quality. Many landmark scientific papers (like that of Watson and Crick, published just five decades ago) were never subjected to peer review, and as David Shatz has pointed out, “many heavily cited papers, including some describing work which won a Nobel Prize, were originally rejected by peer review.” Shatz, a Yeshiva University philosophy professor, outlines some of the charges made against the referee process in his 2004 book Peer Review: A Critical Inquiry. In a word, reviewers are often not really “conversant with the published literature”; they are “biased toward papers that affirm their prior convictions”; and they “are biased against innovation and/or are poor judges of quality.” Reviewers also seem biased in favor of authors from prestigious institutions. Shatz describes a study in which “papers that had been published in journals by authors from prestigious institutions were retyped and resubmitted with a non-prestigious affiliation indicated for the author. Not only did referees mostly fail to recognize these previously published papers in their field, they recommended rejection.”

Academia is not immune to the sin of pride, and its cousin prestige; in fact one could argue that it is an incubator of them. In the early nineteenth century, Oxford dons considered it odd to be concerned with publishing, as their primary task was lecturing. Today, however, one must “publish or perish”–even in an age when luminaries such as René Girard– one of les immortels of the Académie Française— described himself as having “perished” while a professor at Duke. (He said this in a video formerly hosted on the Stanford Magazine website; the link no longer works.) There is a kind of academic inertia in the process of peer review, which can stifle creativity and prolong passing academic fashions.

The editors of The New Atlantis nine years ago put their finger on the kinds of change we are seeing in academia: the rise of online journals, and their instant turnaround, are beginning to make the traditional peer-review process obsolete. I think of an online forum like The Immanent Frame, which hosts lively academic conversations but is much more agile than traditional journals. [No, I do not own stock; just an admirer.] Publishing is, by definition, making one’s thinking public, and as such one is expected to present one’s best work. But if one’s “public” is but a few like-minded academics, will one’s writing really represent the best thinking?

One final thought. The Oxford dons were on to something: teaching and writing are different skills, and academic writing is an even more particular skill. Kudos go to young Rebecca Fried for learning it so quickly, largely on her own. But Rebecca is probably not prepared to teach college students. We diminish the art of teaching if we assume that the most “prestigious” (whatever that means) authors are the best teachers. Perhaps it is time to start thinking about how to uncouple the two skills in the ways we evaluate faculty, and even institutions as a whole.

Previous Post

July 8, 2015

Pope Francis address at Pontifical Catholic University of Ecuador

Next Post

September 8, 2015 Theologians as prophets

Browse Our Archives

Select a Category academic freedom Catholic intellectual tradition epigenetics humanities mission politics science teaching Theology Uncategorized university life

Select a Month November 2015 October 2015 September 2015 August 2015 July 2015 June 2015 May 2015 April 2015 March 2015 February 2015 January 2015 December 2014 November 2014 October 2014 September 2014 August 2014

TAKE THE

Religious Wisdom Quiz

Which Psalm opens with "Blessed is the man"?

Select your answer to see how you score.

Psalm 51 Psalm 119 Psalm 1 Psalm 23