Studies Are Usually Bunk, Study Shows
An interesting detail went overlooked in the fury over fired Google engineer James Damore ’s “diversity memo.” At the end of the document he calls for an end to mandatory “Unconscious Bias training.” Large corporations often force employees into re-education classes, this one a dull, hourlong, 41-slide seminar supported by study after study. Can these studies be trusted? Doubtful. Hands down, the two most dangerous words in the English language today are “studies show.”
The world is inundated with the manipulation of flighty studies to prove some larger point about mankind in the name of behavioral science. Pop psychologists have churned out mountains of books proving some intuitive point that turns out to be wrong. It’s “sciencey,” with a whiff of (false) authenticity.
Malcolm Gladwell is the master. In his 2008 book, “Outlier,” he argues that studies show no one is born better than anyone else. Instead success comes to those who put in 10,000 hours of practice. That does sound right, but maybe Steph Curry shoots hoops for 10,000 hours because he is better than everyone at basketball in the first place. Meanwhile I watch 10,000 hours of TV. Facing criticism, Mr. Gladwell somewhat recanted: “In cognitively demanding fields, there are no naturals.” News alert: Professional sports are cognitively demanding.
Many of the studies quoted in newspaper articles and pop-psychology books are one-offs anyway. In August 2015, the Center for Open Science published a study in which 270 researchers spent four years trying to reproduce 100 leading psychology experiments. They successfully replicated only 39. Yes, I see the irony of a study debunking a study, but add to this a Nature survey of 1,576 scientists published last year. “More than 70% of researchers have tried and failed to reproduce another scientist’s experiments,” the survey report concludes. “And more than half have failed to reproduce their own experiments.”
Bunk medical studies are worrisome, but who really cares about pop behavioral science? It’s easy to write this off as trivial, except millions take these studies and their conclusions seriously. The 2008 book “Nudge,” from academics Richard Thaler and Cass Sunstein, called for “libertarian paternalism” to push people in the right direction. But who decides what’s the right direction? Turns out the answer is Mr. Sunstein. He was hired by the Obama administration in 2009 to run the Office of Information and Regulatory Affairs. Call it psychobabble authoritarianism.
In his best seller “Blink,” Mr. Gladwell finds studies suggesting we are all unconsciously biased sexists, racists, genderists, ableists, and a litany of other “ists”—victimhood’s origin story. Newer research has deflated this theory, but the serious conclusions, and boring training seminars they inevitably lead to, remain. In her first debate against Donald Trump, Hillary Clinton channeled her inner Malcolm Gladwell and declared: “Implicit bias is a problem for everyone, not just police.” Everyone? Speak for yourself. It’s as if she called an entire slice of society deplorable.
The world is not binary, but conclusions drawn from studies always are. These studies show whatever someone wants them to. So stay skeptical and remember: Correlation doesn’t equal causation. If only I could find a study that shows this.
Mr. Kessler writes on technology and markets for the Journal.
Copyright ©2019 Dow Jones & Company, Inc. All Rights Reserved.
Appeared in the August 14, 2017, print edition of the Wall Street Journal and can be found here.