Replication and Another Misrepresentation of Behavior Analysis
No sooner do I write about how neat it is that behavioral psychology has built-in mechanisms of replication in many of its investigations than journalist Andrew Ferguson confuses, indeed, seems to equate, behavioral psychology with general psychology and damns behavioral psychology for its failures to replicate. Let's take a look, first, at what he actually says in his October 19, 2015 op/ed piece in The Weekly Standard. Describing the National Public Radio report of the research on which he bases his piece, he first notes that "[t]he subject [of the original article in the journal Science] was the practice of behavioral psychology. Behavioral psychology is a wellspring of modern journalism." Then he summarizes the findings of the Science study as follows: "Over 200 researchers, working as the Reproducibility Project, had gathered 100 studies from three of the most prestigious journals in the field of social psychology. Then they set about to redo the experiments and see if they got the same results. … Nearly two-thirds of the experiments did not replicate, meaning the scientists repeated these studies but could not obtain the results that were found by the original research team."
I start by saying that none of the experiments in the Science article nor, therefore, in the NPR report, have any more to do with behavioral psychology than B.F. Skinner has to do with Sigmund Freud. Here’s what the authors of the Science article say about their sample: “We pursued a quasi-random sample by defining the sampling frame as 2008 articles of three important psychology journals: Psychological Science (PSCI), Journal of Personality and Social Psychology (JPSP), and Journal of Experimental Psychology: Learning, Memory, and Cognition (JEP:LMC). The first is a premier outlet for all psychological research; the second and third are leading disciplinary-specific journals for social psychology and cognitive psychology, respectively ….” Behavioral psychology is the kind of psychology that you read about in the commentaries on this web site, and none of the listed journals, as the authors note, have less than anything to do with behavioral psychology. Ferguson is either uninformed or careless in equating behavioral psychology with the research published in these journals.
My reactions to the Ferguson piece are directed at what he says about behavioral psychology, and not an indictment of social, cognitive, or any other kind of psychology. Most of the research reported in these journals, however, does rely on the kinds of statistical analyses that resulted in the replication issues raised in the Science article. As I noted in my earlier comments on replication, behavior analysts largely reject these methods and use methods more aligned with better assuring the replicability of their findings.
As Ferguson's article progresses, he too addresses some of the weaknesses of the kinds of statistical analyses represented in the failed replications, and he also seems to drop "behavioral psychology" in favor of the more generic label of "behavioral science." Behavioral science includes behavior analysis/behavioral psychology, but also includes all the other types of psychology, along with the other so-called social sciences such as sociology and anthropology.
I do share Ferguson’s concerns about failures to replicate basic findings, but I think he is too quick to lump everyone and everything together and damn them all. He overlooks, or chooses to ignore, the good that the behavioral sciences have done across the spectrum of human behavior. We all learn early on the dangers of stereotyping, and Ferguson's review epitomizes those dangers. Not only are the behavioral sciences damned, but so are behavioral scientists. Of them he observes that "…humility among social scientists never lasts; it's not in the job description. No sooner do social scientists concede the limitations of their work than they begin to exaggerate again." This comment doesn't deserve retort.
But I digress. To get back to my main point. Ferguson is not the first person to mischaracterize and in so doing misrepresent behavior analysis and behavioral psychology (see for example the two articles by Todd and Morris referenced below). Nor, unfortunately, will he be the last, but such mischaracterization suggests two problems: behavior analysts are not communicating effectively with others, and others seem to “know what they want to know” about behavior analysis, making them unfortunately more likely to mischaracterize us without listening to what we say.
Todd, J. T., & Morris, E.K. (1983). Misconception and miseducation: Presentations of radical behaviorism in psychology textbooks. The Behavior Analyst, 6,153-160.
Todd, J. T., & Morris, E. K. (1992). Case histories in the great power of steady misrepresentation. American Psychologist, 47, 1441-1453.