Things You Need to Know About Performance Baselines: Trend and Bounce
Whether you are teacher evaluating students’ learning of a new academic skill, a manager evaluating the efficiency with which employees perform a task, or a basic researcher investigating the effect of some aspect of reinforcement on a pigeon’s key peck, you are concerned with how behavior has changed as a function of whatever you have changed about the situation - your intervention. But change from what? To assess whether an intervention produces behavior change we have to have something to which we can compare our intervention’s effect. That something is the baseline, a period prior to (and, sometimes, after) an intervention takes place in which the intervention is absent.
In this and the next commentary, we will explore several features of performance baselines, beginning in this commentary on how we decide when our baseline is stable enough to allow us to assess the effect of the upcoming intervention.
We talk about two aspects of stable behavior: trend and bounce. Trend refers to whether the behavior is increasing, decreasing, or staying about the same over successive measurement periods. Bounce refers to how much variation there is in the behavior over these same periods. Put the two together and you get a reasonable idea of the stability of the target response.
The gold standard is to have low bounce (not much variability) and little to no trend over the observation periods. Meeting this criterion makes it more likely that even a small, systematic change in either bounce or trend can be seen as a result of the intervention. Low bounce and no trend is not always achievable, however, and we need to consider a more practical approach to establishing the baseline.
If we consider the likely effects of our intervention, that is, that the target behavior will either reduce or improve depending on what the behavior targeted is, we can relax our baseline stability requirement. If an intervention is expected to significantly decrease the target response, then having an increasing trend in the response in the baseline probably is OK, because (hopefully) our intervention will drive the target response the other way. An increasing trend in time spent engaged in temper tantrums across days, for example, likely soon will be reversed when it is ignored. The opposite holds, too: a decreasing trend in baseline is at least acceptable if the intervention is expected to reverse the trend and increase responding over some period of time. An example of the latter is one involving increasing the likelihood of safe behavior in the workplace. More bounce, or variability, can be tolerated if we expect our intervention to have a whopping effect than if we expect the effect to be small in magnitude.
There is a caveat to accepting trends and high bounce in the baseline. Even though we base our acceptance on the expectation that the intervention is going in a particular direction or will have a whopping effect, these things may not happen. If there is high bounce or high variability and the intervention produces, respectively, an effect such that the baseline trend continues during the intervention or a small-magnitude effect, then deviating from the gold standard was not such a great idea after all! In these latter cases, we can’t reliably determine whether our intervention made a difference in behavior.