What’s the difference between statistical and clinical significance? Here’s a quick, non-exhaustive intro.

In short, statistical significance is when the difference in outcomes between an experimental and a control group is greater than would happen by chance alone. For example, in a trial of whether gum chewing promoted return of bowel activity among post-op patients, one post-op group would chew gum and the other group would not.  Then researchers would statistically compare timing of return of bowel activity between the two groups to see if the difference was greater than would occur by chance (p<.05 or p<.01). If the probability (p) level of the statistical test is less than .05 then we have very strong evidence that gum chewing made the difference. [See example of gum chewing trial in free full text Ledari, Barat, & Delavar (2012).]

All well and good.

However, the effect of an intervention may be statistically significant, but not clinically meaningful to practitioners. Or the intervention’s effects may not be statistically significant, and yet still be clinically important enough to be worth the time, cost, and effort it takes to implement.

 What is clinical significance, and how can we tell if something is clinically significant? Two overlapping views:

• “Clinical relevance (also known as clinical significance) indicates whether the results of a study are meaningful or not for several stakeholders.7 A clinically relevant intervention is the one whose effects are large enough to make the associated costs, inconveniences, and harms worthwhile.8” (Armiji-Oliva, 2018). 
• Clinical significance is “the practical importance of research results in terms of whether they have genuine, palpable effects on the daily lives of patients or on the health care decisions made on their behalf” (p. 449, Polit & Beck 2012).

Let me illustrate. Researchers recently examined the effects of a 1300-1500 quiet time on a post-partum unit. Outcome measures showed that women’s exclusive breastfeeding rates increased 14%. However, this change was not statistically significant (p = .39)—a probability value well above p < .05. Nonetheless, researchers concluded that the findings were clinically significant because a higher percent of women exclusively breastfed their infants after quiet time, and arguably for those couplets the difference was “genuine” and “palpable” (p. 449, Polit & Beck). The time, cost, and effort of implementing a low risk quiet time was reasonably associated with producing valuable outcomes for some.

Always remember that the higher the risk of the intervention, the more cautious should be your translation of findings into a particular practice setting.  Don’t overestimate, but don’t overlook, clinical significance in your search to improve patient care.   

Critical thinking: How might issues of statistical versus clinical significance inform the dialogue on mask wearing during the pandemic?

For more info:

• Armijo-Olivo, S. (2018). The importance of determining the clinical significance of research results in physical therapy clinical research. Brazilian Journal of Physical Therapy, 22(3), 175-176.  doi: 10.1016/j.bjpt.2018.02.001
• Polit & Beck (2012) Nursing research on pages 449-456 discuss clinical significance in terms of individuals and groups.