Last week’s blog focused on the strongest types of evidence that you might find when trying to solve a clinical problem. These are: #1 Systematic reviews, Meta-analyses, or Evidence-based clinical practice guidelines based on systematic review of RCTs; & #2 Randomized controlled trials. (For levels of evidence from strongest to weakest, see blog “I like my coffee (and my evidence) strong!”)
So after the two strongest levels of evidence what is the next strongest? #3 level is controlled trials without randomization. (Sometimes called quasi-experimental studies.)
Here’s an example of a controlled trial without randomization: I take two groups of mice and test two types of cheese to find out which one mice like best. I do NOT randomly assign the mice to groups. The experimental group #1 loved Swiss cheese, & the control group #2 refused to eat the cheddar. I assume confidently that mice LOVE Swiss cheese…
Quasi-experiments are a lot of work, yet don’t have the same scientific powerto show cause and effect, as do randomized controlled trials (RCTs).An RCT would provide better support for any hypothesis that X causes Y. [As a quick review of what quasi-experimental versus RCT studies are, see “Of Mice & Cheese” and/or “Out of Control (Groups).”]
So why do quasi-experimental studies at all? Why not always do RCTs when we are testing cause and effect? Here are 3 reasons:
#1 Sometimes ETHICALLY the researcher canNOT randomly assign subjects to a control and an experimental group. If the researcher wants to compare health outcomes of smokers with non-smokers, the researcher cannot assign some people to smoke and others not to smoke! Why? Because we already know that smoking has significant harmful effects. (Of course, in a dictatorship, by using the police a researcher could assign them to smoke or not smoke, but I don’t think we wanna go there.)
#2 Sometimes PHYSICALLY the researcher canNOT randomly assign subjects to control & experimental groups. If the researcher wants to compare health outcomes of
individuals from different countries, it is physically impossible to assign country of origin.
#3 Sometimes FINANCIALLY the researcher canNOT afford to assign subjects randomly to control & experimental groups. It costs $ & time to get a list of subjects and then assign them to control & experimental groups using random numbers table or drawing names from a hat.
Thus, researchers sometimes are left with little alternative, but to do a quasi-experiment as the next best thing to an RCT, then discuss its limitations in research reports.
Critical Thinking: You read a research study in which a researcher recruits the 1st 100 patients on a surgical ward January-March quarter as a control group. Then the researcher recruits the 2nd 100 patients on that same surgical ward April-June for the experimental group. With the experimental group, the staff uses a new, standardized pain script for better pain communications. Then the pain communication outcomes of each group are compared statistically.
Is this a quasi-experiment or a randomized controlled trial (RCT)?
What factors (variables) might be the same among control & experimental groups in this study?
What factors (variables) might be different between control & experimental groups that might affect study outcomes?
How could you design an ethical & possible RCT that would overcome the problems with this study?
Why might you choose to do the study the same way that this researcher did?
In the last “Quasi-wha??” blogpost, I described 1 type of experimental design: Quasi-experimental. To review… In quasi-experimental designs, the researcher manipulates some variable, but either 1) doesn’t randomly assign subjects to a control and experimental group OR 2) doesn’t have a control group at all.
For example, the researcher may introduce pet therapy on unit #1 and avoid pet therapy on unit #2 and then afterwards compare the anxiety levels of patients on the 2 units. That study has a control group (unit #2), but because patients weren’t (& probably couldn’t be) randomly assigned to the units, this would be a quasi-experimental study. The control group in this pet therapy case is what researchers call a “non-equivalent control group.” Non-equivalent means the groups are different in ways that might affect study results! [Note: For review of what constitutes a true experimental study see first part of “Quasi-wha??” blogpost.]
Herein lies a weak link in the cause-and-effect chain. Quasi- designs are NOT as strong as true experimental designs because something other than our treatment (in this case pet therapy) may have created any difference in outcomes (e.g., anxiety levels). Why? Here’s your answer.
In an experimental study, randomly assigning subjects to a
control and a separate experimental group means that all the little, variable weirdities of all subjects are equally distributed to each group. Each group is the same mix of different types of people. This means we can assume that both groups are the exact same type of
people in regard to things that may influence study outcomes, such as attitudes, values, preferences, beliefs, anxiety level, psychology, physiology and so on.
In contrast, in the quasi-experimental pet therapy example above, there is probably something that caused a certain type of person to be on unit #1 and a different type to be on unit #2. Maybe it was their diagnosis, their doctor, their type of surgery, or other. Thus, we cannot assume that people in unit #1 and unit #2 groups are the same before pet therapy, and so any differences between them after pet therapy might have already existed.
So why do quasi-experimental studies at all?? There are great reasons! Stay tuned for next blogpost.
the participants are randomly assigned to groups, &
one group is a control group that gets a placebo or some inert treatment so that outcomes in that group can be compared to the group(s) that did get the treatment.
Non-experimental design in which the researcher doesn’t manipulate anything, but just observes & records what is going on. Some of these are descriptive, correlational, case, or cohort study designs for example.
One particularly interesting “experimental” design is one in which 1 or 2 of the experimental design ideal requirements as listed above are missing. These are called quasi-experimental designs.
In a quasi experimental design
The researcher manipulates some variable, but….
Either the participants are NOT randomly assigned to groups
&/OR there is no control group.
A quasi-experimental design is not as strong as a true experiment in showing that the manipulated variable X causes changes in the outcome variable Y. For example, a true experimental study with manipulation, randomization, and a control group would create much stronger evidence that hospital therapy dogs really reduced patient pain and anxiety. We would not be as confident in the results of a quasi-experimental design examining the exact same thing. In the next blog, we’ll examine why.
Critical thinking: Go to PubMed & use search terms “experiment AND nurse” (without the quotation marks). Open an interesting abstract and look for the 3 elements of a classic experimental design. Now look for “quasi experiment AND nurse” (without the quotation marks.) See what element is missing!