Tag Archives: reading research

“That is so random!” But is it Representative?

What makes a good sample in research?  One thing.  And it isn’t random selection.  (Surprised?)

Portrait of a diversity Mixed Age and Multi-generation Family embracing and standing together. Isolated on white background. [url=http://www.istockphoto.com/search/lightbox/9786738][img]http://dl.dropbox.com/u/40117171/group.jpg[/img][/url]It is representativeness.  No matter how the sample was picked, it must be representative of all those in the larger population, if the researcher wants to say anything about anyone who wasn’t in the study.  Now, of course, it is true that random selection is more likely to give you a representative sample, but it is no guarantee.  Only likely.

What is random sampling?  It is when every member of the larger population has an equal chance of being selected for the study sample.  Example? Drawing names out of a hat.  It is well-accepted practice to generalize research results from a random sample to others like those being studied (assuming that all other aspects of the study are strong).

In contrast a convenience (or nonprobability) sample is when some people are more likely to be chosen to be in the study than others.  You shouldn’t generalize the results of these studies because the samples may Not represent others.

Example of when random sampling doesn’t work: Let’s say you have a mixture of red, green, & yellow apples, and you select a sample that has only yellow apples.  (The red & green ones are going to be offended!–They’re left out.)  You now have a sample that is biased in favor of yellow apples!   Your sample does Not represent the larger population of apples…even if you used random methods to get it.  If you want to apply the study to red & green & yellow apples…well….you must get some of them in your sample, too. The yellow apples might not be at all like the other types and studying just yellow might mislead you into thinking something about the red & green ones that isn’t true!   Of course you could study all the millions of apples in the world and exclude none, but that would be pretty cumbersome and expensive.   So, it’s better to go for a representative sample!

When else doesn’t random sampling create a representative sample?   If I am doing historical research, say on the Nursing Department at California State University/Northridge, then I want to hand pick the specific RNs by name who were in charge of the Department from the beginning.  Randomly selecting nurses from those who worked at the University won’t represent those leaders.

QUESTIONCritical Thinking:  Take a quick look at the linked abstracts. How were the samples selected?  How representative are the samples of a larger population of interest?  Could you generalize the results to other people, and if so to whom?

Want more information on sampline? Check this out.  It takes < 5 minutes:

https://www.youtube.com/watch?feature=endscreen&v=be9e-Q-jC-0&NR=1

“The Sky is Falling!” (or Don’t be an EBP Chicken Little)

We all know the story of Chicken Little, right?  Chicken Little is walking through the forest, an acorn falls and hits her on the head, then Chicken Little  runs about in a panic telling everyone, “The sky is falling! The sky is falling!” A lot of the animals are convinced, and the fox—who knows the truth that it was only an acorn—convinces Chicken Little & some other animals to come into his den to be safe from the falling sky. There he eats them. Interestingly the fox used the correct evidence well. Chicken Little & fox chicken littlecompany used evidence poorly and created a safety hazard for themselves!

Moral of the story? Don’t be a Chicken Little when it comes to reading and applying research to practice. Get all the facts before you share the research findings with others. Don’t read only the “acorn” of abstract, introduction, and discussion, and then assume that you know what the research study shows and that you can apply it to your work. Don’t turn an acorn into a falling sky!

How to avoid being an EBP Chicken Little? To avoid being an Evidence-Based Practice (EBP) Chicken Little, you should follow the example of Samantha in this research fairy tale: “Samantha…read the study abstract. Then, while Chicken Little and her friends waited anxiously, she read the introduction, the literature review, the research questions, the methods section, the findings, and the discussion section. Then she went back and read all the sections again. Finally, as Chicken Little hopped around her impatiently, she reread the findings. “Chicken Little, have you and your friends read the entire study?” asked Samantha.” (source: https://www.son.rochester.edu/student-resources/research-fables/chicken-little.html)

Why go to all this trouble? I’m busy. The reasons to take time and effort to read the WHOLE study are many. First, the subjects may not be at all like your own patient population—what if the researchers studied only “left-pawed albino hamsters”? Second, the research might not be a strong meta-analysis or randomized controlled trial whose results can actually be applied to other times and places—what if the researchers just watched subjects walk around, but didn’t test what makes them walk better?  A third reason is that the results might be statistically significant, but clinically irrelevant!—what if researchers were studying pain, but everyone in the study had 1-2 on the pain scale?

You don’t want to endanger patient safety by misunderstanding and misapplying research and then be “eaten alive” by adverse patient outcomes or by critics, who will see through your mistakes. Remember in the fairy tale Chicken Little and his careless friends misunderstood the facts, and hence were susceptible to being eaten by a fox.

What if you don’t know how to read research? No problem. Everyone who knows how to read research now had to learn it—no one was born knowing.  So,…you can learn it, too!  It doesn’t take magical powers.  Countless resources are online; others are in your hospital or in a university research course. If you check the box on this page to follow the EBP blog, (I hope) it will help, too. Go back and read earlier blogs on sections of a research report.

For more information on how to be an EBP Chicken Little (NOT) see the very creative research fairy tale by Jeanne Grace (copyright Rochester College) at https://www.son.rochester.edu/student-resources/research-fables/chicken-little.html

Critical thinking:

  1. After reading Grace’s fairy tale at the above link list at least three (3) things that Chicken Little might have learned, had she read the whole article!
  2. Compare an abstract with a full article, and check out the differences. Specifically compare the abstract at  http://www.ncbi.nlm.nih.gov/pubmed/2606078 with what you learn about them from the full article at http://www.ncbi.nlm.http://www.ncbi.nlm.nih.gov/pubmed/2606078nih.gov/pmc/articles/PMC4449996/. Did reading the whole article change the way you understand how orQUESTION whether the study might apply to your work? If so, how? And if not, why not?

“Watch & Learn!” – Systematic Reviews of Non-experimental Studies

Today’s top tip: Want to find the strongest research evidence for your project?   Go to http://www.ncbi.nlm.nih.gov/pubmed & add the strongest type of research designs as one of your search terms. For example, add the terms meta-analysis or systematic review to your other search terms. **********************************************

Now to the new!  What is a systematic review of descriptive studies? [Note: For information on stronger levels of research “I like my coffee (and my evidence) strong!)]Cat Fishbowl2

First, remember that in a descriptive study, the researcher merely watches or listens to see what is happening. Descriptive studies do not test interventions.

Second, a systematic review (not to be too silly) is a review that is done systematically in order to include all literature on a particular topic . The authors will tell us where they searched for studies, what search terms they used, and what years they searched. That way we can feel sure that all relevant articles are included.

Therefore, in a systematic review of descriptive studies the authors

  • Collect non-experimental studies related to the problem they are trying to solve,
  • Critically review them, &
  • Write up that analysis for you and me.

You won’t see a lot of numbers or statistics in these reviews of non-experimental studies.

Systematic review of descriptive studies are weaker than other levels of evidence in part because they are critical reviews of non-experimental studies in which the researchers only observed subjects. Those non-experimental studies that they are reviewing may be quantitative with results reported in numbers or qualitative with results reported in words.

Here’s an example with results reported in words (qualitative): Yin, Tse, & Wong (2015) systematically reviewed studies for what factors affect RNs giving PRN opioids in the postop period.   They searched publications 2000-2012 and ended up with 39 relevant studies. Within those 39 articles were descriptive studies that identified 4 basic influences on opioid PRN administration by RNs to postop patients: “(i) nurses’ knowledge and attitudes about pain management; (ii) the situation of nurses’ work practices in administrating range orders for opioid analgesics; (iii) factors that influenced nurses’ work practices; and (iv) perceived barriers to effective pain management from the nurse’s perspective.” [note: In this study a few of the 39 studies were experimental in which something was done to subjects and then outcomes measured, and Yin et al., commented separately on what those showed.]

Critical thinking: What are key differences between a meta-analysis of randomized controlled trials and a systematic review of QUESTIONdescriptive studies?

Reference found with search terms: review of descriptive studies nursing pain – Yin, H.H.,Tse, M.M., & Wong, F.K. (2015). Systematic review of the predisposing, enabling, and reinforcing factors which influence nursing administration of opioids in the postoperative period. Japan Journal of Nursing Science, doi: 10.1111/jjns.12075.

 

Cohort & Case-controlled studies: Going forward & backward

Got a clinical problem?  You probably want to solve it with evidence—STRONG evidence.   Click on this link to see one well-accepted hierarchy from strongest #1 to weakest #7 (Melnyk & Fineout-Overholt, 2005).   Today let’s look at the 4th strongest level of evidence = Case controlled or cohort studies

First a quick review

Click here for a quick review of the strongest 2 levels of evidence (#1 Systematic reviews, Meta-analyses, or Evidence-based clinical practice guidelines based on systematic review of RCTs. #2 Randomized controlled trials)

Click here for a review of the 3rd strongest type of evidence (#3Controlled trials without randomization)

Now on to the new “stuff”  strong

All 3 of the top, strongest levels of evidence are experimental studies (or include available experimental studies). That means the researcher actually does something or gives a treatment to some of the subjects and then records the outcomes. 

The weaker 4 levels of evidence are non-experimental designs. This means that the researcher merely observes & does Not do anything to subjects. So how does that work?!

First, a cohort study (non-experimental). A cohort study starts with a group of people who have something in common and then the researcher observes only & keeps collecting data from them over a long time into the future. Data collection into the future is called a prospective study. An example is the Nurses’ Health Study, in which over 20,000 nurses were identified and followed-up annually with tests and surveys for over 25 years (this study is still ongoing). These studies provide very valuable information, but are obviously very expensive and time-consuming.”(OMERAD EBM course, 2008)

Now a case-controlled study (non-experimental).  In a case controlled study the researcher observes only & collects data over time into the past (not the future). Data collection into the past is called a imagesCAH6C8NTretrospective study. Again, from the OMERAD EBM (2008) site this example: “Patients with a disease are identified who have suffered a bad outcome such as death or recurrence, and compared with patients who have the disease but haven’t suffered the bad outcome. For example, a researcher might  identify a group of breast cancer patients who have died…, and compare them with a similar group of patients with breast cancer who are still living.”

Critical thinking: Which of these would be better for casQUESTIONe-controlled study and which for cohort study.

  1. You are a runner in the Los Angeles marathon and you are interested in how that race can improve cardiovascular health among those who finish. Question: Cohort or Case controlled?
  2. Some finishers of the LA marathon die of heart attacks 20 years later; many survive another 40 years.   Question: Cohort or Case controlled?

For more info see:

 

Of Mice and Cheese: Research with Non-equivalent Groups

Last week’s blog focused on the strongest types of evidence that you might find when trying to solve a clinical problem. These are: #1 Systematic reviews, Meta-analyses, or Evidence-based clinical practice guidelines based on systematic review of RCTs; & #2 Randomized controlled trials. (For levels of evidence from strongest to weakest, see blog “I like my coffee (and my evidence) strong!”)

So after the two strongest levels of evidence what is the next strongest? #3 level is controlled trials without randomization. (Sometimes called quasi-experimental studies.)

Here’s an example of a controlled trial without randomization: I take two groups of mice and test two types of cheese to find out which one mice like best. I do NOT randomly assign the mice to groups. The experimental group #1 loved Swiss cheese, & the control group #2 refused to eat the cheddar. I assume confidently that mice LOVE Swiss cheese & do NOT like cheddar. What’s the problem with my conclusion? If you want to know, then read on!swiss cheese

In my mouse Controlled Trial Without Randomization, the groups were formed by convenience and Not randomly assigned. Thus, any difference in outcomes between groups might be related to some pre-existing difference between groups. My outcome of mice loving Swiss & hating Cheddar might have nothing to do with the experimental treatment.   In fact, I did not know that all my mice in the Swiss cheese group #1 hadn’t eaten in 2 days, and my mice in the cheddar group #2 had just had a full lunch. Ooops.

On the other hand if I had randomly assigned all the mice to two groups, then I could be relatively confident that all little differences between group members were evenly distributed to both groups, so that the groups were equivalent. My two mouse-groups would have probably ended up with a pretty even distribution of both hungry and not-so-hungry mice.   Then if my Swiss cheese group devoured the Swiss and my cheddar group rejected the cheddar, I could be more certain that mice love Swiss and dislike cheddar.

Happy evidence hunting!

What’s an RCT anyway?

  • Question: What is a randomized controlled trial (RCT)? And why should I care?
  • Answer: An RCT is one of the strongest types of studies in showing that a drug or a treatment actually improves a symptom or disease. If I have strep throat, I want to know what antibiotic works best in killing the bacteria, & RCTs are one of the best ways to find that answer.

In the simplest kind of RCT, subjects are randomly assigned to 2 groups.  One group gets the treatment in which we are interested, & it is called the experimental group.   The other group gets either no treatment or standard treatment, & it is called the control group.  

Here’s an example from a study to determine whether chewing gum prevents postoperative ileus after laparotomy for benign gynecologic surgery:  A total of 109 patients were randomly assigned to receive chewing gum (n=51) or routine postoperative care (n=58).  Fewer participants assigned to receive chewing gum … experienced postoperative nausea (16 [31.4%] versus 29 [50.0%]; P=0.049) and postoperative ileus (0 vs. 5 [8.6%]; P=0.032).* There were no differences in the need for postoperative antiemetics, episodes of postoperative vomiting, readmissions, repeat surgeries, time to first hunger, time to toleration of clear liquids, time to regular diet, time to first flatus, or time to discharge. Conclusion?  Postop gum chewing is safe & lowers the incidence of nausea and ileus! (Jernigan, Chen, & Sewell, 2014. Retrieve from PubMed abstract)

Do you see the elements of an RCT in above?

Let’s break it down.

  • Randomized means that 109 subjects were randomly divided into 2 or more groups. In above case, 51 subjects ended up in a gum chewing group & 58 were assigned to a routine care, no gum group.  Randomization increases the chance that the groups will be similar in characteristics such as age, gender, etc.   This allows us to assume that different outcomes between groups are caused by gum-chewing, not by differences in group characteristics.
  • Controlled means that 1 of the groups is used as a control group. It is a comparison group, like the no-gum , standard care group above
  • Trial means that it was a study. The researchers were testing (trying) an intervention and measuring the outcomes to see if it worked.  In this case the intervention was gum chewing and the measure outcomes were nausea and ileus.

Why should you care about RCTs?  Because RCTs are strong evidence that an intervention works (or doesn’t) for your patients

Critical Thinking Exercise:   Go to http://www.ncbi.nlm.nih.gov/pubmed   In the blank box at the very top enter a few key words about the problem in which you are interested + RCT.  For example:  music pain + RCT.   Then read 1 or more of the abstracts looking for random assignment (randomized), control group, and whether it was a study (trial).   You’re on your way!    -Dr.H

*Note: You may remember from other blogs that p<.05 means the difference between groups is probably cause by the intervention—in this case gum chewing.

“Which Came 1st–The chicken or the egg?” (or, Why Correlation is Not Causation)

Correlation is not causation. RNs who want to use research in practice must take this seriously.

What does it mean?   Answer: Just because two things happen together, we cannot say that one causes the other.

Consider the example of drinking coffee and staying awake. The more coffee you drink, the more hours you will stay awake.   But isn’t it also true that the more hours you try to stay awake, the more coffee you will likely drink?

Thus, in a study about coffee drinking and sleep, you may read that coffee and hours of being awake are correlated. In other words, they occur together. When one goes up, the other goes up.   What is not clear is whether the coffee is causing the person to be awake longer, OR whether being awake longer is causing the effect of more coffee consumption.   The unsolved mystery is: “Which is the cause and which is the effect?”[1]

Likewise consider the consistent relationship between chickens and eggs. Every egg was produced by a hen. Every one. In statistical terms this means that on a scale of 0 to 1 (with 0 being no relationship whatsoever and 1 being a relationship that occurs 100% of the time) eggs and chickens have a perfect 100% relationship of 1. (A statistician would write this as r=1.0).   What is unclear is whether (when the world was young), the chicken appeared first and caused the first egg, or the egg came first and caused the first chicken. Again the unsolved mystery is: “Which is a cause and which is the effect?”

Okay, so let’s do some critical thinking about actual research.  You read these results:

“More calls for assistance related to less fall-related patient harm. Surprisingly, longer response time to call lights also related to fewer total falls and less fall-related patient harm. Generally speaking, more call light use related to longer response times.”[2]

When you read this article, what should you be assuming about the researchers’ findings in terms of relationships instead of cause-and-effect? (Hint: Think about chickens & eggs, or coffee & insomnia.)

[1] Bonus info: We call causes “independent variables” and we call effects, “dependent variables”

[2] Tzeng, H.M,. & Yin, C.Y. (2009). Relationship between call light use and response time and inpatient falls in acute care settings. Journal of Clinical Nursing, 18(23), 3333-3341. doi: 10.1111/j.1365-2702.2009.02916.x. Epub 2009 Sep 4.

In Conclusion: “Back to the future”

A great conclusion to a study can take several forms.   One of these is like the abstract. The researcher summarizes the entire study in 100-200 words or so.   Researchers can also end with the suggestions for future research or an intriguing quote.   A great conclusion will give you the “bottom line” of why the study is important to you!

Thus it is sometimes valuable when FIRST encountering a new research article, to scan the abstract, intro, discussion/implications, and conclusion FIRST.   This will give you the big picture—the 30,000 foot level picture. Then you can get down at “ground level” and read the whole research article more carefully.

Research reports are Not mystery novels, and the plot will Not be spoiled if you read the conclusion first!  You may find that doing this makes it easier to understand the article.

If you are writing a research report yourself, then make sure that you keep the conclusion lively and interesting!   You know your project—what is THE main take away that you want readers to have?

CRITICAL THINKING….

Assume that you read the following Conclusion from Brown & McCormack (2006) BEFORE reading the rest of the article.   What ideas would you look for in the article that show up here in this “end-of-the-article-abstract-and-implications”? Which of their conclusions would you check out within the main article?

     This ethnographic study highlighted a number of issues that affected the older persons’ pain experience in the acute surgical setting. Additionally, it provided insight into how nurses approached the assessment and management of pain in this patient group. The study demonstrated the value of applying multiple sources and methods of data collection in order to obtain a more complete view of the competing forces that operate within the ward environment.

     Data analysis revealed three action cycles for further developmental work – pain assessment practices, knowledge/ insight and strategies to cope with episodes of uncontrolled pain and organization of care, along with ward culture, have been identified as having an inhibitory effect on pain management in older people. In addition, recognition that patient barriers may contribute to ineffective pain management is a point worthy of consideration.

     Improving pain management practices, therefore, requires healthcare professionals to reflect on reactions, values and beliefs surrounding pain and examine how these have the potential to influence the care provided. Consequently, there is a need for a focused, collaborative, interdisciplinary approach to challenge current pain management practices and implement change. There is a growing acknowledgement that successful interventions must deploy multiple strategies, targeting aspects of the individual, the organization, its culture and characteristics of the message, simultaneously (Kitson 2001). (p.1296)

Reference: Brown, D., & McCormack, B. (2006). Determining factors that have an impact upon effective evidence-based pain management with older people, following colorectal surgery: An ethnographic study. The Authors. Journal compilation, 1987-1298. doi: 10.1111/j.1365-2702.2006.01553.x

“What are you implying?”—the question to ask about Findings*

As you read closer and closer to the end of a research report, you should start asking, “What are the implications of what this researcher found?”   In other words now that the findings show X what is the Y that we do in response?

Sometimes the researcher labels a section IMPLICATIONS. Other times implications are included in the DISCUSSION section.

What implications you look for may depend on your role. Are you a direct, inpatient care RN? Then you want to know what the research implies about the need to maintain or change practice.   Are you in staff development or teaching clinical students? Then you want to know whether this means you should be teaching something or some “how-to” differently.   Management/administrator? Then what does this mean for leadership or organizations. And,…if you’re a researcher, then you want to know what is the next question raised by this study, OR perhaps does this study need to be repeated before we can feel confident in the findings.   (Of course, if you’re a student looking at a study may mean that you are one step closer to completing one of those evidence-based assignment papers.)

If you look carefully, you will see that the researcher tells you what they think the implications are for patient care, education, management, research, students, patients, or others.

Research does not give final answers. Exhilaratingly a research article often raises more questions than it answers—especially because any research project can only narrowly be designed to examine one teeny area of reality.   (OK. Perhaps only researchers would find that thrilling.)

So, as you read think: What do these research findings mean for RN practice?

Critical Thinking Practice: Find the implications in this excerpt from the Discussion section of Brown & McCormack (2005):  The study revealed that accurate and holistic pain assessment for older people were (sic) deficient in the acute surgical setting…. As a number of older people experienced hearing difficulties, it was also possible that patients did not respond because they misunderstood or simply did not hear what they were being asked. Herr and Mobily (1991) suggest that a reliable assessment of the older persons’ pain can be best obtained if they are offered privacy rather than asked to discuss pain in a public location. Whilst this can be difficult to achieve in a ward environment, measures such as drawing the curtains or moving closer to the patient, may afford some improved degree of enhanced communication and privacy for pain assessment. (p.1295)

——————————

*In earlier posts at http://discoveringyourinnerscientist.blogspot.com/ , I summarized what titles, abstracts, introductions, methods, results, and discussions sections of a research report are all about.