Tag Archives: research critique

True or False: Experiment or Not

Experiments are the way that we confirm that one thing causes another.   If the study is not an experiment (or combined experiments in a meta-analysis), then the research does not show cause and effect. imagesCALQ0QK9

Experiments are one of the strongest types of research.

So…how can you tell a true experiment from other studies?   Hazel B can tell you in 4:04 and simple language at https://www.youtube.com/watch?v=x2i-MrwdTqI&index=1&list=PL7A7F67C6B94EB97E

Go for it!

[After watching video:  Note that the variable that is controlled by the researcher is call the Independent variable or Cause variable because it creates a change in something else. That something else that changes is the Dependent variable or Outcome variable.]Learning


  1. Based on the video, can you explain why true experiments are often called randomized controlled trial (RCT)?
  2. Take a look at The Effect of the Physical and Mental Exercises During Hemodialysis on Fatigue: A Controlled Clinical Trial, that is free in full-text via PubMed. How does it meet the criteria of a true experiment as described by Hazel B in the video?

FOR MORE INFORMATION:   Go to “What’s an RCT Anyway?” (https://discoveringyourinnerscientist.wordpress.com/2015/01/23/whats-a-randomized-controlled-trial/ )


Self-Report Data: “To use or not to use. That is the question.”

[Note: The following was inspired by and benefited from Rob Hoskin’s post at http://www.sciencebrainwaves.com/the-dangers-of-self-report/]Penguins

If you want to know what someone thinks or feels, you ask them, right?

The same is true in research, but it is good to know the pros and cons of using the “self-report method” of collecting data in order to answer a research question.  Most often self-report is done in ‘paper & pencil’ or SurveyMonkey form, but it can be done by interview.

Generally self-report is easy and inexpensive, and sometimes facilitates research that might otherwise be impossible.  To answer well, respondents must be honest, have insight into themselves, and understand the questions.  Self-report is an important tool in much behavioral research.

But, using self-report to answer a research question does have its limits. People may tend to answer in ways that make themselves look good (social desirability bias), agree with whatever is presented (social acquiescence bias), or answer in either extreme terms (extreme response set bias) or always pick the non-commital middle Hypothesisnumbers.  Another problem will occur if the reliability  and validity of the self-report questionnaire is not established.  (Reliability is consistency in measurement and validity is the accuracy of measuring what it purports to measure.) Additionally, self-reports typically provide only a)ordinal level data, such as on a 1-to-5 scale, b) nominal data, such as on a yes/no scale, or c) qualitative descriptions in words without categories or numbers.  (Ordinal data=scores are in order with some numbers higher than others, and nominal data = categories. Statistical calculations are limited for both and not possible for qualitative data unless the researcher counts themes or words that recur.)

Gold_BarsAn example of a self-report measure that we regard as a gold standard for clinical and research data = 0-10 pain scale score.   An example of a self-report measure that might be useful but less preferred is a self-assessment of knowledge (e.g., How strong on a 1-5 scale is your knowledge of arterial blood gas interpretation?)  The use of it for knowledge can be okay as long as everyone understands that it is perceived level of knowledge.

Critical Thinking: What was the research question in this study? Malaria et al. (2016) Pain assessment in elderly with behavioral and psychological symptoms of dementia. Journal of Alzheimer’s Disease as posted on PubMed.gov questionat http://www.ncbi.nlm.nih.gov/pubmed/26757042 with link to full text.  How did the authors use self-report to answer their research question?  Do you see any of the above strengths & weaknesses in their use?

For more information: Be sure to check out Rob Hoskins blog: http://www.sciencebrainwaves.com/the-dangers-of-self-report/



Telling the Future: The Research Hypothesis

What is a research hypothesis?   A research hypothesis is a predicted answer; an educated guess.  It is a statement of the outcome that a researcher expects to find in an experimental study.Hypothesis

Why care?  Because it tells you precisely the problem that the research study is about!  Either the researcher’s prediction turns out to be true (supported by data) or not!
A hypothesis includes 3 key elements: 1) the population of interest, 2) the experimental treatment, & 3) the outcome expected.  It is a statement of cause and effect. The experimental treatment that the researcher manipulates is called the independent or cause variable.  The result of the study is an outcome that is called the dependent variable because it depends on the independent/cause variable.

For example, let’s take the hypothesis “Heart failure patients who receive exmeds2perimental drug X will have better cardiac function than will heart failure patients who receive standard drug Y.”  You can see that the researcher is manipulating the drug (independent variable) that patients will receive.  And patient cardiac outcomes are expected to vary—in fact cardiac function is expected to be better—for patients who receive the experimental drug X.

Ideally that researcher will randomly assign subjects to an experimental group that receives drug X and a control group that receives standard therapy drug Y.   Outcome cardiac function data will be collected and analyzed to see if the researcher’s predicted answer (AKA hypothesis) is true.

In a research article, the hypothesis is usually stated right at the end of the introduction or background section.

If you see a hypothesis, how can you tell what is the independent/cause variable and the dependent/effect/outcome variable?question   1st – Identify the population in the hypothesis—the population does not vary (& so, it is not a variable).   2nd – Identify the independent variable–This will be the one that is the cause & it will vary.  3rd – Identify the dependent variable–This will be the one that is the outcome & its variation depends on changes/variation in the independent variable.

PRACTICE:  What are the population, independent variable(s) & dependent variable(s) in these actual research study titles that reflect the research hypotheses:

FOR MORE INFORMATION:  See SlideShare by Domocmat (n.d.) Formulating hypothesis at http://www.slideshare.net/kharr/formulating-hypothesis-cld-handout


Introduction to Introductions!

I have a lot of new readers, so let’s revisit the standard sections of a research article.  They are:

  • Introduction (or Background)
  • Review of literature
  • Methods
  • Results (or findings)
  • Discussion & Implications
  • Conclusion

If we begin at the beginning, then we should ask: “What’s in an Introduction?”  Here’s the answer:

“[a] …Background of the problem or issue being examined,

[b] …Existing literature on the subject, and

[c] …Research questions, objectives, and possibly hypothesis” (p. 6, Davies & Logan, 2012)

This is the very 1st section of the body of the research article.  In it you will find a description of the problem that the researcher is studying, why the problem is a priority, and sometimes what is already known about the problem.  The description of what is already known may or may not be labelled separately as a Review of Literature.

KEYKey point #1: Articles & research that are reviewed in the Intro/Background should be mostly within the past 5-7 years.  Sometimes included are classic works that may be much older OR sometimes no recent research exists.   If recent articles aren’t used, this should raise some questions in your mind.   You know well that healthcare changes all the time!!  If there are no recent studies the author should explain.

Key point #2The last sentence or two in the Intro/Background is the research question or hypothesis.  If you need to know the research question/hypothesis right away, you can skip straight to the end of the Intro/background—and there it should be!

Happy research reading!

Critical Thinking: Do the sections of the abstract AND the sections of the research article match above headings?  Does it match the description of Introduction? Take a look at the free article by Kennedy et al. (2014). Is there a relationship between personality and choice of nursing specialty: An integrative literature, BMC Nursing, 13(40). Retrieved from the link http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4267136/.  question


“The Sky is Falling!” (or Don’t be an EBP Chicken Little)

We all know the story of Chicken Little, right?  Chicken Little is walking through the forest, an acorn falls and hits her on the head, then Chicken Little  runs about in a panic telling everyone, “The sky is falling! The sky is falling!” A lot of the animals are convinced, and the fox—who knows the truth that it was only an acorn—convinces Chicken Little & some other animals to come into his den to be safe from the falling sky. There he eats them. Interestingly the fox used the correct evidence well. Chicken Little & fox chicken littlecompany used evidence poorly and created a safety hazard for themselves!

Moral of the story? Don’t be a Chicken Little when it comes to reading and applying research to practice. Get all the facts before you share the research findings with others. Don’t read only the “acorn” of abstract, introduction, and discussion, and then assume that you know what the research study shows and that you can apply it to your work. Don’t turn an acorn into a falling sky!

How to avoid being an EBP Chicken Little? To avoid being an Evidence-Based Practice (EBP) Chicken Little, you should follow the example of Samantha in this research fairy tale: “Samantha…read the study abstract. Then, while Chicken Little and her friends waited anxiously, she read the introduction, the literature review, the research questions, the methods section, the findings, and the discussion section. Then she went back and read all the sections again. Finally, as Chicken Little hopped around her impatiently, she reread the findings. “Chicken Little, have you and your friends read the entire study?” asked Samantha.” (source: https://www.son.rochester.edu/student-resources/research-fables/chicken-little.html)

Why go to all this trouble? I’m busy. The reasons to take time and effort to read the WHOLE study are many. First, the subjects may not be at all like your own patient population—what if the researchers studied only “left-pawed albino hamsters”? Second, the research might not be a strong meta-analysis or randomized controlled trial whose results can actually be applied to other times and places—what if the researchers just watched subjects walk around, but didn’t test what makes them walk better?  A third reason is that the results might be statistically significant, but clinically irrelevant!—what if researchers were studying pain, but everyone in the study had 1-2 on the pain scale?

You don’t want to endanger patient safety by misunderstanding and misapplying research and then be “eaten alive” by adverse patient outcomes or by critics, who will see through your mistakes. Remember in the fairy tale Chicken Little and his careless friends misunderstood the facts, and hence were susceptible to being eaten by a fox.

What if you don’t know how to read research? No problem. Everyone who knows how to read research now had to learn it—no one was born knowing.  So,…you can learn it, too!  It doesn’t take magical powers.  Countless resources are online; others are in your hospital or in a university research course. If you check the box on this page to follow the EBP blog, (I hope) it will help, too. Go back and read earlier blogs on sections of a research report.

For more information on how to be an EBP Chicken Little (NOT) see the very creative research fairy tale by Jeanne Grace (copyright Rochester College) at https://www.son.rochester.edu/student-resources/research-fables/chicken-little.html

Critical thinking:

  1. After reading Grace’s fairy tale at the above link list at least three (3) things that Chicken Little might have learned, had she read the whole article!
  2. Compare an abstract with a full article, and check out the differences. Specifically compare the abstract at  http://www.ncbi.nlm.nih.gov/pubmed/2606078 with what you learn about them from the full article at http://www.ncbi.nlm.http://www.ncbi.nlm.nih.gov/pubmed/2606078nih.gov/pmc/articles/PMC4449996/. Did reading the whole article change the way you understand how orQUESTION whether the study might apply to your work? If so, how? And if not, why not?

“Is it?” “It is!” Expert opinion as valuable evidence for practice.

Remember back when you asked your mom why you should make your bed, set the table, or do some other then-distasteful task? Maybe you said, “Do I have to?”

Because I Said SoRemember her answer? Sometimes it was just: “Because I said so!” Was that enough evidence to support your practice of setting the table or making your bed?  You bet! After all she was THE expert on such things.

Likewise…is expert opinion good evidence for your practice? Yes, it is. EXPERT OPINION of individuals or committees is the 7th level of evidence for nursing practice (Melnyk & Fineout-Overholt, 2005), and should be considered.

Of course the first question that you must ask is: “Is the person/committee (who is telling you how to prevent falls, promote safety, teach patients, and so on and on) an actual EXPERT on that topic?” The answer is a matter of judgment. If the person/committee has special education, credentials, or experience or is a recognized authority on the topic about which they are giving advice, then you could reasonably conclude yes, they are experts. In that case the advice should be considered evidence for practice.    (Caution: Your judgment of their expertise matters!–don’t just follow along.  Don’t forget that person who is expert in one area may not be an expert in another.)

The 2nd question that you must ask is; “Does any research or stronger level of evidence exist on the topic?”

  • If it does NOT exist, then you should use that expert opinion in combination with scientific principles, anecdotal case reports, and theory. Or you might create some new research yourself. (Source=Iowa EBP Model)
  • If it DOES EXIST, then you should pay most attention to the stronger evidence and interpret the weaker evidence of expert opinion in that light.

QUESTIONCritical thinking:  Try your new knowledge in this example. Many educators and professionals who run journal clubs consider journal clubs effective based on feedback from participants. At least in 2008, 80% of experimental studies suggested that journal clubs helped with learning and being able to critically review a research article. However, no research is available on whether the learning from journal clubs actually translates into practice (Deenadayalan et al., 2008). You are considering a journal club. What would you decide to do and why?

For more, see:

“Watch & Learn!” – Systematic Reviews of Non-experimental Studies

Today’s top tip: Want to find the strongest research evidence for your project?   Go to http://www.ncbi.nlm.nih.gov/pubmed & add the strongest type of research designs as one of your search terms. For example, add the terms meta-analysis or systematic review to your other search terms. **********************************************

Now to the new!  What is a systematic review of descriptive studies? [Note: For information on stronger levels of research “I like my coffee (and my evidence) strong!)]Cat Fishbowl2

First, remember that in a descriptive study, the researcher merely watches or listens to see what is happening. Descriptive studies do not test interventions.

Second, a systematic review (not to be too silly) is a review that is done systematically in order to include all literature on a particular topic . The authors will tell us where they searched for studies, what search terms they used, and what years they searched. That way we can feel sure that all relevant articles are included.

Therefore, in a systematic review of descriptive studies the authors

  • Collect non-experimental studies related to the problem they are trying to solve,
  • Critically review them, &
  • Write up that analysis for you and me.

You won’t see a lot of numbers or statistics in these reviews of non-experimental studies.

Systematic review of descriptive studies are weaker than other levels of evidence in part because they are critical reviews of non-experimental studies in which the researchers only observed subjects. Those non-experimental studies that they are reviewing may be quantitative with results reported in numbers or qualitative with results reported in words.

Here’s an example with results reported in words (qualitative): Yin, Tse, & Wong (2015) systematically reviewed studies for what factors affect RNs giving PRN opioids in the postop period.   They searched publications 2000-2012 and ended up with 39 relevant studies. Within those 39 articles were descriptive studies that identified 4 basic influences on opioid PRN administration by RNs to postop patients: “(i) nurses’ knowledge and attitudes about pain management; (ii) the situation of nurses’ work practices in administrating range orders for opioid analgesics; (iii) factors that influenced nurses’ work practices; and (iv) perceived barriers to effective pain management from the nurse’s perspective.” [note: In this study a few of the 39 studies were experimental in which something was done to subjects and then outcomes measured, and Yin et al., commented separately on what those showed.]

Critical thinking: What are key differences between a meta-analysis of randomized controlled trials and a systematic review of QUESTIONdescriptive studies?

Reference found with search terms: review of descriptive studies nursing pain – Yin, H.H.,Tse, M.M., & Wong, F.K. (2015). Systematic review of the predisposing, enabling, and reinforcing factors which influence nursing administration of opioids in the postoperative period. Japan Journal of Nursing Science, doi: 10.1111/jjns.12075.