Tag Archives: research methods

OUT OF CONTROL (groups)! The weak link in the cause-&-effect chain

Welcome back after a bit of silence on my end!welcome[1]

In the last “Quasi-wha??” blogpost, I described 1 type of experimental design: Quasi-experimental.  To review… In quasi-experimental designs, the researcher manipulates some variable, but either 1) doesn’t randomly assign subjects to a control and experimental group OR 2) doesn’t have a control group at all.

For example, the researcher may introduce pet therapy on unit #1 and avoid pet therapy on unit #2 and then afterwards compare the anxiety levels of patients on the 2 units.  That study has a control group (unit #2), but because patients weren’t (& probably couldn’t be) randomly assigned to the units, this would be a quasi-experimental study. The control group in this pet therapy case is what researchers call a “non-equivalent control group.”   Non-equivalent means the groups are different in ways that might affect study results! [Note: For review of what constitutes a true experimental study see first part of  “Quasi-wha??” blogpost.]

WeaknessHerein lies a weak link in the cause-and-effect chain. Quasi- designs are NOT as strong as true experimental designs because something other than our treatment (in this case pet therapy) may have created any difference in outcomes (e.g., anxiety levels).  Why?   Here’s your answer.

ApplesOranges
Unit #1

In an experimental study, randomly assigning subjects to a

ApplesOranges
Unit #2

control and a separate experimental group means that all the little, variable weirdities of all subjects are equally distributed to each group.  Each group is the same mix of different types of people. This means we can assume that both groups are the exact same type of
people in regard to things that may influence study outcomes, such as attitudes, values, preferences, beliefs, anxiety level, psychology, physiology and so on.

ApplesOranges2
Unit #1=Apples.        Unit #2=Oranges

In contrast, in the quasi-experimental pet therapy example above, there is probably something that caused a certain type of person to be on unit #1 and a different type to be on unit #2.  Maybe it was their diagnosis, their doctor, their type of surgery, or other.  Thus, we cannot assume that people in unit #1 and unit #2 groups are the same before pet therapy, and so any differences between them after pet therapy might have already existed.

So why do quasi-experimental studies at all?? There are great reasons!  Stay tuned for next blogpost.

Critical thinking: Check out free full-text, quasi-experiment Gough et al., (2017). Tweet for Behavior Change: Using Social Media for the Dissemination of Public Health Messages.  

  1. What makes this a quasi-experimental design?  [Hint: Does it have a control group? Were subjects randomly assigned to groups?  Are both randomization & control group missing?]
  2. What might have caused the change in behavior, instead of the tweets? 
  3. What contribution do you think the study makes to improving practice?

For more information on studies with non-randomized control groups see “Of Mice & Cheese”  or comment below.  Let’s talk!

What are you asking? (or “Can HCAHPS sometimes be a DIRECT measure?”)

In a prior blog (Direct speaking about INdirect outcomes: HCAHPS as a measurement*), I argued that HCAHPS questions were indirect measures of outcomes.  Indirect measures are weaker than direct measures because they are influenced by tons of variables that have nothing to do with the outcome of interest.  But wait!! There’s more!  HCAPS can sometimes be a DIRECT measure; it all depends on what you want to know.

(If you know this, then you are way ahead of many when it comes to measuring outcomes accurately!!)

KEYKEY POINTS:

  • If your research question is what do patients remember about hospitalization then HCAHPS is a DIRECT measure of what patients remember.  
  • However if your research question is what did hospital staff actually do  then HCHAPS is an INDIRECT* measure of what staff did. 

What is HCAHPS?  HCAHPS (pronounced “H-caps”)  questions are patient perceptions of what happened, which may or may not be what actually happened.    Patients are asked to remember their care that happened in the past, and memories may be less than accurate. (See this link for more on what HCAHPS is: http://www.hcahpsonline.org/Files/HCAHPS_Fact_Sheet_June_2015.pdf )

Example:  HCAHPS question #16 is, “Before giving you any new medicine, how often did hospital staff tell you what the medicine was for?”    Whether the patient answers yes or thinkerno, the response tells us only how the patient remembers it.

Why is this important?     

  • Because if you want to know whether or not RNs actually taught inpatients about their medications, then for the most direct & accurate measure you will have to observe RNs .
  • However, if you want to know whether patients remember RNs teaching them about discharge medications, then HCAHPS question #16 is one of the most direct & accurate measure of what they remember.

*FOR MORE INFORMATION on why you want to use DIRECT measures SanDiegoCityCollegeLearningResource_-_bookshelfsee https://discoveringyourinnerscientist.com/2016/11/04/direct-speaking-about-idirect-outcomes-hcahps-as-a-measurement/

CRITICAL THINKING Pick any HCAHPS question at this link and write a research question that for which it would be a DIRECT outcome measure: question(http://www.hcahpsonline.org/files/March%202016_Survey%20Instruments_English_Mail.pdf)

For your current project, how are you DIRECTLY measuring outcomes?

Ouch! Whose Pain Feels Worse?

levels-of-evidenceIs pain experience as diverse as our populations?  This week I came across an interesting meta-analysis.

A meta-analysis (MA) is one of the strongest types of evidence there is. Some place it at the top; others, 2nd after evidence-based clinical practice guidelines.  (For more on strength of  evidence, click here.)

MA is not merely a review of literature, but is a statistical integration of studies on the same topic.  MA that is based on integration of randomized controlled trials experiment(RCTs) or experimental studies is the strongest type of MA.  MA based on descriptive or non-experimental studies is  a little less strong, because it just describes things as they seem to be; & it cannot show that one thing causes another.

MA example: This brand, new MA included 41  peer-reviewed, English-language, experimental studies with humans:  Kim HJ, Yang GS, Greenspan JD, Downton KD, Griffith KA, Renn CL, Johantgen M, Dorsey SG. Racial and ethnic differences in experimental pain sensitivity: Systematic review and meta-analysis. Pain. 2016 Sep 24 [Epub ahead of print] doi: 10.1097/j.pain.0000000000000731. PMID: 27682208.    All 41 studies used experimental pain stimuli such as heat, cold, ischemic, electrical and others and compared differences between racial/ethnic groups.

Pain reliefMain findings?  “AAs [African Americans], Asians, and Hispanics had higher pain sensitivity compared to NHWs [non-Hispanic Whites], particularly lower pain tolerance, higher pain ratings, and greater temporal summation of pain.” (https://www.ncbi.nlm.nih.gov/pubmed/27682208)  (Temporal summation is the increase in subjective pain ratings as a pain stimulus is repeated again and again.)

Critical thinking:  Given that this is a well-done meta-analysis and that the pain was created by researchers in each study, how should this changequestion your practice?  Or should it?   How can you use the findings with your patients?  Should each patient be treated as a completely unique individual? Or what are the pros & cons of using this MA to give us a starting point with groups of patients?  [To dialogue about this, comment below.]

For more info? Request the full Kim et al. article via interlibrary loan from your med center or school Heart Bookslibrary using reference above.   It is available electronically pre-publication.   Also check out my blog on strength of different types of evidence.

Happy evidence hunting. -Dr.H

True or False: Experiment or Not

Experiments are the way that we confirm that one thing causes another.   If the study is not an experiment (or combined experiments in a meta-analysis), then the research does not show cause and effect. imagesCALQ0QK9

Experiments are one of the strongest types of research.

So…how can you tell a true experiment from other studies?   Hazel B can tell you in 4:04 and simple language at https://www.youtube.com/watch?v=x2i-MrwdTqI&index=1&list=PL7A7F67C6B94EB97E

Go for it!

[After watching video:  Note that the variable that is controlled by the researcher is call the Independent variable or Cause variable because it creates a change in something else. That something else that changes is the Dependent variable or Outcome variable.]Learning

CRITICAL THINKING:  

  1. Based on the video, can you explain why true experiments are often called randomized controlled trial (RCT)?
  2. Take a look at The Effect of the Physical and Mental Exercises During Hemodialysis on Fatigue: A Controlled Clinical Trial, that is free in full-text via PubMed. How does it meet the criteria of a true experiment as described by Hazel B in the video?

FOR MORE INFORMATION:   Go to “What’s an RCT Anyway?” (https://discoveringyourinnerscientist.wordpress.com/2015/01/23/whats-a-randomized-controlled-trial/ )

Self-Report Data: “To use or not to use. That is the question.”

[Note: The following was inspired by and benefited from Rob Hoskin’s post at http://www.sciencebrainwaves.com/the-dangers-of-self-report/]Penguins

If you want to know what someone thinks or feels, you ask them, right?

The same is true in research, but it is good to know the pros and cons of using the “self-report method” of collecting data in order to answer a research question.  Most often self-report is done in ‘paper & pencil’ or SurveyMonkey form, but it can be done by interview.

Generally self-report is easy and inexpensive, and sometimes facilitates research that might otherwise be impossible.  To answer well, respondents must be honest, have insight into themselves, and understand the questions.  Self-report is an important tool in much behavioral research.

But, using self-report to answer a research question does have its limits. People may tend to answer in ways that make themselves look good (social desirability bias), agree with whatever is presented (social acquiescence bias), or answer in either extreme terms (extreme response set bias) or always pick the non-commital middle Hypothesisnumbers.  Another problem will occur if the reliability  and validity of the self-report questionnaire is not established.  (Reliability is consistency in measurement and validity is the accuracy of measuring what it purports to measure.) Additionally, self-reports typically provide only a)ordinal level data, such as on a 1-to-5 scale, b) nominal data, such as on a yes/no scale, or c) qualitative descriptions in words without categories or numbers.  (Ordinal data=scores are in order with some numbers higher than others, and nominal data = categories. Statistical calculations are limited for both and not possible for qualitative data unless the researcher counts themes or words that recur.)

Gold_BarsAn example of a self-report measure that we regard as a gold standard for clinical and research data = 0-10 pain scale score.   An example of a self-report measure that might be useful but less preferred is a self-assessment of knowledge (e.g., How strong on a 1-5 scale is your knowledge of arterial blood gas interpretation?)  The use of it for knowledge can be okay as long as everyone understands that it is perceived level of knowledge.

Critical Thinking: What was the research question in this study? Malaria et al. (2016) Pain assessment in elderly with behavioral and psychological symptoms of dementia. Journal of Alzheimer’s Disease as posted on PubMed.gov questionat http://www.ncbi.nlm.nih.gov/pubmed/26757042 with link to full text.  How did the authors use self-report to answer their research question?  Do you see any of the above strengths & weaknesses in their use?

For more information: Be sure to check out Rob Hoskins blog: http://www.sciencebrainwaves.com/the-dangers-of-self-report/

 

 

METHODS in the Research Madness

[This is a re-post from 2014.  If you weren’t a reader then….read on…..]

fisheye booksResearch article sections are: Title, Abstract, Introduction/background,Methods, Results, Discussion, & Implications/Conclusions

METHODS =  Design, Sample, Setting, & Data collection instrument

Sometimes these above elements of METHODS are subheadings.

Sometimes not.

  • Key point #1: Design= overall plan for answering the question or proving the hypothesis.  KEYThe 2 basic types of design are 1) experimental & 2) non-experimental.   In experimental, the researcher does something to the subjects and measures the effects of that something.  In non-experimental, the research merely observes and describes what is happening without doing anything to change it.
  •  KEYKey point #2: Setting=where the study is conducted: home, hospital, office, classroom, on KEYan ocean cruise, or other.
  • Key point #3: Sample includes who/what subjects were in & excluded from the study; how many subjects were in the study; & whether subjects were selected using random methods or non-random methods.   In random selection every eligible subject has the same chance of being selected. That’s called probability sampling.  An example is drawing names from a hat.  In non-random selection only the most nearby subjects are asked to be in the study. That’s called non-probability or convenience sampling.  An example, using a clipboard to survey people who walk into a mall one day. [Note: Subjects can be people, animals, charts, hospitals, or nations.]

(Whew!….Enough for now.)

Critical Thinking Exercise:  Find the Design, Setting, & Sample in this excerpt of Methods from Mohammedkarimi et al, (2014): question

“A double-blind, randomized clinical trial (RCT) was performed among 90 adult patients with acute headache in Shahid Rahnemoon Emergency Center of Yazd city of Iran (45 patients in lidocaine group and 45 patients in placebo group). Patients with history of epilepsy, allergy to lidocaine, signs of skull base fracture, Glasgow Coma Scale (GCS) < 15, patients younger than 14 years and patients who had received any medication in previous 2 h were excluded.”

“Who’s in?” Sampling Made Simple

So many great resources are out there at your fingertips.   In the next few blogs, I’ll point you to some of them.Heart Books

If you have not yet discovered the hidden jewel of the tab, “Research Made Simple,” on the Evidence-Based Nursing website at http://ebn.bmj.com/ , you should check out its full text pdf articles. An especially nice resource if you are in graduate school or maybe even a BSN course on research.

Here is a sample of one from June 9, 2014 of the population of articles at EBN.

ball sample“Sample selection is a key factor in research design and can determine whether research questions will be answered. …It is critical to take the time to clearly identify the population of interest for the specific research question. Nursing researchers are usually interested in answering questions about very specific patient populations which can span an incredible array of possibilities applying to international, national, local and organisational contexts. Research populations closely reflect nursing specialties, some of which are gender (eg, pregnant women) and age specific (eg, adolescent diabetes). It is rarely feasible to conduct a study that reaches every patient in the population of interest, therefore a subset or sample of that population is selected for study.” (para 1-2 Retrieved from http://ebn.bmj.com/content/17/2/32.short?g=w_ebn_research_tab ) [note: I added the bold]

I invite you to use the comments section to post URLs for other great sites!

Critical Thinking: Read the previous blog on representatives & then QUESTIONdecide for yourself whether this single EBN article on sampling is representative of clear & simple explanations. Samples can be things, not just people. What questions should you ask me about my sampling procedure?

For More: Check out tab “Research Made Simple,” on the Evidence-Based Nursing website at http://ebn.bmj.com/ .   OR try this <5 minute simple sampling youtube video  https://www.youtube.com/watch?v=Gs-gLeYuDZw