Tag Archives: research methods

“Please answer….” (cont.)

What do people HATE about online surveys?   If you want to improve your response rates, check out SurveyMonkey Eric V’s (May Mail2017)  Eliminate survey fatigue: Fix 3 things your respondents hate 

For more info: Check out my earlier post “Please Answer!”


“Please answer!” – How to increase the odds in your favor when it comes to questionnaires

Self-report by participants is one of the most common ways that researchers collect data, yet it is fraught with problems.   Some worries for researchers are: “Will participants be honest or will they say what they think I want to hear?”   “Will they understand the DifferentGroupsquestions correctly?”  “Will those who respond (as opposed to those who don’t respond) have unique ways of thinking so that my respondents do not represent everyone well?” and a BIG worry “Will they even fill out and return the questionnaire?”

One way to solve at least the latter 2 problems is to increase the response rate, and Edwards et al (2009 July 8) reviewed randomized trials  to learn how to do just that!!Questionnaire faces

If you want to improve your questionnaire response rates, check it out!  Here is Edwards et al.’s plain language summary as published in Cochrane Database of Systematic Reviews, where you can read the entire report.

Methods to increase response to postal and electronic questionnaires

MailPostal and electronic questionnaires are a relatively inexpensive way to collect information from people for research purposes. If people do not reply (so called ‘non-responders’), the research results will tend to be less accurate. This systematic review found several ways to increase response. People can be contacted before they are sent a postal questionnaire. Postal questionnaires can be sent by first class post or recorded delivery, and a stamped-return envelope can be provided. Questionnaires, letters and e-mails can be made more personal, and preferably kept short. Incentives can be offered, for example, a small amount of money with Remember jpga postal questionnaire. One or more reminders can be sent with a copy of the questionnaire to people who do not reply.


Critical/reflective thinking:  Imagine that you were asked to participate in a survey.  Which of these strategies do you think would motivate or remind you to respond and why?

For more info read the full report: Methods to increase response to postal and electronic questionnaires


Words vs. Numbers: What does it all mean?

There are several ways to classify types of research.   One way is qualitative versus quantitative–in other words, WORD  vs. NUMBER data, methods, & analysis.

  1. Qualitative research focuses on words (or sometimes images) and their meanings.
  2. Quantitative research focuses on numbers or counting things and statistical analysis that yields probable meaning.

If you watch this short, easy-to-understand youtube clip, you’ll have all the basics that you need to understand these!   Enjoy!

Critical thinking:  Go to PubMed for this QUANTitative study on spiritual issues in care (https://www.ncbi.nlm.nih.gov/pubmed/28403299) and compare it to this PubMed QUALitative study (https://www.ncbi.nlm.nih.gov/pubmed/27853263) in terms of data, methods, & analysis)

For more information: See earlier posts

“Should you? Can you?”

ApplesOranges2Quasi-experiments are a lot of work, yet don’t have the same scientific power to show cause and effect, as do randomized controlled trials (RCTs).   An RCT would provide better support for any hypothesis that X causes Y.   [As a quick review of what quasi-experimental versus RCT studies are, see “Of Mice & Cheese” and/or “Out of Control (Groups).”]

So why do quasi-experimental studies at all?  Why not always do RCTs when we are testing cause and effect?  Here are 3 reasons:

#1  Sometimes ETHICALLY the researcher canNOT randomly assign subjects to a control Smokingand an experimental group.  If the researcher wants to compare health outcomes of smokers with non-smokers, the researcher cannot assign some people to smoke and others not to smoke!  Why?  Because we already know that smoking has significant harmful effects. (Of course, in a dictatorship, by using the police a researcher could assign them to smoke or not smoke, but I don’t think we wanna go there.)

#2 Sometimes PHYSICALLY the researcher canNOT randomly assign subjects to control & Country of Originexperimental groups.   If the researcher wants to compare health outcomes of
individuals from different countries, it is physically impossible to assign country of origin.

#3 Sometimes FINANCIALLY the researcher canNOT afford to assign subjects randomly PiggyBankto control & experimental groups.   It costs $ & time to get a list of subjects and then assign them to control & experimental groups using random numbers table or drawing names from a hat.

Thus, researchers sometimes are left with little alternative, but to do a quasi-experiment as the next best thing to an RCT, then discuss its limitations in research reports.

Critical Thinking: You read a research study in which a researcher recruits the 1st 100 patients on a surgical ward January-March quarter as a control group.  Then the researcher recruits the 2nd 100 patients on that same surgical ward April-June for the experimental group.  With the experimental group, the staff uses a new, standardized pain script for better pain communications.  Then the pain communication outcomes of each group are compared statistically.

  • Is this a quasi-experiment or a randomized controlled trial (RCT)?
  • What factors (variables) might be the same among control & experimental groups in this study?
  • What factors (variables) might be different between control & experimental groups that might affect study outcomes?
  • How could you design an ethical & possible RCT that would overcome the problems with this study?
  • Why might you choose to do the study the same way that this researcher did?

For more info: see “Of Mice & Cheese” and/or “Out of Control (Groups).”

OUT OF CONTROL (groups)! The weak link in the cause-&-effect chain

Welcome back after a bit of silence on my end!welcome[1]

In the last “Quasi-wha??” blogpost, I described 1 type of experimental design: Quasi-experimental.  To review… In quasi-experimental designs, the researcher manipulates some variable, but either 1) doesn’t randomly assign subjects to a control and experimental group OR 2) doesn’t have a control group at all.

For example, the researcher may introduce pet therapy on unit #1 and avoid pet therapy on unit #2 and then afterwards compare the anxiety levels of patients on the 2 units.  That study has a control group (unit #2), but because patients weren’t (& probably couldn’t be) randomly assigned to the units, this would be a quasi-experimental study. The control group in this pet therapy case is what researchers call a “non-equivalent control group.”   Non-equivalent means the groups are different in ways that might affect study results! [Note: For review of what constitutes a true experimental study see first part of  “Quasi-wha??” blogpost.]

WeaknessHerein lies a weak link in the cause-and-effect chain. Quasi- designs are NOT as strong as true experimental designs because something other than our treatment (in this case pet therapy) may have created any difference in outcomes (e.g., anxiety levels).  Why?   Here’s your answer.

Unit #1

In an experimental study, randomly assigning subjects to a

Unit #2

control and a separate experimental group means that all the little, variable weirdities of all subjects are equally distributed to each group.  Each group is the same mix of different types of people. This means we can assume that both groups are the exact same type of
people in regard to things that may influence study outcomes, such as attitudes, values, preferences, beliefs, anxiety level, psychology, physiology and so on.

Unit #1=Apples.        Unit #2=Oranges

In contrast, in the quasi-experimental pet therapy example above, there is probably something that caused a certain type of person to be on unit #1 and a different type to be on unit #2.  Maybe it was their diagnosis, their doctor, their type of surgery, or other.  Thus, we cannot assume that people in unit #1 and unit #2 groups are the same before pet therapy, and so any differences between them after pet therapy might have already existed.

So why do quasi-experimental studies at all?? There are great reasons!  Stay tuned for next blogpost.

Critical thinking: Check out free full-text, quasi-experiment Gough et al., (2017). Tweet for Behavior Change: Using Social Media for the Dissemination of Public Health Messages.  

  1. What makes this a quasi-experimental design?  [Hint: Does it have a control group? Were subjects randomly assigned to groups?  Are both randomization & control group missing?]
  2. What might have caused the change in behavior, instead of the tweets? 
  3. What contribution do you think the study makes to improving practice?

For more information on studies with non-randomized control groups see “Of Mice & Cheese”  or comment below.  Let’s talk!

What are you asking? (or “Can HCAHPS sometimes be a DIRECT measure?”)

In a prior blog (Direct speaking about INdirect outcomes: HCAHPS as a measurement*), I argued that HCAHPS questions were indirect measures of outcomes.  Indirect measures are weaker than direct measures because they are influenced by tons of variables that have nothing to do with the outcome of interest.  But wait!! There’s more!  HCAPS can sometimes be a DIRECT measure; it all depends on what you want to know.

(If you know this, then you are way ahead of many when it comes to measuring outcomes accurately!!)


  • If your research question is what do patients remember about hospitalization then HCAHPS is a DIRECT measure of what patients remember.  
  • However if your research question is what did hospital staff actually do  then HCHAPS is an INDIRECT* measure of what staff did. 

What is HCAHPS?  HCAHPS (pronounced “H-caps”)  questions are patient perceptions of what happened, which may or may not be what actually happened.    Patients are asked to remember their care that happened in the past, and memories may be less than accurate. (See this link for more on what HCAHPS is: http://www.hcahpsonline.org/Files/HCAHPS_Fact_Sheet_June_2015.pdf )

Example:  HCAHPS question #16 is, “Before giving you any new medicine, how often did hospital staff tell you what the medicine was for?”    Whether the patient answers yes or thinkerno, the response tells us only how the patient remembers it.

Why is this important?     

  • Because if you want to know whether or not RNs actually taught inpatients about their medications, then for the most direct & accurate measure you will have to observe RNs .
  • However, if you want to know whether patients remember RNs teaching them about discharge medications, then HCAHPS question #16 is one of the most direct & accurate measure of what they remember.

*FOR MORE INFORMATION on why you want to use DIRECT measures SanDiegoCityCollegeLearningResource_-_bookshelfsee https://discoveringyourinnerscientist.com/2016/11/04/direct-speaking-about-idirect-outcomes-hcahps-as-a-measurement/

CRITICAL THINKING Pick any HCAHPS question at this link and write a research question that for which it would be a DIRECT outcome measure: question(http://www.hcahpsonline.org/files/March%202016_Survey%20Instruments_English_Mail.pdf)

For your current project, how are you DIRECTLY measuring outcomes?

Ouch! Whose Pain Feels Worse?

levels-of-evidenceIs pain experience as diverse as our populations?  This week I came across an interesting meta-analysis.

A meta-analysis (MA) is one of the strongest types of evidence there is. Some place it at the top; others, 2nd after evidence-based clinical practice guidelines.  (For more on strength of  evidence, click here.)

MA is not merely a review of literature, but is a statistical integration of studies on the same topic.  MA that is based on integration of randomized controlled trials experiment(RCTs) or experimental studies is the strongest type of MA.  MA based on descriptive or non-experimental studies is  a little less strong, because it just describes things as they seem to be; & it cannot show that one thing causes another.

MA example: This brand, new MA included 41  peer-reviewed, English-language, experimental studies with humans:  Kim HJ, Yang GS, Greenspan JD, Downton KD, Griffith KA, Renn CL, Johantgen M, Dorsey SG. Racial and ethnic differences in experimental pain sensitivity: Systematic review and meta-analysis. Pain. 2016 Sep 24 [Epub ahead of print] doi: 10.1097/j.pain.0000000000000731. PMID: 27682208.    All 41 studies used experimental pain stimuli such as heat, cold, ischemic, electrical and others and compared differences between racial/ethnic groups.

Pain reliefMain findings?  “AAs [African Americans], Asians, and Hispanics had higher pain sensitivity compared to NHWs [non-Hispanic Whites], particularly lower pain tolerance, higher pain ratings, and greater temporal summation of pain.” (https://www.ncbi.nlm.nih.gov/pubmed/27682208)  (Temporal summation is the increase in subjective pain ratings as a pain stimulus is repeated again and again.)

Critical thinking:  Given that this is a well-done meta-analysis and that the pain was created by researchers in each study, how should this changequestion your practice?  Or should it?   How can you use the findings with your patients?  Should each patient be treated as a completely unique individual? Or what are the pros & cons of using this MA to give us a starting point with groups of patients?  [To dialogue about this, comment below.]

For more info? Request the full Kim et al. article via interlibrary loan from your med center or school Heart Bookslibrary using reference above.   It is available electronically pre-publication.   Also check out my blog on strength of different types of evidence.

Happy evidence hunting. -Dr.H

%d bloggers like this: