Risks, Resources, Readiness
3 things to consider when adapting or adopting research evidence to/in a particular practice setting according to Stetler (2001).
Check out the 1-minute video summary by DrH at https://www.instagram.com/martyhrn/
Tag Archives: nursing
Testing the Test (or an intro to “Does the measurement measure up?”)
When reading a research article, you may be tempted only to read the Introduction & Background, then go straight to the Discussion, Implications, and Conclusions at the end. You skip all those pesky, procedures, numbers, and p levels in the Methods & Results sections.
Perhaps you are intimidated by all those “research-y” words like content validity, construct validity, test-retest reliability, and Cronbach’s alpha because they just aren’t part of your vocabulary….YET!
WHY should you care about those terms, you ask? Well…let’s start with an example. If your bathroom scale erratically measured your weight each a.m., you probably would toss it and find a more reliable and valid bathroom scale. The quality of the data from that old bathroom scale would be useless in learning how much you weighed. Similarly in research, the researcher wants useful outcome data. And to get that quality data the person must collect it with a measurement instrument that consistently (reliably) measures what it claims to measure (validity). A good research instrument is reliable and valid. So is a good bathroom scale.
Let’s start super-basic: Researchers collect data to answer their research question using an instrument. That test or tool might be a written questionnaire, interview questions, an EKG machine, an observation checklist, or something else. And whatever instrument the researcher uses needs to give them correct data answers.
For example, if I want to collect BP data to find out how a new med is working, I need a BP cuff that collects systolic and diastolic BP without a lot of artifacts or interference. That accuracy in measuring BP only is called instrument validity. Then if I take your BP 3 times in a row, I should get basically the same answer and that consistency is called instrument reliability. I must also use the cuff as intended–correct cuff size and placement–in order to get quality data that reflects the subject’s actual BP.
The same thing is true with questionnaires or other measurement tools. A researcher must use an instrument for the intended purpose and in the correct way. For example, a good stress scale should give me accurate data about a person’s stress level (not their pain, depression, or anxiety)–in other words it should have instrument validity. It should measure stress without a lot of artifacts or interference from other states of mind.
NO instrument is 100% valid–it’s a matter of degree. To the extent that a stress scale measures stress, it is valid. To the extent that it also measures other things besides stress–and it will–it is less valid. The question you should ask is, “How valid is the instrument?” often on a 0 to 1 scale with 1 being unachievable perfection. The same issue and question applies to reliability.
Reliability & validity are interdependent. An instrument that yields inconsistent results under the same circumstances cannot be valid (accurate). Or, an instrument can consistently (reliably) measure the wrong thing–that is, it can measure something other than what the researcher intended to measure. Research instruments need both strong reliability AND validity to be most useful; they need to measure the outcome variable of interest consistently.
Valid for a specific purpose: Researchers must also use measurement instruments as intended. First, instruments are often validated for use with a particular population; they may not be valid for measuring the same variable in other populations. For example, different cultures, genders, professions, and ages may respond differently to the same question. Second, instruments may be valid in predicting certain outcomes (e.g., SAT & ACT have higher validity in predicting NCLEX success than does GPA). As Sullivan (2011) wrote: “Determining validity can be viewed as constructing an evidence-based argument regarding how well a tool measures what it is supposed to do. Evidence can be assembled to support, or not support, a specific use of the assessment tool.”
In summary….
- Instrument validity = how accurate the tool is in measuring a particular variable
- Instrument reliability = how consistently the tool measures whatever it measures
Fun Practice: In your own words relate the following article excerpt to the concept of validity? “To assess content validity [of the Moral Distress Scale], 10 nurses were asked to provide comments on grammar, use of appropriate words, proper placement of phrases, and appropriate scoring.“ From p.3, Ghafouri et al. (2021). Psychometrics of the moral distress scale in Iranian mental health nurses. BMC Nursing. https://doi.org/10.1186/s12912-021-00674-4
“Two roads diverged in a yellow wood…” R.Frost
TIME TO REPUBLISH THIS ONE:
Below is my adaptation of one of the clearest representations that I have ever seen of when the roads diverge into quality improvement, evidence-based practice, & research. Well done, Dr. E.Schenk PhD MHI, RN-BC!
“It’s a jungle out there:” Predatory Journals
Cool. You completed your project and now want to publish it.
Beware! Predatory journals are ready to snap up your work! It helps them, but not you.
What is a predatory journal? One that can eat you and your paper alive.
Predatory journals don’t meet quality peer-review standards.- The predator may post your manuscript online, which then vanishes from access.
- If you are writing an article and cite a predator-published manuscript, you undercut your own credibility.
- If you’re counting on the article for credit towards tenure, your personnel reviewers may toss a predator-published article aside. 0 credit for you & questions about your own credibility. [For more on this in nursing see: Owens, J.K. & Chinn, P. (2018, January 20). “Reference letters & the specter of publications in predatory journals, Nurse Author & Editor, 28(1), 2. Retrieved from http://naepub.com/peer-review/2018-28-1-2/]
Many predators are Open Access Journals. What are Open Access Journals? Ones “that use a funding model that does not charge readers or their institutions for access” (https://doaj.org/faq#definition).
Open Access Journals may be legitimate OR predatory.
How can you identify predatory journals? While perhaps not Mighty Mouse—yep, I’m showing my age—help from the Directory of Open Access Journals (DOAJ) is on the way. DOAJ provides a searchable list of LEGITIMATE open access journals (click here). A quick search for “nursing” yielded 7.
How does DOAJ define quality of journals? Quality open access journals “must exercise peer-review with an editor and an editorial board or editorial review (particularly in the Humanities) carried out by at least two editors” (https://doaj.org/faq#definition).
Is there a list of predatory open access journals? YES. To see one that is updated, click here. Also, you can help! If you find an open access journal that claims to have the DOAJ quality seal, but isn’t on the DOAJ legitimate journal list, DOAJ wants to hear from you!
For more on DOAJ, see https://doaj.org/: “DOAJ is a community-curated online directory that indexes and provides access to high quality, open access, peer-reviewed journals. DOAJ is independent.”
Stay safe. As Randy Newman sings, “It’s a jungle out there.”
-Dr.H
Want to change the world? Make a list
For new graduate RNs (& those who help them) entering the workforce, Dr. Pat Benner PhD RN FAAN, who wrote Novice to Expert, has some great, very practical advice: 
https://www.youtube.com/watch?v=yxsBVPxS_zg (1:56) (hint: Remember the only behavior you can control is your own!)
And…it’s pretty good advice for any who assume a new job, too!
For More Information: If you want to know what it feels like sometimes to be a new 
grad RN, check out the 2 main themes and the subthemes voiced by new graduates in the free full-text: Hussein et al., (2017). New graduate nurses‘ experiences in a clinical specialty: a follow up study of newcomer perceptions of transitional support. BMC Nursing, 16(42). doi: 10.1186/s12912-017-0236-0. eCollection 2017.
Critical Thinking: Whether or not you are a new grad, did you have experiences similar to those in Hussein’s study? Can you use Benner’s suggestions to deal with the issues?
Listen up! Don’t interrupt!
Researchers collect two types of data in their studies
- Numbers (called quantitative data)
- Words & narratives (called qualitative data)
One source of rich word or narrative (qualitative) data for answering nursing questions is nurses’ stories. Dr. Pat Benner RN, author of Novice to Expert explains two things we can do to help nurses fully tell their stories so we can learn the most from their practice.
- Listen well without interrupting
- Help nurses ‘unpack’ their stories
Check out this excellent 2:59 video of Dr. Benner’s and revolutionize how you learn about nursing from nursing stories: Preview: The use of Narratives
Critical thinking: For a study using narratives in research see Leboul et al. (2017). Palliative sedation challenging the professional competency of health care providers and staff: A qualitative focus group and personal written narrative study. [full text available thru PubMed at https://www.ncbi.nlm.nih.gov/pubmed/28399846]. 1) Do you think the authors listened and unpacked information from the focus groups & written narratives; 2) Do you think there might be a difference in the way people write narratives and verbally tell narratives? 3) How might that difference if any affect the research findings?
For more information: Check out The Power of Story by Wang & Geale (2015) at http://www.sciencedirect.com/science/article/pii/S2352013215000496
“Should you? Can you?”
Quasi-experiments are a lot of work, yet don’t have the same scientific power to show cause and effect, as do randomized controlled trials (RCTs). An RCT would provide better support for any hypothesis that X causes Y. [As a quick review of what quasi-experimental versus RCT studies are, see “Of Mice & Cheese” and/or “Out of Control (Groups).”]
So why do quasi-experimental studies at all? Why not always do RCTs when we are testing cause and effect? Here are 3 reasons:
#1 Sometimes ETHICALLY the researcher canNOT randomly assign subjects to a control 
and an experimental group. If the researcher wants to compare health outcomes of smokers with non-smokers, the researcher cannot assign some people to smoke and others not to smoke! Why? Because we already know that smoking has significant harmful effects. (Of course, in a dictatorship, by using the police a researcher could assign them to smoke or not smoke, but I don’t think we wanna go there.)
#2 Sometimes PHYSICALLY the researcher canNOT randomly assign subjects to control & 
experimental groups. If the researcher wants to compare health outcomes of
individuals from different countries, it is physically impossible to assign country of origin.
#3 Sometimes FINANCIALLY the researcher canNOT afford to assign subjects randomly 
to control & experimental groups. It costs $ & time to get a list of subjects and then assign them to control & experimental groups using random numbers table or drawing names from a hat.
Thus, researchers sometimes are left with little alternative, but to do a quasi-experiment as the next best thing to an RCT, then discuss its limitations in research reports.
Critical Thinking: You read a research study in which a researcher recruits the 1st 100 patients on a surgical ward January-March quarter as a control group. Then the researcher recruits the 2nd 100 patients on that same surgical ward April-June for the experimental group. With the experimental group, the staff uses a new, standardized pain script for better pain communications. Then the pain communication outcomes of each group are compared statistically.
- Is this a quasi-experiment or a randomized controlled trial (RCT)?
- What factors (variables) might be the same among control & experimental groups in this study?
- What factors (variables) might be different between control & experimental groups that might affect study outcomes?
- How could you design an ethical & possible RCT that would overcome the problems with this study?
- Why might you choose to do the study the same way that this researcher did?
For more info: see “Of Mice & Cheese” and/or “Out of Control (Groups).”
Quasi- wha??
Two basic kinds of research design exist:
- Experimental design in which
- the researcher manipulates some variable,
- the participants are randomly assigned to groups, &
- one group is a control group that gets a placebo or some inert treatment so that outcomes in that group can be compared to the group(s) that did get the treatment.
- the researcher manipulates some variable,
- Non-experimental design in which the researcher doesn’t manipulate anything, but just observes & records what is going on. Some of these are descriptive, correlational, case, or cohort study designs for example.
One particularly interesting “experimental” design is one in which 1 or 2 of the experimental design ideal requirements as listed above are missing. These are called quasi-experimental designs.
In a quasi experimental design
- The researcher manipulates some variable, but….
- Either the participants are NOT randomly assigned to groups
- &/OR there is no control group.
A quasi-experimental design is not as strong as a true experiment in showing that the manipulated variable X causes changes in the outcome variable Y. For example, a true experimental study with manipulation, randomization, and a control group would create much stronger evidence that hospital therapy dogs really reduced patient pain and anxiety. We would not be as confident in the results of a quasi-experimental design examining the exact same thing. In the next blog, we’ll examine why.
For more info: Check out earlier blog: “What is an RCT anyway?” at https://discoveringyourinnerscientist.com/2015/01/23/whats-a-randomized-controlled-trial/
Critical thinking: Go to PubMed & use search terms “experiment AND nurse” (without the quotation marks). Open an interesting abstract and look for the 3 elements of a classic experimental design. Now look for “quasi experiment AND nurse” (without the quotation marks.) See what element is missing!
So you want to do a research study…..
So you want to do a research study? Wonderful!
Here are 5 bits of advice to get started:
- If you haven’t done a scientific research study before or don’t have a PhD, then realize that your project will go much more smoothly if you consult with a PhD or someone with experience.
You bring the great clinical ideas, & the experienced researcher will bring research design expertise. The design is the overall research plan for getting and analyzing the data to answer your question or to find out how well your new ideas work. That person 
will know the technical things you need to plan into your study in order to make the study ‘sparkle’ and to get approval from human subjects review committees. The person doesn’t have to be an expert on your topic. You fill that role, or soon will!
- If you have access to a librarian who is good at helping you look for current literature, s/he is one of your Best friends in getting a project done.
Searching for on-target literature from the millions of publications out there takes some special skills. Of course you can learn these on your own, but how much nicer to talk with a librarian about the key ideas in your project and allow them to use their special skills to help you. As an experience researcher, I can tell you that good 
librarians are worth their weight in gold! Librarians can help you find what others have learned about your topic already, and then you can build on that knowledge. [note: check out Finding the Needles in the Haystacks: Evidence Hunting Efficiently & Effectively for more]
- Because it’s your first foray into research, you might want to stick with a descriptive study.
What does that mean? It means that you will collect data about what the current situation is. For example, you might measure the average days to return of bowels sounds on your unit, OR the number of minutes it takes to do some task, OR the interruptions of patient sleep during the night. 
This will help you to establish whether or not there really is a problem to be solved. Descriptive studies are much simpler to conduct and analyze than experimental studies in which you measure something, make an improvement, and then see if the improvement improved things. For example, you would measure sleep interruptions, institute a quiet time, and then measure sleep interruptions again to see if there were fewer. [check out “What it is.” – a primer on descriptive studies for more]
- Pick a topic you are really jazzed about!
Every researcher from time to time can feel ‘bogged down’ or bored with what they are doing, & one of the best protections against that is making sure you think the topic is super-interesting in the first place. If you get a little bored or stuck later don’t be surprised; it just means you’re pretty normal. Those stuck times might even feel like “hitting the wall” in a long race, and once you get past it things get better. Remind yourself why you loved the topic in the first place. Talk to your PhD friend or a mentor for encouragement. Take a little break. Read something really interesting about your topic.
- Have fun!
While not every step of the research study process will make you want to jump up, sing, and dance, the process as a whole is really rewarding and great fun. You will be empowered by new learning—not just about your topic, but about how to do research!
Critical thinking: What’s a topic of interest to YOU? Write a descriptive question that you could answer with research. (Check out You Got A Problem With That? Try PICO*for more help.)
What are you asking? (or “Can HCAHPS sometimes be a DIRECT measure?”)
In a prior blog (Direct speaking about INdirect outcomes: HCAHPS as a measurement*), I argued that HCAHPS questions were indirect measures of outcomes. Indirect measures are weaker than direct measures because they are influenced by tons of variables that have nothing to do with the outcome of interest. But wait!! There’s more! HCAPS can sometimes be a DIRECT measure; it all depends on what you want to know.
(If you know this, then you are way ahead of many when it comes to measuring outcomes accurately!!)
KEY POINTS:
- If your research question is what do patients remember about hospitalization then HCAHPS is a DIRECT measure of what patients remember.
- However if your research question is what did hospital staff actually do then HCHAPS is an INDIRECT* measure of what staff did.
What is HCAHPS? HCAHPS (pronounced “H-caps”) questions are patient perceptions of what happened, which may or may not be what actually happened. Patients are asked to remember their care that happened in the past, and memories may be less than accurate. (See this link for more on what HCAHPS is: http://www.hcahpsonline.org/Files/HCAHPS_Fact_Sheet_June_2015.pdf )
Example: HCAHPS question #16 is, “Before giving you any new medicine, how often did hospital staff tell you what the medicine was for?” Whether the patient answers yes or 
no, the response tells us only how the patient remembers it.
Why is this important?
- Because if you want to know whether or not RNs actually taught inpatients about their medications, then for the most direct & accurate measure you will have to observe RNs .
- However, if you want to know whether patients remember RNs teaching them about discharge medications, then HCAHPS question #16 is one of the most direct & accurate measure of what they remember.
*FOR MORE INFORMATION on why you want to use DIRECT measures 
see https://discoveringyourinnerscientist.com/2016/11/04/direct-speaking-about-idirect-outcomes-hcahps-as-a-measurement/
CRITICAL THINKING: Pick any HCAHPS question at this link and write a research question that for which it would be a DIRECT outcome measure: 
(http://www.hcahpsonline.org/files/March%202016_Survey%20Instruments_English_Mail.pdf)
For your current project, how are you DIRECTLY measuring outcomes?
Discovering Your Inner Scientist 