Tag Archives: research methods

Correlation research, non-experimental research, Uncategorized

Correlation Studies: Primer on Design Part 2

Leave a comment

REMEMBER:

Research design = overall plan for a study.

The 2 major categories of research study design are:

  1. Non-experimental, observation-only studies, &
  2. Experimental testing of an intervention studies.

Correlation study designs are in that first category. Correlation studies focus on whether changes in at least one variable are statistically related to changes in another. In other words, do two or more variables change at the same time.

Such studies do not test whether one variable causes change in the other. Instead they are analogous to the chicken-and-egg dilemma in which one can confirm that the number of chickens and eggs are related to each other, but no one can say which came first or which caused the other. Correlation study questions may take this form, “Is there a relationship between changes in [variable x] and changes in [variable y]?” while a correlation hypothesis might be a prediction that, “As [variable x] increases, [variable y] decreases.”

An example of a question appropriate to this design is, “Are nurses’ age and educational levels related to their professional quality of life?” Sometimes a yet-unidentified, mediating variable may be creating the changes in one or all correlated variables. For example, rising nurse age and education may make them likely to choose certain work settings with high professional quality of life; this means the mediating variable of work setting—not age or education—might be creating a particular professional quality of life.

Alert! Correlation is not causation.
Data collection, nursing research, Outcome measurement, quantitative research, reading research, Reliability & validity, research, research methods

Testing the Test (or an intro to “Does the measurement measure up?”)

Leave a comment

When reading a research article, you may be tempted only to read the Introduction & Background, then go straight to the Discussion, Implications, and Conclusions at the end. You skip all those pesky, procedures, numbers, and p levels in the Methods & Results sections.

Perhaps you are intimidated by all those “research-y” words like content validity, construct validity, test-retest reliability, and Cronbach’s alpha because they just aren’t part of your vocabulary….YET!

WHY should you care about those terms, you ask? Well…let’s start with an example. If your bathroom scale erratically measured your weight each a.m., you probably would toss it and find a more reliable and valid bathroom scale. The quality of the data from that old bathroom scale would be useless in learning how much you weighed. Similarly in research, the researcher wants useful outcome data. And to get that quality data the person must collect it with a measurement instrument that consistently (reliably) measures what it claims to measure (validity). A good research instrument is reliable and valid. So is a good bathroom scale.

Let’s start super-basic: Researchers collect data to answer their research question using an instrument. That test or tool might be a written questionnaire, interview questions, an EKG machine, an observation checklist, or something else. And whatever instrument the researcher uses needs to give them correct data answers.

For example, if I want to collect BP data to find out how a new med is working, I need a BP cuff that collects systolic and diastolic BP without a lot of artifacts or interference. That accuracy in measuring BP only is called instrument validity. Then if I take your BP 3 times in a row, I should get basically the same answer and that consistency is called instrument reliability. I must also use the cuff as intended–correct cuff size and placement–in order to get quality data that reflects the subject’s actual BP.

The same thing is true with questionnaires or other measurement tools. A researcher must use an instrument for the intended purpose and in the correct way. For example, a good stress scale should give me accurate data about a person’s stress level (not their pain, depression, or anxiety)–in other words it should have instrument validity. It should measure stress without a lot of artifacts or interference from other states of mind.

NO instrument is 100% valid–it’s a matter of degree. To the extent that a stress scale measures stress, it is valid. To the extent that it also measures other things besides stress–and it will–it is less valid. The question you should ask is, “How valid is the instrument?” often on a 0 to 1 scale with 1 being unachievable perfection. The same issue and question applies to reliability.

Reliability & validity are interdependent. An instrument that yields inconsistent results under the same circumstances cannot be valid (accurate). Or, an instrument can consistently (reliably) measure the wrong thing–that is, it can measure something other than what the researcher intended to measure. Research instruments need both strong reliability AND validity to be most useful; they need to measure the outcome variable of interest consistently.

Valid for a specific purpose: Researchers must also use measurement instruments as intended. First, instruments are often validated for use with a particular population; they may not be valid for measuring the same variable in other populations. For example, different cultures, genders, professions, and ages may respond differently to the same question. Second, instruments may be valid in predicting certain outcomes (e.g., SAT & ACT have higher validity in predicting NCLEX success than does GPA). As Sullivan (2011) wrote: “Determining validity can be viewed as constructing an evidence-based argument regarding how well a tool measures what it is supposed to do. Evidence can be assembled to support, or not support, a specific use of the assessment tool.”

In summary….

  1. Instrument validity = how accurate the tool is in measuring a particular variable
  2. Instrument reliability = how consistently the tool measures whatever it measures

Fun Practice: In your own words relate the following article excerpt to the concept of validity? “To assess content validity [of the Moral Distress Scale], 10 nurses were asked to provide comments on grammar, use of appropriate words, proper placement of phrases, and appropriate scoring.“ From p.3, Ghafouri et al. (2021). Psychometrics of the moral distress scale in Iranian mental health nurses. BMC Nursing. https://doi.org/10.1186/s12912-021-00674-4

Data collection, Disseminating findings, Ethics, pilot study, Research design, research methods

Pilot sTUdies: Look before you leap! (a priori vs. posthoc)

Leave a comment

Why does it matter if a study is labeled a “pilot”?

SHORT ANSWER: …Because a pilot is about testing research methods,….not about answering research questions.

If a project has “pilot” in the title, then you as a reader should expect a study that examines whether certain research methods work (methodologic research). Methods include things like timing of data collection, sampling strategies, length of questionnaire, and so on. Pilots suggest what methods will effectively to answer researchers’ questions. Advance prep in methods makes for a smooth research landing.

Small sample = Pilot? A PILOT is related to study goals and design–not sample size. Of course pilots typically have small samples, but a small sample does not a pilot study make. Sometimes journals may tempt a researcher to call their study a pilot because of small samples. Don’t go there. Doing so means after-the-fact, posthoc changes that were Not the original, a priori goals and design.

Practical problems? If researchers label a study a “pilot” after it is completed (post hoc), they raise practical & ethical issues. At a practical level, researchers must create feasibility questions & answers. (See NIH.) The authors should drop data analysis that answers their original research questions.

Ethics? This ethically requires researchers 1) to say they planned something that they didn’t or 2) to take additional action. Additional action may be complete transparency about the change and seeking modification to original human subjects’ committee approvals. An example of one human subjects issue is that you informed your subjects that their data would answer a particular research question, and now you want to use their data to answer something else–methods questions!

Options? You can just learn from your small study and go for a bigger one, including improving methods. Some journals will consider publication of innovative studies even when small.

Look first, then leap: Better to look a priori, before leaping. If you think you might have trouble with your methods, design a pilot. If you made the unpleasant discovery that your methods didn’t work as you hoped, you can 1) disseminate your results anyway or 2) rethink ethical and practical issues.

Who’s with me? The National Institutes of Health agree: https://nccih.nih.gov/grants/whatnccihfunds/pilot_studies . NIH notes that common misuses of “pilots” are determining safety, efficacy of intervention, and effect size.

Who disagrees? McGrath argues that clinical pilots MAY test safety and efficacy, as well as feasibility. (See McGrath, J. M. (2013). Not all studies with small samples are pilot studies, Journal of Perinatal & Neonatal Nursing, 27(4): 281-283. doi: 10.1097/01.JPN.0000437186.01731.bc )

Data collection, Evidence based nursing, evidence based practice, Methodologic research, Methods, research, Research design, research methods

Trial Balloons & Pilot Studies

Leave a comment

A pilot study is to research what a trial balloon is to politics. 

In politics, a trial balloon is communicating a law or policy idea via media to see how the intended audience reacts to it.  A trial balloon does not answer the question, “Would this policy (or law) work?” Instead a trial balloon answers questions like “Which people hate the idea of the policy/law–even if it would work?” or “What problems might enacting it create?” In other words, a trial balloon answers questions that a politician wants to know BEFORE implementing a policy so that the policy or law can be tweaked to be successfully put in place.

meeting2

In research, a pilot study is sort of like a trial balloon. It is “a small-scale test of the methods and procedures” of a planned full-scale study (Porta, Dictionary of Epidemiology, 5th edition, 2008). A pilot study answers questions that we want to know BEFORE doing a larger study, so that we can tweak the study plan and have a successful full-scale research project. A pilot study does NOT answer research questions or hypotheses, such as “Does this intervention work?”  Instead a pilot study answers the question “Are these research procedures workable?”

A pilot study asks & answers: “Can I recruit my target population? Can the treatments be delivered per protocol? Are study conditions acceptable to participants?” and so on.   A pilot study should have specific measurable benchmarks for feasibility testing. For example if the pilot is finding out whether subjects will adhere to the study, then adherence might be defined as  “70 percent of participants in each [group] will attend at least 8 of 12 scheduled group sessions.”  Sample size is based on practical criteria such as  budget, participant flow, and the number needed to answer feasibility questions (ie. questions about whether the study is workable).

A pilot study does NOT:  Test hypotheses (even preliminarily); Use inferential statistics; Assess safety of a treatment; Estimate effect size; Demonstrate safety of an intervention.

A pilot study is not just a small study.

Next blog: Why this matters!!

For more info read the source of all quotes in this blog: Pilot Studies: Common Uses and Misuses @ https://nccih.nih.gov/grants/whatnccihfunds/pilot_studies

Methods, Prospective design, Research design, Survey

After taste…I mean “after test”

Leave a comment

Let’s say you want to find out how well students’ think they learned theory in your class.

One option is to do a pre/post test: You distribute the same survey before and after the class asking them to rate on 1-4 scale how well they think they know the new material. Then you compare their ratings.

Another option is to do posttest only: You could give them a survey after the class that Surveyasks them to rate 1-4 their knowledge before the class and 1-4 their knowledge now. Then you compare their ratings.

One research option is stronger than the other.  Which one is it? and Why?  (hint: think retrospective/prospective)

EBP, Evidence based nursing, evidence based practice, research, Research student, Uncategorized

2019: It is…….

Leave a comment

I’m not a New Year’s resolution person.  I used to be and then I realized that I wanted to hit the restart button more often than every 365 days.  So…my aim for this blog remains pretty much unchanged:   Make research processes and ideas understandable for every RN.

DifficultToBeSimpleAlthough “to be simple is difficult,” that’s my goal. Ljourneyet me know what’s difficult for you in research, because it probably is for others as well.  Let’s work on the difficult together so that you can use the BEST Evidence in your practice.

The 2019 journey begins today, and tomorrow, and the tomorrows after that!

FOR MORE: Go to PubMed. Search for a topic of interest. Send me the article & we’ll critique together.

evidence based practice, Evidence hierarchy

Dispelling the nice or naughty myth: retrospective observational study of Santa Claus.

2 Comments

Naughty NiceGo to full article

Abstract

“OBJECTIVE:  To determine which factors influence whether Santa Claus will visit children in hospital on Christmas Day.

DESIGN:  Retrospective observational study.

SETTING:  Paediatric wards in England, Northern Ireland, Scotland, and Wales.

PARTICIPANTS:  186 members of staff who worked on the paediatric wards (n=186) during Christmas 2015.

MAIN OUTCOME MEASURES:  Presence or absence of Santa Claus on the paediatric ward GIFTSduring Christmas 2015. This was correlated with rates of absenteeism from primary school, conviction rates in young people (aged 10-17 years), distance from hospital to North Pole (closest city or town to the hospital in kilometres, as the reindeer flies), and contextual socioeconomic deprivation (index of multiple deprivation).

RESULTS:  Santa Claus visited most of the paediatric wards in all four countries: 89% in England, 100% in Northern Ireland, 93% in Scotland, and 92% in Wales. The odds of him not visiting, however, were significantly higher for paediatric wards in areas of higher socioeconomic deprivation in England (odds ratio 1.31 (95% confidence interval 1.04 to 1.71) in England, 1.23 (1.00 to 1.54) in the UK). In contrast, there was no correlation with school absenteeism, conviction rates, or distance to the North Pole.

CONCLUSION:  The results of this study dispel the traditional belief that Santa Claus rewards children based on how nice or naughty they have been in the previous year. Santa Claus is less likely to visit children in hospitals in the most deprived areas. Potential solutions include a review of Santa’s contract or employment of local Santas in poorly represented region.”  Park et al. (2016).BMJ. 2016 Dec 14;355:i6355. doi: 10.1136/bmj.i6355.

How would you translate this into practice?   Questions to help you with this endeavor:   Where does this retrospective, observational research fall on the evidence hierarchy?  EBNIs it quantitative or qualitative research?  Experimental or non-experimental research? How generalizable is this research? What are the risks,resources, and readiness of people in potentially using the findings (Stetler & Marram, 1996; Stetler, 2001)?   What might happen if you try to apply the abstract information to practice without reading the full article?  Do you think the project done in Europe is readily applicable to America?  What would be the next level of research that you might undertake to better confirm these findings?
Enjoy your holiday season! -Dr H
Data collection, Questionnaires, research, research methods, Survey

“Please answer….” (cont.)

Leave a comment

What do people HATE about online surveys?   If you want to improve your response rates, check out SurveyMonkey Eric V’s (May Mail2017)  Eliminate survey fatigue: Fix 3 things your respondents hate 

For more info: Check out my earlier post “Please Answer!”

Cochrane, Literature review, Outcome measurement, Questionnaires, Reliability & validity, research methods

“Please answer!” – How to increase the odds in your favor when it comes to questionnaires

1 Comment

Self-report by participants is one of the most common ways that researchers collect data, yet it is fraught with problems.   Some worries for researchers are: “Will participants be honest or will they say what they think I want to hear?”   “Will they understand the DifferentGroupsquestions correctly?”  “Will those who respond (as opposed to those who don’t respond) have unique ways of thinking so that my respondents do not represent everyone well?” and a BIG worry “Will they even fill out and return the questionnaire?”

One way to solve at least the latter 2 problems is to increase the response rate, and Edwards et al (2009 July 8) reviewed randomized trials  to learn how to do just that!!Questionnaire faces

If you want to improve your questionnaire response rates, check it out!  Here is Edwards et al.’s plain language summary as published in Cochrane Database of Systematic Reviews, where you can read the entire report.

Methods to increase response to postal and electronic questionnaires

MailPostal and electronic questionnaires are a relatively inexpensive way to collect information from people for research purposes. If people do not reply (so called ‘non-responders’), the research results will tend to be less accurate. This systematic review found several ways to increase response. People can be contacted before they are sent a postal questionnaire. Postal questionnaires can be sent by first class post or recorded delivery, and a stamped-return envelope can be provided. Questionnaires, letters and e-mails can be made more personal, and preferably kept short. Incentives can be offered, for example, a small amount of money with Remember jpga postal questionnaire. One or more reminders can be sent with a copy of the questionnaire to people who do not reply.

 

Critical/reflective thinking:  Imagine that you were asked to participate in a survey.  Which of these strategies do you think would motivate or remind you to respond and why?

For more info read the full report: Methods to increase response to postal and electronic questionnaires

 

Methods, Qualitative research, quantitative research

Words vs. Numbers: What does it all mean?

Leave a comment

There are several ways to classify types of research.   One way is qualitative versus quantitative–in other words, WORD  vs. NUMBER data, methods, & analysis.

  1. Qualitative research focuses on words (or sometimes images) and their meanings.
  2. Quantitative research focuses on numbers or counting things and statistical analysis that yields probable meaning.

If you watch this short, easy-to-understand youtube clip, you’ll have all the basics that you need to understand these!   Enjoy!

Critical thinking:  Go to PubMed for this QUANTitative study on spiritual issues in care (https://www.ncbi.nlm.nih.gov/pubmed/28403299) and compare it to this PubMed QUALitative study (https://www.ncbi.nlm.nih.gov/pubmed/27853263) in terms of data, methods, & analysis)

For more information: See earlier posts

Making research accessible to RNs

%d bloggers like this: