Category Archives: quantitative research

Data collection, nursing research, Outcome measurement, quantitative research, reading research, Reliability & validity, research, research methods

Testing the Test (or an intro to “Does the measurement measure up?”)

Leave a comment

When reading a research article, you may be tempted only to read the Introduction & Background, then go straight to the Discussion, Implications, and Conclusions at the end. You skip all those pesky, procedures, numbers, and p levels in the Methods & Results sections.

Perhaps you are intimidated by all those “research-y” words like content validity, construct validity, test-retest reliability, and Cronbach’s alpha because they just aren’t part of your vocabulary….YET!

WHY should you care about those terms, you ask? Well…let’s start with an example. If your bathroom scale erratically measured your weight each a.m., you probably would toss it and find a more reliable and valid bathroom scale. The quality of the data from that old bathroom scale would be useless in learning how much you weighed. Similarly in research, the researcher wants useful outcome data. And to get that quality data the person must collect it with a measurement instrument that consistently (reliably) measures what it claims to measure (validity). A good research instrument is reliable and valid. So is a good bathroom scale.

Let’s start super-basic: Researchers collect data to answer their research question using an instrument. That test or tool might be a written questionnaire, interview questions, an EKG machine, an observation checklist, or something else. And whatever instrument the researcher uses needs to give them correct data answers.

For example, if I want to collect BP data to find out how a new med is working, I need a BP cuff that collects systolic and diastolic BP without a lot of artifacts or interference. That accuracy in measuring BP only is called instrument validity. Then if I take your BP 3 times in a row, I should get basically the same answer and that consistency is called instrument reliability. I must also use the cuff as intended–correct cuff size and placement–in order to get quality data that reflects the subject’s actual BP.

The same thing is true with questionnaires or other measurement tools. A researcher must use an instrument for the intended purpose and in the correct way. For example, a good stress scale should give me accurate data about a person’s stress level (not their pain, depression, or anxiety)–in other words it should have instrument validity. It should measure stress without a lot of artifacts or interference from other states of mind.

NO instrument is 100% valid–it’s a matter of degree. To the extent that a stress scale measures stress, it is valid. To the extent that it also measures other things besides stress–and it will–it is less valid. The question you should ask is, “How valid is the instrument?” often on a 0 to 1 scale with 1 being unachievable perfection. The same issue and question applies to reliability.

Reliability & validity are interdependent. An instrument that yields inconsistent results under the same circumstances cannot be valid (accurate). Or, an instrument can consistently (reliably) measure the wrong thing–that is, it can measure something other than what the researcher intended to measure. Research instruments need both strong reliability AND validity to be most useful; they need to measure the outcome variable of interest consistently.

Valid for a specific purpose: Researchers must also use measurement instruments as intended. First, instruments are often validated for use with a particular population; they may not be valid for measuring the same variable in other populations. For example, different cultures, genders, professions, and ages may respond differently to the same question. Second, instruments may be valid in predicting certain outcomes (e.g., SAT & ACT have higher validity in predicting NCLEX success than does GPA). As Sullivan (2011) wrote: “Determining validity can be viewed as constructing an evidence-based argument regarding how well a tool measures what it is supposed to do. Evidence can be assembled to support, or not support, a specific use of the assessment tool.”

In summary….

  1. Instrument validity = how accurate the tool is in measuring a particular variable
  2. Instrument reliability = how consistently the tool measures whatever it measures

Fun Practice: In your own words relate the following article excerpt to the concept of validity? “To assess content validity [of the Moral Distress Scale], 10 nurses were asked to provide comments on grammar, use of appropriate words, proper placement of phrases, and appropriate scoring.“ From p.3, Ghafouri et al. (2021). Psychometrics of the moral distress scale in Iranian mental health nurses. BMC Nursing. https://doi.org/10.1186/s12912-021-00674-4

Descriptive research, Qualitative research, quantitative research, research, Research design

Research: What it is and isn’t

Leave a comment

WHAT RESEARCH IS

Research is using the scientific process to ask and answer questions by examining new or existing data for patterns. The data are measurements of variables of interest. The simplest definition of a variable is that it is something that varies, such as height, income, or country of origin. For example, a researcher might be interested in collecting data on triceps skin fold thickness to assess the nutritional status of preschool children. Skin fold thickness will vary.

Research is often categorized in different ways in terms of: data, design, broad aims, and logic.

Qualitative Data
  • Design. Study design is the overall plan for conducting a research study, and there are three basic designs: descriptive, correlational, and experimental.
    1. Descriptive research attempts to answer the question, “What exists?” It tells us what the situation is, but it cannot explain why things are the way they are. e.g., How much money do nurses make?
    2. Correlational research answers the question: “What is the relationship” between variables (e.g., age and attitudes toward work). It cannot explain why those variables are or are not related. e.g., relationship between nurse caring and patient satisfaction
    3. Experimental research tries to answer “Why” question by examining cause and effect connections. e.g., gum chewing after surgery speeds return of bowel function. Gum chewing is a potential cause or “the why”
  • Aims. Studies, too, may be either applied research or basic research. Applied research is when the overall purpose of the research is to uncover knowledge that may be immediately used in practice (e.g., whether a scheduled postpartum quiet time facilitates breastfeeding). In contrast, basic research is when the new knowledge has no immediate application (e.g., identifying receptors on a cell wall).
  • Logic. Study logic may be inductive or deductive. Inductive reasoning is used in qualitative research; it starts with specific bits of information and moves toward generalizations [e.g., This patient’s pain is reduced after listening to music (specific); that means that music listening reduces all patients pain (general)]. Deductive reasoning is typical of quantitative research; it starts with generalizations and moves toward specifics [e.g., If listening to music relaxes people (general), then it may reduce post-operative pain (specific)]. Of course the logical conclusions in each case should be tested with research!

WHAT RESEARCH IS NOT:

Research as a scientific process is not going to the library or searching online to find information. It is also different from processes of applying research and non-research evidence to practice (called Evidence-Based Practice or EBP). And it is not the same as Quality Improvement (QI). See Two Roads Diverged for a flowchart to help differentiate research, QI and EBP.

experiment, nonrandom trials, quantitative research, RCT, research, Research design, research methods

iS IT 2? OR 3?

Leave a comment

Credible sources often disagree on technicalities. Sometimes this includes classification of research design. Some argue that there are only 2 categories of research design:

  1. True experiments. True experiments have 3 elements: 1) randomization to groups, 2) a control group and an 3) intervention; and
  2. Non-experiments. Non-experiments may have 1 to none of those 3 elements.
Within-subject Control Group

Fundamentally, I agree with the above. But what about designs that include an intervention and a control group, but Not randomization?

Those may be called quasi-experiments; the most often performed quasi-experiment is pre/post testing of a single group. The control group are subjects at baseline and the experimental group are the same subjects after they receive a treatment or intervention. That means the control group is a within-subjects control group (as opposed to between-group control). Quasi-experiments can be used to answer cause-and-effect hypothesis when an experiment may not be feasible or ethical.

One might even argue that a strength of pre/post, quasi-experiments is that we do Not have to Assume that control and experimental groups are equivalent–an assumption we would make about the subjects randomized (randomly assigned) to a control or experimental group. Instead the control and experimental  are exactly equivalent because they are the same persons (barring maturation of subjects and similar threats to validity that are also true of experiments).

I think using the term quasi-experiments makes it clear that persons in the study receive an intervention. Adding “pre/post” means that the

This image has an empty alt attribute; its file name is intervention.jpg
Baseline ->Intervention->Post

researcher is using a single group as their own controls. I prefer to use the term non-experimental to mean a) descriptive studies (ones that just describe the situation) and b) correlation studies (ones without an intervention that look for whether one factor is related to another).

What do you think? 2? or 3?

Data, quantitative research, reading research, research

Goldilocks and the 3 Levels of Data

2 Comments

Actually when it comes to quantitative data, there are 4 levels, but who’s counting? (Besides Goldilocks.)

  1. Nominal  (categorical) data are names or categories: (gender, religious affiliation, days of the week, yes or no, and so on)
  2. Ordinal data are like the pain scale.  Each number is higher (or lower) than the next but the distances between numbers are not equal.  In others words 4 is not necessarily twice as much as 2; and 5 is not half of 10.
  3. Interval data are like degrees on a thermometer.  Equal distance between them, but no actual “0”.  0 degrees is just really, really cold.
  4. Ratio data are those  with real 0 and equal intervals (e.g., weight, annual salary, mg.)

(Of course if you want to collect QUALitative word data, that’s closest to categorical/nominal, but you don’t count ANYTHING.  More on that another time.)

CRITICAL THINKING:   Where are the levels in Goldilocks and the 3 levels of data at this link:  https://son.rochester.edu/research/research-fables/goldilocks.html ?? Would you measure soup, bed, chairs, bears, or other things differently?  Why was the baby bear screaming in fright?

Methods, Qualitative research, quantitative research

Words vs. Numbers: What does it all mean?

Leave a comment

There are several ways to classify types of research.   One way is qualitative versus quantitative–in other words, WORD  vs. NUMBER data, methods, & analysis.

  1. Qualitative research focuses on words (or sometimes images) and their meanings.
  2. Quantitative research focuses on numbers or counting things and statistical analysis that yields probable meaning.

If you watch this short, easy-to-understand youtube clip, you’ll have all the basics that you need to understand these!   Enjoy!

Critical thinking:  Go to PubMed for this QUANTitative study on spiritual issues in care (https://www.ncbi.nlm.nih.gov/pubmed/28403299) and compare it to this PubMed QUALitative study (https://www.ncbi.nlm.nih.gov/pubmed/27853263) in terms of data, methods, & analysis)

For more information: See earlier posts

Data, Data collection, Disseminating findings, Evidence based nursing, evidence based practice, Florence Nightingale, History of research, Outcome measurement, Professional Nursing, quantitative research, research, Uncategorized

Nightingale: Avant garde in meaningful data

Leave a comment

In honor of Nurse Week, I offer this tribute to the avant garde research work of Florence Nightingale in the Crimea that saved lives and set a precedent worth following.

Nightingale was a “passionate statistician” knowing that outcome data are convincing when one wants to change the world.  She did not merely collect the data, but also documented it in a way that revealed its critical meaning for care.

As noted by John H. Lienhard (1998-2002): Nightingale coxcombchart“Once you see Nightingale’s graph, the terrible picture is clear. The Russians were a minor enemy. The real enemies were cholera, typhus, and dysentery. Once the military looked at that eloquent graph, the modern army hospital system was inevitable.  You and I are shown graphs every day. Some are honest; many are misleading….So you and I could use a Florence Nightingale today, as we drown in more undifferentiated data than anyone could’ve imagined during the Crimean War.” (Source: Leinhard, 1998-2002)

As McDonald (2001) writes in the BMJ free, full-text,  Nightingale was “a systemic thinker and a “passionate statistician.”  She insisted on improving care by making policy & care decisions based on “the best available government statistics and expertise, and the collection of new material where the existing stock was inadequate.”(p.68)

Moreover, her display of the data brought its message home through visual clarity!

Thus while Nightingale adhered to some well-accepted, but mistaken, scientific theories of the time (e.g., miasma) her work was superb and scientific in the best sense of the word.   We could all learn from Florence.

CRITICAL THINKING:   What issue in your own practice could be solved by more data?  How could you collect that data?   If you have data already, how can you display it so that it it meaningful to others and “brings the point home”?

FOR MORE INFO:

HAPPY NURSE WEEK TO ALL MY COLLEAGUES.  

MAY YOU GO WHERE THE DATA TAKES YOU!

Evidence based nursing, evidence based practice, experiment, nonrandom trials, quantitative research, quasi-experimental, RCT, reading research, Research design

Quasi- wha??

4 Comments

Two basic kinds of research design exist:  

  1. Experimental design in which
    • the researcher manipulates some variable,randomized
    • the participants are randomly assigned to groups, &
    • one group is a control group that gets a placebo or some inert treatment so that outcomes in that group can be compared to the group(s) that did get the treatment.
  2. Non-experimental design in which the researcher doesn’t manipulate anything, but just observes & records what is going on.   Some of these are descriptive, correlational, case, or cohort study designs for example.

One particularly interesting “experimental” design is one in which 1 or 2 of the experimental design ideal requirements as listed above are missing.  These are called quasi-experimental designs.

thinking3In a quasi experimental design

  • The researcher manipulates some variable, but….
  • Either the participants are NOT randomly assigned to groups
  • &/OR there is no control group.

A quasi-experimental design is not as strong as a true experiment in showing that the manipulated variable X causes changes in the outcome variable Y.  For example, a true experimental study with manipulation, randomization, and a control group would create much stronger evidence that hospital therapy dogs really reduced patient pain and anxiety.  We would not be as confident in the results of a quasi-experimental design examining the exact same thing.  In the next blog, we’ll examine why.

For more info:  Check out earlier blog:    “What is an RCT anyway?” at https://discoveringyourinnerscientist.com/2015/01/23/whats-a-randomized-controlled-trial/Idea2

Critical thinking:  Go to PubMed & use search terms “experiment AND nurse” (without the quotation marks).  Open an interesting abstract and look for the 3 elements of a classic experimental design. Now look for “quasi experiment AND nurse” (without the quotation marks.)  See what element is missing!

Data, Data collection, Qualitative research, quantitative research, reading research, research methods, Uncategorized

DATA COLLECTION SECTION! (Methods in the Madness)

Leave a comment

Key point! The data collection section of a research article includes: who collects what data when, where & how.

In previous blogs we’ve looked at title, introduction, and other elements of methods section (design, sample, & setting). In this one let’s take a look at data collection.

Data are a collection of measurements. For example, student scores on a classroom test might be 97, 90, 88, 85, & so on. Each single score is a datum; collectively they are data.

What data are collected is answered in this section. The data (or measurements) can be counting-hashmarksnumbers OR words. For example, numbers data might include patient ratings of their pain on a 0-10 scale. An example of word data would asking participants to describe something in words without counting the words or anything else.  For example, word data might include patient descriptions pain in words, like word-art“stabbing,”  “achy,” and so on.  Sometimes a researcher collects both number and word data in the same study to give a more complete description.  You can see how knowing the patient’s pain rating and hearing a description would give you a much clearer picture of pain.

  • Studies reporting data in numbers are called quantitative studies
  • Studies reporting data in words/descriptions are called qualitative studies
  • Studies reporting number & word data are called mixed methods studies

How the data are collected includes what instrument or tool was used to gather data (e.g., observation, biophysical measure, or self-report) and how consistently & accurately that tool measures what it is supposed to measure (e.g., reliability & validity). Also included is who collected the data and the procedures that they followed—how did they obtain consent, interaction with subjects, timing of data collection and so on.

Now you know!

Critical thinking question: Did these authors use qualitative or quantitative data collection methods?  Coelho, A., Parola, V., Escobar-Bravo, M., & Apostolo, J. (2016). Comfort experience in palliative care, BMD Palliative care, 15(71). doi: 10.1186/s12904-016-0145-0.  Explain your answer.

experiment, Literature review, quantitative research, RCT, reading research

What’s an RCT anyway?

3 Comments
  • Question: What is a randomized controlled trial (RCT)? And why should I care?
  • Answer: An RCT is one of the strongest types of studies in showing that a drug or a treatment actually improves a symptom or disease. If I have strep throat, I want to know what antibiotic works best in killing the bacteria, & RCTs are one of the best ways to find that answer.

In the simplest kind of RCT, subjects are randomly assigned to 2 groups.  One group gets the treatment in which we are interested, & it is called the experimental group.   The other group gets either no treatment or standard treatment, & it is called the control group.  

Here’s an example from a study to determine whether chewing gum prevents postoperative ileus after laparotomy for benign gynecologic surgery:  A total of 109 patients were randomly assigned to receive chewing gum (n=51) or routine postoperative care (n=58).  Fewer participants assigned to receive chewing gum … experienced postoperative nausea (16 [31.4%] versus 29 [50.0%]; P=0.049) and postoperative ileus (0 vs. 5 [8.6%]; P=0.032).* There were no differences in the need for postoperative antiemetics, episodes of postoperative vomiting, readmissions, repeat surgeries, time to first hunger, time to toleration of clear liquids, time to regular diet, time to first flatus, or time to discharge. Conclusion?  Postop gum chewing is safe & lowers the incidence of nausea and ileus! (Jernigan, Chen, & Sewell, 2014. Retrieve from PubMed abstract)

Do you see the elements of an RCT in above?

Let’s break it down.

  • Randomized means that 109 subjects were randomly divided into 2 or more groups. In above case, 51 subjects ended up in a gum chewing group & 58 were assigned to a routine care, no gum group.  Randomization increases the chance that the groups will be similar in characteristics such as age, gender, etc.   This allows us to assume that different outcomes between groups are caused by gum-chewing, not by differences in group characteristics.
  • Controlled means that 1 of the groups is used as a control group. It is a comparison group, like the no-gum , standard care group above
  • Trial means that it was a study. The researchers were testing (trying) an intervention and measuring the outcomes to see if it worked.  In this case the intervention was gum chewing and the measure outcomes were nausea and ileus.

Why should you care about RCTs?  Because RCTs are strong evidence that an intervention works (or doesn’t) for your patients

Critical Thinking Exercise:   Go to http://www.ncbi.nlm.nih.gov/pubmed   In the blank box at the very top enter a few key words about the problem in which you are interested + RCT.  For example:  music pain + RCT.   Then read 1 or more of the abstracts looking for random assignment (randomized), control group, and whether it was a study (trial).   You’re on your way!    -Dr.H

*Note: You may remember from other blogs that p<.05 means the difference between groups is probably cause by the intervention—in this case gum chewing.