Category Archives: Data collection

Pilot sTUdies: Look before you leap! (a priori vs. posthoc)

Why does it matter if a study is labeled a “pilot”?

SHORT ANSWER: …Because a pilot is about testing research methods,….not about answering research questions.

If a project has “pilot” in the title, then you as a reader should expect a study that examines whether certain research methods work (methodologic research). Methods include things like timing of data collection, sampling strategies, length of questionnaire, and so on. Pilots suggest what methods will effectively to answer researchers’ questions. Advance prep in methods makes for a smooth research landing.

Small sample = Pilot? A PILOT is related to study goals and design–not sample size. Of course pilots typically have small samples, but a small sample does not a pilot study make. Sometimes journals may tempt a researcher to call their study a pilot because of small samples. Don’t go there. Doing so means after-the-fact, posthoc changes that were Not the original, a priori goals and design.

Practical problems? If researchers label a study a “pilot” after it is completed (post hoc), they raise practical & ethical issues. At a practical level, researchers must create feasibility questions & answers. (See NIH.) The authors should drop data analysis that answers their original research questions.

Ethics? This ethically requires researchers 1) to say they planned something that they didn’t or 2) to take additional action. Additional action may be complete transparency about the change and seeking modification to original human subjects’ committee approvals. An example of one human subjects issue is that you informed your subjects that their data would answer a particular research question, and now you want to use their data to answer something else–methods questions!

Options? You can just learn from your small study and go for a bigger one, including improving methods. Some journals will consider publication of innovative studies even when small.

Look first, then leap: Better to look a priori, before leaping. If you think you might have trouble with your methods, design a pilot. If you made the unpleasant discovery that your methods didn’t work as you hoped, you can 1) disseminate your results anyway or 2) rethink ethical and practical issues.

Who’s with me? The National Institutes of Health agree: https://nccih.nih.gov/grants/whatnccihfunds/pilot_studies . NIH notes that common misuses of “pilots” are determining safety, efficacy of intervention, and effect size.

Who disagrees? McGrath argues that clinical pilots MAY test safety and efficacy, as well as feasibility. (See McGrath, J. M. (2013). Not all studies with small samples are pilot studies, Journal of Perinatal & Neonatal Nursing, 27(4): 281-283. doi: 10.1097/01.JPN.0000437186.01731.bc )

Trial Balloons & Pilot Studies

A pilot study is to research what a trial balloon is to politics

In politics, a trial balloon is communicating a law or policy idea via media to see how the intended audience reacts to it.  A trial balloon does not answer the question, “Would this policy (or law) work?” Instead a trial balloon answers questions like “Which people hate the idea of the policy/law–even if it would work?” or “What problems might enacting it create?” In other words, a trial balloon answers questions that a politician wants to know BEFORE implementing a policy so that the policy or law can be tweaked to be successfully put in place.

meeting2

In research, a pilot study is sort of like a trial balloon. It is “a small-scale test of the methods and procedures” of a planned full-scale study (Porta, Dictionary of Epidemiology, 5th edition, 2008). A pilot study answers questions that we want to know BEFORE doing a larger study, so that we can tweak the study plan and have a successful full-scale research project. A pilot study does NOT answer research questions or hypotheses, such as “Does this intervention work?”  Instead a pilot study answers the question “Are these research procedures workable?”

A pilot study asks & answers:Can I recruit my target population? Can the treatments be delivered per protocol? Are study conditions acceptable to participants?” and so on.   A pilot study should have specific measurable benchmarks for feasibility testing. For example if the pilot is finding out whether subjects will adhere to the study, then adherence might be defined as  “70 percent of participants in each [group] will attend at least 8 of 12 scheduled group sessions.”  Sample size is based on practical criteria such as  budget, participant flow, and the number needed to answer feasibility questions (ie. questions about whether the study is workable).

A pilot study does NOT Test hypotheses (even preliminarily); Use inferential statistics; Assess safety of a treatment; Estimate effect size; Demonstrate safety of an intervention.

A pilot study is not just a small study.

Next blog: Why this matters!!

For more info read the source of all quotes in this blog: Pilot Studies: Common Uses and Misuses @ https://nccih.nih.gov/grants/whatnccihfunds/pilot_studies

Is History “Bunk”? We report. You Decide.

History?  Really?  Fascinating!  Ever thought about all the stories behind your own present life?

Check out this youtube dramatized documentary about Nurse Mary Seacole.  I promise – you’ll enjoy: https://www.youtube.com/watch?v=RIrim4r-LbY   

You can be a part of documenting such stories, including your own.  Can I pique your interest with these examples about historical research?

1. Artifacts: Example = http://acif.org/ The American Collectors of Infant Feeders:

Infant feeder
CREDIT http://acif.org/

The American Collectors of Infant Feeders is a non-profit organization whose primary purpose is to gather and publish information pertaining to the feeding of infants throughout history. The collecting of infant feeders and related items is promoted.

2. Interviews: Example = http://www.oralhistory.org/  Want to do interviews of interesting faculty, students, leaders, “ordinary” nurses?  Check out the Oral History Association    In addition to fostering communication among its members, the OHA encourages standards of excellence in the collection, preservation, dissemination and uses of oral testimony.

scrapbook
CREDIT https://archives.mc.duke.edu/blog/nursing-materials-displa

3. Stories from the “ordinary: Example: http://www.murphsplace.com/mother/main.html My Mother’s War – “Helen T.Burrey was an American nurse who served as a Red Cross Nurse during World War I. She documented her experience in both a journal and a scrapbook which has been treasured by her daughter, Mary Murphy. Ms Murphy has placed many of these items on the Internet for people to access and it provides a first-hand account of that experience. Additionally she has a variety of links to other WWI resources.” (quoted from AAHN Resources online)

Army history
CREDIT http://e-anca.org/

4. Ethnic studies: Example=https://libguides.rowan.edu/blacknurses  Black Nurses in History “This is a ‘bibliography and guide to web resources’ from the UMDNJ and Coriell Research Library. Included are Mamie O. Hail, Mary Eliza Mahoney, Jessie Sleet Scales, Mary Seacole, Mabel Keaton Staupers, Susie King Taylor, Sojourner Truth, Harriet Tubman.” (quoted from AAHN Resources online)

Want more?  

Critical thinking:  Don’t forget to save your own materials.  Your life is history!  What in your life is most interesting?  Have you written it down or dictated it into your iphone voice memo? There is GREAT interest in “ordinary” men and women.  Many times items are tossed because they are “just letters” or “only old records,” or “stuff.” Just Don’t Do It.

 

“Please answer….” (cont.)

What do people HATE about online surveys?   If you want to improve your response rates, check out SurveyMonkey Eric V’s (May Mail2017)  Eliminate survey fatigue: Fix 3 things your respondents hate 

For more info: Check out my earlier post “Please Answer!”

Nightingale: Avante garde in meaningful data

In honor of Nurse Week, I offer this tribute to the avante garde research work of Florence Nightingale in the Crimea that saved lives and set a precedent worth following.

Nightingale was a “passionate statistician” knowing that outcome data are convincing when one wants to change the world.  She did not merely collect the data, but also documented it in a way that revealed its critical meaning for care.

As noted by John H. Lienhard (1998-2002): Nightingale coxcombchart“Once you see Nightingale’s graph, the terrible picture is clear. The Russians were a minor enemy. The real enemies were cholera, typhus, and dysentery. Once the military looked at that eloquent graph, the modern army hospital system was inevitable.  You and I are shown graphs every day. Some are honest; many are misleading….So you and I could use a Florence Nightingale today, as we drown in more undifferentiated data than anyone could’ve imagined during the Crimean War.” (Source: Leinhard, 1998-2002)

As McDonald (2001) writes in the BMJ free, full-text,  Nightingale was “a systemic thinker and a “passionate statistician.”  She insisted on improving care by making policy & care decisions based on “the best available government statistics and expertise, and the collection of new material where the existing stock was inadequate.”(p.68)

Moreover, her display of the data brought its message home through visual clarity!

Thus while Nightingale adhered to some well-accepted, but mistaken, scientific theories of the time (e.g., miasma) her work was superb and scientific in the best sense of the word.   We could all learn from Florence.

CRITICAL THINKING:   What issue in your own practice could be solved by more data?  How could you collect that data?   If you have data already, how can you display it so that it it meaningful to others and “brings the point home”?

FOR MORE INFO:

HAPPY NURSE WEEK TO ALL MY COLLEAGUES.  

MAY YOU GO WHERE THE DATA TAKES YOU!

 

 

Listen up! Don’t interrupt!

Researchers collect two types of data in their studiescounting-sheetword-art

  1. Numbers (called quantitative data)
  2. Words & narratives (called qualitative data)

StorytellerOne source of rich word or narrative (qualitative) data for answering nursing questions is nurses’ stories.  Dr. Pat Benner RN, author of Novice to Expert explains two things we can do to help nurses fully tell their stories so we can learn the most from their practice.

  1. Listen well without interrupting
  2. Help nurses ‘unpack’ their stories 

Check out this excellent 2:59 video of Dr. Benner’s and revolutionize how you learn about nursing from nursing stories:  Preview: The use of Narratives 

Critical thinking:  For a study using narratives in research see  Leboul et al. (2017).  Palliative sedation challenging the professional competency of health care providers and staff: A qualitative focus group and personal written narrative study.  [full text available thru PubMed at https://www.ncbi.nlm.nih.gov/pubmed/28399846].    1) Do you think the authors listened and unpacked information from the focus groups & written narratives; 2)  Do you think there might be a difference in the way people write narratives and verbally tell narratives?   3) How might that difference if any affect the research findings?

For more information:  Check out The Power of Story  by Wang & Geale (2015) at http://www.sciencedirect.com/science/article/pii/S2352013215000496

 

Direct speaking about INdirect outcomes: HCAHPS as a measurement

When you first plan a project, you need to know what OUTCOMES you want to achieve.  You need STRONG outcomes to show your project worked! imagesCALQ0QK9

Outcome measures are tricky & can be categorized into Indirect & Direct measures:

  1. INDIRECT outcome measures are often affected by many factors, not just your innovation
  2. DIRECT outcome measures are specific to what you are trying to accomplish.

For example: If you want to know your patient’s weight, you put them on the scale (direct). weight-scaleYou don’t merely ask them how much they weigh (indirect).

Another example?  If you planned music to reduce pain, you might a) measure how many patients were already using music and their pain scores (& perhaps those not using music and their pain scores), b) begin your music intervention, and c) thmusicen directly measure how many patients started using it after you started your intervention and their pain scores.  These data DIRECTLY target your inpatient outcomes versus looking at INDIRECT HCAHPS answers of discharged patients’ feelings after the fact in response to “During this hospital stay, how often was your pain well controlled?”

Nurses often decide to measure their project outcomes ONLY with indirect HCAHPS scores.  I hope you can see this is not as good as DIRECT measures.

So why use HCAHPS at all?measuring-tape

  • They reflect institutional priorities related to quality and reimbursement
  • Data are already collected for you
  • Data are available for BEFORE and AFTER comparisons of your project outcomes
  • It doesn’t cost you any additional time or money to get the data

Disadvantages of indirect HCAHPS measures?

  • HCAHPS data are indirect measures that are affected by lots of different things, and so they may have little to do with effect of your project.
  • HCAHPS responders often do Not represent all patients because the number responding is so small–sometimes just 1 or 2

Still, I think it’s good to include HCAHPS.  Just don’t limit yourself to that. Include also a DIRECT measure of outcomethat targets the precisely what you hope will be the result of your study.

imagesCALQ0QK9You need STRONG outcomes to convince others that your project works to improve care!

CRITICAL THINKING:  McClelland, L.E., &  Vogus, T.J. (2014) used HCHAPS as an outcome measure in their study, Compassion practices & HCAHPS: Does rewarding and supporting questionworkplace compassion influence patient perceptions?    What were the strengths & weaknesses of using HCHAPS in this study? [hint: check out the discussion section]  What would be a good direct measure that you could add to HCAHPS outcomes to improve the study?

FOR MORE INFORMATION:  Whole books of measurement instruments are available through the library or a librarian can help you search for something that will measure motivation, pain, anxiety, medication compliance, or whatever it is you are looking for!!  You can limit your own literature searches by selecting “instrument” as part of your search, or you can consult with a nurse researcher for more help.