Data collection, Disseminating findings, Ethics, pilot study, Research design, research methods

Pilot sTUdies: Look before you leap! (a priori vs. posthoc)

Leave a comment

Why does it matter if a study is labeled a “pilot”?

SHORT ANSWER: …Because a pilot is about testing research methods,….not about answering research questions.

If a project has “pilot” in the title, then you as a reader should expect a study that examines whether certain research methods work (methodologic research). Methods include things like timing of data collection, sampling strategies, length of questionnaire, and so on. Pilots suggest what methods will effectively to answer researchers’ questions. Advance prep in methods makes for a smooth research landing.

Small sample = Pilot? A PILOT is related to study goals and design–not sample size. Of course pilots typically have small samples, but a small sample does not a pilot study make. Sometimes journals may tempt a researcher to call their study a pilot because of small samples. Don’t go there. Doing so means after-the-fact, posthoc changes that were Not the original, a priori goals and design.

Practical problems? If researchers label a study a “pilot” after it is completed (post hoc), they raise practical & ethical issues. At a practical level, researchers must create feasibility questions & answers. (See NIH.) The authors should drop data analysis that answers their original research questions.

Ethics? This ethically requires researchers 1) to say they planned something that they didn’t or 2) to take additional action. Additional action may be complete transparency about the change and seeking modification to original human subjects’ committee approvals. An example of one human subjects issue is that you informed your subjects that their data would answer a particular research question, and now you want to use their data to answer something else–methods questions!

Options? You can just learn from your small study and go for a bigger one, including improving methods. Some journals will consider publication of innovative studies even when small.

Look first, then leap: Better to look a priori, before leaping. If you think you might have trouble with your methods, design a pilot. If you made the unpleasant discovery that your methods didn’t work as you hoped, you can 1) disseminate your results anyway or 2) rethink ethical and practical issues.

Who’s with me? The National Institutes of Health agree: https://nccih.nih.gov/grants/whatnccihfunds/pilot_studies . NIH notes that common misuses of “pilots” are determining safety, efficacy of intervention, and effect size.

Who disagrees? McGrath argues that clinical pilots MAY test safety and efficacy, as well as feasibility. (See McGrath, J. M. (2013). Not all studies with small samples are pilot studies, Journal of Perinatal & Neonatal Nursing, 27(4): 281-283. doi: 10.1097/01.JPN.0000437186.01731.bc )

Uncategorized

The Difference Between “Historic” and “Historical”

1 Comment

Doing history research?

Grammar Party

Photo of Big Ben at night
London’s Big Ben is a historic clock.

This post will teach you the difference between historic and historical. These two words have similar meanings and get confused a lot, so don’t feel bad that you haven’t memorized their definitions.

Historic describes an important and momentous event, person, place, or thing in history.

  • The Revolutionary War was a historic event in the United States.
  • Marie Curie is a historic figure in scientific history.
  • Big Ben is a historic clock.

Historical describes anything that belonged to an earlier time period and relates to history.

  • Grandma found historical dinner plates at the yard sale.
  • The farmhouse from the 1800s is historical.
  • We looked at a historical map of our town to learn its original design.

To remember the difference, think about historic as being something big (meaningful to many) and historical as something small (meaningful to a few).

For instance, the…

View original post 121 more words

Data collection, Evidence based nursing, evidence based practice, Methodologic research, Methods, research, Research design, research methods

Trial Balloons & Pilot Studies

Leave a comment

A pilot study is to research what a trial balloon is to politics. 

In politics, a trial balloon is communicating a law or policy idea via media to see how the intended audience reacts to it.  A trial balloon does not answer the question, “Would this policy (or law) work?” Instead a trial balloon answers questions like “Which people hate the idea of the policy/law–even if it would work?” or “What problems might enacting it create?” In other words, a trial balloon answers questions that a politician wants to know BEFORE implementing a policy so that the policy or law can be tweaked to be successfully put in place.

meeting2

In research, a pilot study is sort of like a trial balloon. It is “a small-scale test of the methods and procedures” of a planned full-scale study (Porta, Dictionary of Epidemiology, 5th edition, 2008). A pilot study answers questions that we want to know BEFORE doing a larger study, so that we can tweak the study plan and have a successful full-scale research project. A pilot study does NOT answer research questions or hypotheses, such as “Does this intervention work?”  Instead a pilot study answers the question “Are these research procedures workable?”

A pilot study asks & answers: “Can I recruit my target population? Can the treatments be delivered per protocol? Are study conditions acceptable to participants?” and so on.   A pilot study should have specific measurable benchmarks for feasibility testing. For example if the pilot is finding out whether subjects will adhere to the study, then adherence might be defined as  “70 percent of participants in each [group] will attend at least 8 of 12 scheduled group sessions.”  Sample size is based on practical criteria such as  budget, participant flow, and the number needed to answer feasibility questions (ie. questions about whether the study is workable).

A pilot study does NOT:  Test hypotheses (even preliminarily); Use inferential statistics; Assess safety of a treatment; Estimate effect size; Demonstrate safety of an intervention.

A pilot study is not just a small study.

Next blog: Why this matters!!

For more info read the source of all quotes in this blog: Pilot Studies: Common Uses and Misuses @ https://nccih.nih.gov/grants/whatnccihfunds/pilot_studies

reading research, Research student, Writing

Easy to read. Hard to write.

Leave a comment

Musings:  For me the most difficult to write sections of a research report are the Intro/Background and Discussion.  And yet,  those are apparently the easiest to read for many.   My students at least tend to read only those sections and skip the rest.

Why? For the author, Intro/Background and Discussion require hard, critical thinking about what is already known about the topic (Intro/Background) and then what one’s findings mean in light of that (Discussion).  For research consumers, the language used in these sections is more familiar, ordinary sounding words.  On the other hand, writing the technical nature of other sections (Methods, Instruments, Results) is pretty straightforward with scientifically standardized vocabulary and structure.  But, for readers, those same sections contain potentially unfamiliar research terminology that is not part of everyday conversation– i.e., scientific vocabulary.  Quantitative studies often create more reader difficulty.

My solution for myself as a writer? To spend time making sure that the first sentence of every paragraph in Intro/Background and Discussion makes a step-by-step argument supported by the rest of the paragraph. Follow standardized structure for the rest.  Keep  language  precise  yet  simple  as  possible.

Solution for research readers? Read the whole article understanding what you can and keep a research glossary handy (e.g., https://sites.google.com/site/nursingresearchaid/week-1. Even if practice doesn’t make you perfect, it works in learning a new language–whether it is  a ‘foreign’ language or a scientific one.

Critical Thinking:  Test out your reading skills with this article https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6503597/   .  Do the authors make systematic arguments in Intro/Background & Discussion? What makes this article hard or easy to read?

Happy Summer! -Dr.H

Conferences, Disseminating findings, Ethics

Part 2: It’s a jungle out there! Flaky academic conferences

1 Comment

Flaky conferences can taken advantage of your time, money and energy.  My own publications in bona fide journals have triggered an onslaught of emails from probably predatory conferences–World Congresses of this and that (global health, nursing, education, etc.).  The cartoon below totally resonates!  Thanks PHD Comics.

http://phdcomics.com/comics.php?f=1704

 

Disseminating findings, Ethics, Publication, Writing

“It’s a jungle out there:” Predatory Journals

Leave a comment

Cool. You completed your project and now want to publish it.

Beware! Predatory journals are ready to snap up your work!  It helps them, but not you.

What is a predatory journal?  One that can eat you and your paper alive.

  • tiger junglePredatory journals don’t meet quality peer-review standards.
  • The predator may post your manuscript online, which then vanishes from access.
  • If you are writing an article and cite a predator-published manuscript, you undercut your own credibility.
  • If you’re counting on the article for credit towards tenure, your personnel reviewers may toss a predator-published article aside. 0 credit for you & questions about your own credibility. [For more on this in nursing see: Owens, J.K. & Chinn, P. (2018, January 20). “Reference letters & the specter of publications in predatory journals, Nurse Author & Editor, 28(1), 2. Retrieved from http://naepub.com/peer-review/2018-28-1-2/]

Many predators are Open Access Journals. What are Open Access Journals?  Ones “that use a funding model that does not charge readers or their institutions for access” (https://doaj.org/faq#definition).

Open Access Journals may be legitimate OR predatory.

How can you identify predatory journals? While perhaps not Mighty Mouse—yep, I’m showing my age—help from the Directory of Open Access Journals (DOAJ) is on the way. DOAJ provides a searchable list of LEGITIMATE open access journals (click here).  A quick search for “nursing” yielded 7.

How does DOAJ define quality  of journals? Quality open access journals “must exercise peer-review with an editor and an editorial board or editorial review (particularly in the Humanities) carried out by at least two editors” (https://doaj.org/faq#definition).

Is there a list of predatory open access journals? YES. To see one that is updated, click here. Also, you can help! If you find an open access journal that claims to have the DOAJ quality seal, but isn’t on the DOAJ legitimate journal list, DOAJ wants to hear from you!

For more on DOAJ, see https://doaj.org/: “DOAJ is a community-curated online directory that indexes and provides access to high quality, open access, peer-reviewed journals. DOAJ is independent.”

predator raptorStay safe. As Randy Newman sings, “It’s a jungle out there.”

-Dr.H

Uncategorized

Where did those comments go??

Leave a comment

Hi All out there in the nursing Research/EBP universe.

Notice that TO SEE COMMENTS on the blog post, there is a comments link under blogpost title.  In the most recent post, I added info in a comment about Scholar Scams.

Couldn’t resist sharing our beautiful flowers here in SoCal!

IMG_5965

 

Making research accessible to RNs

%d bloggers like this: