Headlines are blaring: “New study shows that lockdowns had minimal effect on COVID-19 mortality.”

The January 2022 systematic review and meta-analysis that underlies that news is Herby, Jonung, & Hanke’s “A Literature Review and Meta Analysis of the Effects of Lockdowns on COVID 19 Mortality” in Applied Economics .

Scientists label systematic reviews and meta-analyses as the strongest type of scientific evidence (pyramid of evidence). Of course the strength of the systematic review/meta-analysis depends on whether it is well or poorly done, so never put your research-critique brain in neutral. This one seems well done.

In systematic reviews, researchers follow a methodical, focused process that describes their selection and analysis of all studies on a topic. Meta-analyses treat all the data from those selected studies as a single study. Researchers will specify their process and parameters for selecting studies, and they typically publish a table of evidence that summarizes key information about each study. Herby et al. did so. (Note: systematic reviews should not be confused with integrative reviews in which authors are less systematic and are giving background info.)

For example, from Herby et al’s study cited above: “This study employed a systematic search and screening procedure in which 18,590 studies are identified… After three levels of screening, 34 studies ultimately qualified. Of those 34 eligible studies, 24 qualified for inclusion in the meta-analysis. They were separated into three groups: lockdown stringency index studies, shelter-in-place-order (SIPO) studies, and specific [non-pharmaceutical intervention] NPI studies. An analysis of each of these three groups support the conclusion that lockdowns have had little to no effect on COVID-19 mortality.”

See the full publication below. And rather than reading it beginning to end, first 1) read the abstract; 2) identify parameters used to select the 34 eligible studies and 24 meta-analysis studies, 3) scan the table of evidence, and 4) read the discussion beginning page 40. Then read the complete article, and cut yourself some slack—-just try understand what you can depending on your research expertise.

What do you think? Are the studies that support their conclusions strong? What are the SCIENTIFIC objections to their conclusions? What do they identify as policy implications, and do you agree or disagree?

[NOTE THAT THIS ARTICLE LINK MAY BE GOOD FOR ONLY 30 DAYS, but a librarian can help you get it after that.] Happy evidence hunting.