mahiwaga

Aside from the overhyped and clickbaity headlines, one of the basic problems of popular medical journalism is that they generally elevate meta-analyses over randomized controlled trials.¹

It isn’t that meta-analyses aren’t as useful, but they can certainly be more unreliable due to having to summarize possibly quite disparate studies with widely varying sample populations with significant differences in experimental design and in expected end-points and measurements. Sure, you can account for a lot of this with statistical analysis, but this tends to introduce additional assumptions that may or may not be warranted, and is another step that’s susceptible to bias.²

• Meta-analysis: Its strengths and limitations • 2008 Jun • E Walker, A Hernandez, and M Kattan • Cleveland Clinic Journal of Medicine
• Meta-analysis: pitfalls and hits • 2013 • T Greco, A Zangrillo, G Biondi-Zoccai, and G Landoni • Heart Lung Vessel • PubMed

• Including all relevant material–good, bad, and indifferent–in meta-analysis admits the subjective judgments that meta-analysis was designed to avoid. Several problems arise in meta-analysis: regressions are often non-linear; effects are often multivariate rather than univariate; coverage can be restricted; bad studies may be included; the data summarised may not be homogeneous; grouping different causal factors may lead to meaningless estimates of effects; and the theory-directed approach may obscure discrepancies. Meta-analysis may not be the one best method for studying the diversity of fields for which it has been used.

• A meta-analysis is a safer starting point than a single study – but it won’t necessarily be more reliable.

  A meta-analysis is usually part of a systematic review. It’s a heavy-duty effort, and it’s often described as the ultimate study, outweighing all others. The last word. A single study becomes a puny thing, to be ignored even.

  But while combined results can carry a lot of weight, there are 3 main problems with the idea that a meta-analysis always trumps a single study.

  Firstly, a systematic review and meta-analysis isn’t a formal experimental study. It’s a non-experimental or descriptive study. There are subjective judgments every step of the way, giving small teams of like-minded people plenty of room to steer in a desired direction if they want to. A bad or patchy meta-analysis might not come to as reliable conclusions as a well-conducted, adequately powered single study.

  Secondly, it’s not at all unusual for a meta-analysis to be heavily dominated by a single study. A study by Paul Glasziou and colleagues in 2010 found that even when there were several trials, the most precise one carried on average half the weight of the results – and around 80% of the time the conclusion of the meta-analysis was pretty much the same as that single study. Understanding and discussing that dominant study is critical.

  The third problem is so big, it gets the next place on this listicle: a single new study can overturn the results of a meta-analysis.…

  A meta-analysis is a snapshot in time – it can even be out-of-date the day it’s published.

Pope Francis understands that the root cause of terrorism are the inequalities caused by the history of imperialism and colonialism and perpetuated by global capitalism.