Reading through the chapters of Motulsky’s Intuitive Biostatistics, my attention
was caught by the section on meta-analyses in Chapter 43. Given the problem of
reproducibility in scientific research, the meta-analysis seems to be one of
the great tools for addressing the problem of reproducibility; confirming or
refuting prevailing scientific theories and advancing humanity’s body of scientific
knowledge.
Looking at the “Assumptions of Meta-Analysis”, I was intrigued
by the idea that meta-analyses fall into two general categories, each based on
a different assumption. Either
(A)
All subjects are sampled from one large
population; each scientific study is estimating the same effect. Measurement
error comes from random selection of subjects, or
(B)
Each study population is unique, and the
differences in population and random selection of subjects both contribute to
the error.
Sound familiar? To me, these two assumptions seemed derived
from the two philosophies of scientific realism we discussed at the beginning
of class. One being that the truth (the large population) exists somewhere, and
that science’s job is to uncover it, the other being that unobservable truth is
irrelevant, and that utility of knowledge is paramount as is relates to
advancement of medicine or technology. The idea that a population’s true
response to a therapeutic exists is reflected in assumption (A), whereas (B)
describes the anti-realist philosophy that there is no “global” population
response, only individual subset responses as described in each sub-study of
the meta-analysis.
The value of the meta-analysis under the framework of (B),
then, would be to predict the efficacy of a given therapeutic in the next
population of patients, given all those who have been tested before. Motulsky
goes on to described this second model as the more commonly used one underlying
most meta-analyses. The anti-realist philosophy is commonly associated with
being applied and utilitarian, though I’m wondering if there’s a fundamental
application of the anti-realism paradigm. Specifically, what does the idea that
each sub-population in a meta-analysis is inherently disconnected from the
others mean for drawing scientific conclusions from a meta-analysis? Is there a
connection to the scientific philosophies of confirmation and falsification inherent
in the above assumptions? Do the answers to these questions even affect the
conclusions we can draw from meta-analyses, or are they irrelevant exercises in
navel-gazing?
