Global health data is a lot like sausage: the more you know about what it is made from and how it gets processed, the less appetizing it becomes.
I’ve just returned from a week in Seattle where I was attending the first Global Health Metrics and Evaluation Conference hosted by the Institute for Health Metrics and Evaluation (IHME) at the University of Washington, the Lancet, the Harvard School of Public Health, The University of Queensland School of Public Health and the London School of Hygiene and Tropical Medicine. Over 600 people, mostly data junkies like myself, congregated to discuss the collection, aggregation, dissemination and analysis of global health data. For a conference on a topic which might seem incredibly dry and boring to most people, it was amazing how lively were the debates and how much controversy the conference generated. In short, it was a lot fun.
Over the past few years, a revolution in global health data has been underway. Chris Murray, armed with tens of millions of dollars, has established IHME and has built the powerhouse in global health metrics. For years, Chris’ army has been mining thousands of new and existing data sources to generate alternative estimates using new methodologies for many of the most important and commonly used measures in global health, including new estimates of child mortality, maternal mortality, and health system resources. Many of the big controversies in global health policy over the past year have been due to changes in our beliefs over some aspect of global health. IHME correctly deserves much of the credit for making us think differently about where global health data comes from and what it is made from.
Overall, I am supportive of these efforts: all estimates are prone to biases and therefore allowing competing views may allow us to understand where such biases exist. I think IHME has made the world a much better place in which to do research. However, no organization is value judgement free so whether it is data from IHME or the WHO or some other agency it will be biased in some way. The choice of weights, the use of different inclusion and exclusion criteria, and the selection of different methods to impute missing data all introduce some biases.
This global health data revolution has raised many important questions for researchers which have been bubbling up over the years and many of these issues finally boiled over during the discussion last week. For example, does the use of new and increasingly complex statistical methods to generate estimates really get us closer to the true value? What is the responsibility of those generating these estimates to discuss how these methods differ from previous estimates? To what extent are efforts to generate new internationally comparable estimates undermining local and national efforts to generate capacity and produce country-owned estimates? Who owns global health data and what rig? If metrics are so imperfect — for example maternal mortality — should any weight be placed on it at all?
But for all the debate about the details about global health data what is clear to me is that global health metrics matter to the rest of the world — but not always in the same way that geeks like me think about data. We worry a lot about how data is generated, but there is not always a strong correlation between the strength of the methodological foundation upon which a health metrics is built and the power that this metric can have. Just look at how estimates of HIV prevalence, maternal mortality, and my new least-favorite-metric malaria incidence have generated significant attention from politicians and global health policy makers despite the incredible challenges to their measurement. I don’t think we have done nearly enough thinking about how different audiences process the data that is generated and how it is used (or abused) for policy purposes. Better and more accurate data needs to be the goal, but also understanding how data influences policy and action is equally important.
My big take home from the conference is that global health datasets should come with a warning: the use of statistical methods might be hazardous to your understanding of population health.Share on Facebook