Last fall, I blogged about a seemingly alarming new research findings published in the Lancet by Dhingra and co-authors that claimed that the estimates of mortality from malaria in India were being underreported by a factor of 10. I like many others, asked how could the WHO’s estimates be so wrong?
Five months later, it seems that this controversy is still buzzing away (mosquito pun intended). The Lancet recently published a series of correspondences have seriously questioned the validity of these newly published estimates and have left me scratching my head once again.
I was particularly troubled by some of the arguments made by Naman Shah and co-authors (you may know Naman from his excellent malaria blog topnamen). Apparently a small validation exercise had been carried out in India to validate the use of the verbal autopsy methods used in this study but the validation exercise found large discrepancies between what was being counted as a malaria death using verbal autopsy methods and what was being clinically validated in hospitals. I saw the same data presented last week at the GHME conference in Seattle which makes me believe that this is a really big faux pas in the health metrics world.
So while I am no expert on these things, I am surprised that if one is using a new method and that there is data available from a validation exercise that shows large discrepancies that these limitations would not become apparent during the peer review process – in particular given how much controversy there is about the use of verbal autopsy methods in general and in particular regarding the use of them in other geographic contexts for malaria.
It is great to have new methods to track and monitor things that have been challenging to measure in the past, but as we all know, newer is not always better, and I believe that new methods need to demonstrate their superiority before we take them at face value. This might be one case where we might need a bit more convincing.Share on Facebook
Global health data is a lot like sausage: the more you know about what it is made from and how it gets processed, the less appetizing it becomes.
I’ve just returned from a week in Seattle where I was attending the first Global Health Metrics and Evaluation Conference hosted by the Institute for Health Metrics and Evaluation (IHME) at the University of Washington, the Lancet, the Harvard School of Public Health, The University of Queensland School of Public Health and the London School of Hygiene and Tropical Medicine. Over 600 people, mostly data junkies like myself, congregated to discuss the collection, aggregation, dissemination and analysis of global health data. For a conference on a topic which might seem incredibly dry and boring to most people, it was amazing how lively were the debates and how much controversy the conference generated. In short, it was a lot fun.
Over the past few years, a revolution in global health data has been underway. Chris Murray, armed with tens of millions of dollars, has established IHME and has built the powerhouse in global health metrics. For years, Chris’ army has been mining thousands of new and existing data sources to generate alternative estimates using new methodologies for many of the most important and commonly used measures in global health, including new estimates of child mortality, maternal mortality, and health system resources. Many of the big controversies in global health policy over the past year have been due to changes in our beliefs over some aspect of global health. IHME correctly deserves much of the credit for making us think differently about where global health data comes from and what it is made from.
Overall, I am supportive of these efforts: all estimates are prone to biases and therefore allowing competing views may allow us to understand where such biases exist. I think IHME has made the world a much better place in which to do research. However, no organization is value judgement free so whether it is data from IHME or the WHO or some other agency it will be biased in some way. The choice of weights, the use of different inclusion and exclusion criteria, and the selection of different methods to impute missing data all introduce some biases.
This global health data revolution has raised many important questions for researchers which have been bubbling up over the years and many of these issues finally boiled over during the discussion last week. For example, does the use of new and increasingly complex statistical methods to generate estimates really get us closer to the true value? What is the responsibility of those generating these estimates to discuss how these methods differ from previous estimates? To what extent are efforts to generate new internationally comparable estimates undermining local and national efforts to generate capacity and produce country-owned estimates? Who owns global health data and what rig? If metrics are so imperfect — for example maternal mortality — should any weight be placed on it at all?
But for all the debate about the details about global health data what is clear to me is that global health metrics matter to the rest of the world — but not always in the same way that geeks like me think about data. We worry a lot about how data is generated, but there is not always a strong correlation between the strength of the methodological foundation upon which a health metrics is built and the power that this metric can have. Just look at how estimates of HIV prevalence, maternal mortality, and my new least-favorite-metric malaria incidence have generated significant attention from politicians and global health policy makers despite the incredible challenges to their measurement. I don’t think we have done nearly enough thinking about how different audiences process the data that is generated and how it is used (or abused) for policy purposes. Better and more accurate data needs to be the goal, but also understanding how data influences policy and action is equally important.
My big take home from the conference is that global health datasets should come with a warning: the use of statistical methods might be hazardous to your understanding of population health.Share on Facebook
As I expected, there is a lot of discussion this year about the importance of Non-Communicable Diseases and how they should be address by the Global Health apparatus. This fall, the United Nations General Assembly will be hosting a “high level” meeting on them this fall in New York. CSIS is running a contest for you to have your say. Do you have anything interesting to say on this? If so, you can enter their blog contest to voice your opinion. The deadline is March 15, 2011.
If you are looking for some background material, I recommend you have a look at these two excellent reports by J. Stephen Morrison, Devi Sridhar, and Peter Piot as well as this report from Rachel Nugent and Andrea Feigel at the Center for Global Development.Share on Facebook
There is a new and exciting toy coming our way this week from Seattle…and I don’t mean the iPad2 (although I am looking forward to that too as I will soon be inheriting my husband’s *old* iPad). What I am talking about is a new web portal called the Global Health Data Exchange (GHDx) hosted by the Institute for Health Metrics and Evaluation.
Getting good quality publicly accessible data in global health is challenging – I know. The big exception of course are the excellent Demographic and Health Surveys, which I have used a lot over the years, but there are tons of great datasets out there, including hundreds of great household surveys and censuses that exist in statistical offices around the world, it has just been very difficult to get access to them. It looks like this is about to change.
On Monday, during the Global Health Metrics and Evaluation conference in Seattle the IHME will be launching this new web portal where they have collected and made publicly available lots of new and hopefully very useful data. They claim the site will have upwards of 1000 datasets by the time it launches on Monday and hopes to grow it substantially in the coming months.
I managed to set up an account today and started to play around and was quite impressed with the offerings. There are 889 household surveys listed (the data was not available for download yet) from the “Afghanistan – Kabul Global Youth Tobacco Survey 2004″ to the “Zimbabwe Young Adult Reproductive Health Survey 2001-2002″.
I’ll be attending the GHME conference in Seattle next week where I will be presenting some work with co-authors Nandini Oomman (CGD), Christina Droggitis (CGD), and Jing Dong (NYU) on maternal mortality declines. I hope to be blogging and tweeting away (follow me, the conference, or the #GHME2011).Share on Facebook
I am giving a lecture on the supply of health services this evening in my Health Economics class. It was therefore fitting to receive an email this morning notifying me of the launch of a new website from the World Bank where they have centralized information and resources related to health human resources. You can find this website at www.worldbank.org/hrh.
According to the World Health Report 2006, there are approximately 60 million health workers globally, but despite this there are important shortages and imbalances in the workforce everywhere. As I will try to argue in class today, without human resources there would be no health services, yet it is sometimes easy to forget that health systems are made up of people. Health care is a service industry but despite this human resources are frequently neglected by planners and researchers. It is therefore great to see another site where information on this topic is centralized and made available for others to learnShare on Facebook