Health Analytics for Robust Policy – this website – is a collection of essays, lecture notes, definitions, references, and reflections about science and policy. It serves as a platform to share our team’s ideas, philosophy, methodologies, and practical advice about how to conduct science in an environment where the end-goal of our work is to improve health through better policy.
Science and policy are important activities that involve analysis and inference, but while science is about knowing, policy is about doing. Policy should use the best science available, but even when evidence is weak, policy decisions will get made and implemented. Doing nothing is a policy that requires the same degree of scrutiny as doing something. Unlike science, the question is not whether the evidence is strong enough, but what course of action is best supported by the evidence. The core difference between inference for science and policy is how to weigh uncertainty.
We can’t expect policies to wait for the same level of certainty required by journals for peer reviewed publications, but it is not unreasonable to expect a high level of rigor from the analyses that support policies. To serve as a guide for policy, we have been developing an inferential framework under the banner of robust analytics. In policy, robust advice would not change if the supporting analysis had been done in a reasonable but slightly different way. In practical terms, robust analytics involve:
going to great lengths to characterize and quantify uncertainty;
building ensembles of scenarios that cover the total uncertainty;
formulating policy advice that emerges from analyzing optimal policies across the ensemble.
Ideally, analysis of policy recommendations across an ensemble would result in a consensus policy recommendation, but it’s possible that there isn’t a clear winner. While the analysis must choose one policy to recommend, the advice could include a discussion of alternative policies that appear to be nearly as good. Robust advice must be thorough, but it can be tenuous.
The natural complement to robust analytics is adaptive management. Health policies are made and adjusted repeatedly on monthly, annual, or multi-year cycles. These policy cycles make it possible to implement policies designed to test hypotheses, accompanied by surveillance to measure outcomes. Policies can thus be used to learn and adapt, but this often comes at a cost. Robust analytics and adaptive management ask us to think carefully about the value of information and how to prioritize collecting the data we lack. The possibility of learning asks us to weigh how evidence could change our minds about something. Ideally, robust policy advice would come with a recommendation about how to fill critical data gaps and reduce uncertainty the next time around. The iterative process of learning and doing is at the core of adaptive management. Correctly striking a balance between these two activities is especially important when the problem we seek to address is dynamic, not static.
Learning in repeated analysis is not a standard part of statistical training, so we must develop and learn new concepts and methods. This website – Health Analytics for Robust Policy – is a resource for people who want to develop, apply, and teach methods to support health policy using the principles of robust analytics or adaptive management. It is a guide to material – concepts, ideas, algorithms, software, commentaries and lessons – that can support robust analytics. We emphasize that adaptive management is focused on policies affecting populations, usually made by national ministries of health; it is not about individual patients or medical practice.
We don’t want this to become a pseudo-journal, so the material is written in a style that could fall anywhere between lecture notes, a user’s guide, vignettes that teach a single concept, or commentary (see the Style Guide). To navigate, use the link at the top of the page – HUB.
September 25, 2023
The core activities of this team are Robust Analytics for Malaria Policy (RAMP) and Adaptive Malaria Control. Some of the ideas we need for RAMP were published recently in Spatial Dynamics of Malaria Transmission [1] – you can read it online at PLoS Comp Bio.
We developed a new mathematical and computational framework for malaria simulation modeling that was modular, flexible, and extensible. The modularity was enforced through rigorous interfaces describing blood feeding and parasite transmission; and egg laying and emergence. The article includes some new methods for building and analyzing models to ensure they would have the capability of dealing with parasite dispersal and spatial heterogeneity. A core motivation was that robust analytics should be done at the right level of complexity, so frameworks should facilitate scalable complexity to make it possible to extend or elaborate on models.
The software has been implemented in two software packages:
We’ve been working on extending the functionality to include exogenous forcing by weather and malaria control. Check back later for new developments, and maybe for some tutorials.
September 25, 2023
Hello, World! – This website was born. New stuff goes in the other tab. Over time, as new stuff gets added, the old stuff goes above this in reverse chronological order.
David L Smith is a Professor in the Department Health Metrics Sciences, School of Medicine, University of Washington. It’s a common name, so Dave curates his Google Scholar Page.
Austin R Carter is a research scientist at the Institute for Health Metrics and Evaluation, University of Washington.
This research was supported by several grants:
We gratefully acknowledge a grant from the Bill and Melinda Gates Foundation (INV 030600). The grant funds an active collaboration with Uganda’s National Malaria Control Division. Adaptive Malaria Control implements research funded by several previous grants from BMGF.
We gratefully acknowledge a grant to support development of Adaptive Vector Control, in collaboration with the Bioko Island Malaria Elimination Project, the National Institute of Allergies and Infectious Diseases (R01 AI163398).
Many of the ideas were developed through a grant from the National Institutes of Allergies and Infectious Diseases (2U19AI089674) that funded Program for Resistance, Immunology, Surveillance, and Modeling of Malaria in Uganda (PRISM), which was an International Center of Excellence for Malaria Research (ICEMR).
Software to support modeling for West Nile Virus transmission dynamics and control in Harris County, Texas was funded by a grant from the National Science Foundation, Directorate for Technology, Innovation, and Partnerships (TIP) as part of the Convergence Accelerator Program (NSF 2040688)
The research grew out of mosquito working groups discussions as part of RAPIDD (Research and Policy for Infectious Disease Dynamics). Over that time, many of us benefitted from the unwavering support and inspiration of F. Ellis McKenzie.
The funders had no role in the development of this material.