Evaluating Large-Scale Health Programs Specialization

This course provides an introduction to designing and evaluating to strengthen the health system. After successful completion of all learning activities, course participants will be able to:

1. Define health systems strengthening 2. Describe health systems frameworks and how to incorporate them into evaluation planning 3. Describe how to design and prioritize implementation of health systems programs 4. Detail approaches to evaluate health systems programs, including data sources and study design 5. Introduce tools to assist with evaluation planning and simple models to assess health system intervention impact 6. Describe how to interpret health system evaluation findings and sustainability and scalability implications The development of this course was supported by a grant from Government Affairs Canada (GAC) for the Real Accountability, Data Analysis for Results (RADAR) project.

This course provides in-depth knowledge about implementation strength, quality of care, and service utilization, which are essential components of health program delivery. This course is primarily aimed at implementers, managers, funders, and evaluators of health programs in low- and middle-income settings (LMISs) targeting women and children, and undergraduate and graduate students in health-related fields.

Those who complete this course successfully will be able to: 1. Explain why assessing health program delivery is an essential part of any large-scale program evaluation in LMISs. 2. Design an assessment of delivery for a health program in a LMIS, including proposed documentation of the program, indicators, measurement methods and tools, relevant contextual factors, and a plan for data analysis. 3. Interpret assessment results in the context of a large-scale effectiveness evaluation, and explain how they can be used to improve program policies and implementation. 4. Describe the RADAR tools for assessing implementation strength and the quality of care, and access guidelines for the use of the tools. The development of this course was supported by a grant from Government Affairs Canada (GAC) for the Real Accountability, Data Analysis for Results (RADAR) project.

This course provides an introduction to household surveys for program evaluation in low-and middle-income countries. The course will equip you with skills to:

1. Explain what coverage is, why it’s important in evaluations, and how it is measured 2. Describe what household surveys can and cannot measure 3. Plan, implement, and analyze household survey, including: 4. Calculate an appropriate household survey sample size 5. Explain the resources required for a household survey 6. Identify an appropriate sampling design 7. Design a questionnaire 8. Describe challenges for coverage survey planning and implementation 9. Present and describe how the RADAR tool can be used The development of this course was supported by a grant from Government Affairs Canada (GAC) for the Real Accountability, Data Analysis for Results (RADAR) project.

We want to provide you some information about our course “Measuring and Modeling Impact in evaluations”. The purpose of this course is to give you a better understanding of different measures of impact that could be used in the evaluation of a program in the areas of maternal and child health and nutrition. For each of the measures presented, we will discuss current sources of data you might draw on as well as describe the methods that can be used to measure these. When we describe the methods, we also try to identify the strengths and weakness of the methods as well as their suitably for use in an evaluation. The course also discusses how modeling can be used in evaluations as either a replacement for measuring impact or to supplement measured impact. The last two lessons in this course focus on giving you an introduction and training on how the Lives Saved Tool (LiST) works and how to use it. This model can be used to estimate most – if not all – of the impact measures we describe in the course and can be an important part of both planning and estimating impact in an evaluation of a large-scale program. While this course is self-contained, it is also linked to other courses on evaluation. We developed this course for public health program managers and evaluators and assume the students in the course will have a background in public health with a focus on maternal and child health in low- and middle-income countries.

The development of this course was supported by a grant from Government Affairs Canada (GAC) for the Real Accountability, Data Analysis for Results (RADAR) project.

This course is for implementers, managers, funders, and evaluators of health programs targeting women and children in low- and middle-income countries as well as undergraduate and graduate students in health-related fields. Course participants will learn how to 1) transform quantitative components of an evaluation measurement plan into a sound analysis plan to address the evaluation questions, 2) conduct quantitative analyses of primary or secondary surveys or other available data, 3) interpret the meaning of the analysis results and their implications, and 4) disseminate the evaluation findings to program implementers, local and global stakeholders.

It is highly recommended that course participants have the statistical skills to conduct and understand quantitative analysis. The development of this course was supported by a grant from Government Affairs Canada (GAC) for the Real Accountability, Data Analysis for Results (RADAR) project.