Policy Evaluation Resources

These resources include step-by-step information, tools, and sample questions for evaluating policy work.

The Challenge of Assessing Policy and Advocacy Activities: Strategies for a Prospective Evaluation Approach

The California Endowment and Blueprint Research & Design, Inc.

Recommends an evaluation strategy for prospectively evaluating policy change activities: 1) Develop a conceptual model for the policy process, 2) Develop a theory of change, 3) Define benchmarks and indicators, and 4) Collect data on benchmarks. Focuses on measuring progress as well as outcomes.

Audience: The document is geared towards policy grant projects but is applicable to all policy projects.

 

The Challenge of Assessing Policy and Advocacy Activities: PART II—Moving from Theory to Practice

The California Endowment and Blueprint Research & Design, Inc.

Compilation of feedback from a meeting to discuss Part I of the report and generate ideas on how to implement it.

Audience: The document is geared towards policy grant projects but is applicable to all policy projects.

 

Continuous Progress Online Toolkit

The Aspen Institute

Provides a step-by-step guide for planning policy change activities and conducting evaluations before, during, and after. Discusses monitoring progress as well as impacts.

Audience: Organizations involved in policy work.

 

Evaluating Philanthropic Support of Public Policy Advocacy: A Resource for Funders

Northern California Grantmakers

Provides case studies of different approaches to evaluating policy work, analyzes the potential strengths and weaknesses of these different approaches, and raises questions to be considered when designing an evaluation. Provides information on evaluating the incremental steps that lead to policy change as well as assessing policy outcomes.

Audience: The guide is geared towards funders. However, the evaluation approach and lessons learned may be helpful to those developing their own policy evaluation plan.

 

Four-part webinar series on policy evaluation

National Collaborative on Childhood Obesity Research

Aims to increase skills of researchers and practitioners in policy evaluation effectiveness. Topics include Basics of Design to Evaluate Policy Interventions, Addressing Pitfalls to Research in Real World Settings, Enhancing the Usefulness of Evidence to Inform Practice, and Communicating Results Effectively.

Audience: Researchers and practitioners

 

A Guide to Measuring Advocacy and Policy

Annie E. Casey Foundation and Organizational Research Services

Presents a framework for evaluating outcomes that occur as a result of advocacy and policy change efforts. The approach breaks down into three steps: 1) Start with a theory of change, 2) Identify outcome categories, and 3) Select a practical and strategic approach to measurement. The document provides a menu of broad outcomes (e.g., shift in social norms) as well as examples of outcomes (e.g., changes in awareness) and the strategies associated with the broad outcomes (e.g., media activities), and unit of analysis (e.g., population groups).

Audience: The primary audience is foundation officers and other grant-making organizations, but it also applies to organizations engaged in policy work.

A Handbook of Data Collection Tools: Companion to A Guide to Measuring Advocacy and Policy

Annie E. Casey Foundation and Organizational Research Services

Companion to “A Guide to Measuring Advocacy and Policy.” Provides examples of tools and processes for collecting useful information from policy efforts. Provides data collection options for broad outcome areas including: Shifts in Social Norms, Strengthened Organizational Capacity, Strengthened Alliances, Strengthened Base of Support, Improved Policies, and Changes in Impact.

Audience: The primary audience is foundation officers and other grant-making organizations, but it also applies to organizations engaged in policy work.

Policy Evaluation Framework

Center TRT

Includes a policy evaluation framework with a sample logic model, sample evaluation plan, and tips for developing a logic model and evaluation plan. Also includes sample evaluation questions for formative evaluation, process evaluation, and outcome evaluation as well as suggestions for data collection.

Audience: Specifically geared towards policies targeting environmental and behavioral obesity-related outcomes, but could be adapted to other policy topics.

 

Practical Policy Evaluation Webinar

Center TRT

Provides an overview of the evaluation framework, a case-study presentation of the application of the framework, and a Q & A session.

Audience: Specifically geared towards policies targeting environmental and behavioral obesity-related outcomes, but could be adapted to other policy topics.

A User’s Guide to Advocacy Evaluation Planning

Harvard Family Research Project

The tool has four steps to complete in sequence: focusing, mapping, prioritizing, and designing. Focusing involves understanding who the evaluation’s users are and how they will use the evaluation, as well as what evaluation questions they want answered; mapping involves developing a visual map of the strategy being evaluated (logic model); prioritizing involves prioritizing what to evaluate and how; and designing involves deciding what data points will be captured and how. Completing all the steps in the tool will generate the core elements of an advocacy evaluation plan, including which outcomes will be measured and how. Includes a planning worksheet.

Audience: Organizations involved in policy work, evaluators, funders, or other stakeholders.

 

Topic-Specific Policy Evaluation

 

These resources were developed specifically for practitioners working in the field of nutrition, physical activity, and obesity and tobacco. However, they may be able to be adapted for other health policy topics.

 

Evaluation of State Nutrition, Physical Activity, and Obesity Plans

Centers for Disease Control and Prevention

This tool was specifically developed for state nutrition, physical activity, and obesity (NPAO) programs in order to evaluate their state plans, but it also includes more general information on evaluating initiatives identified in the plan. Clarifies approaches to and methods of evaluation.

Audience: State NPAO programs, but it can be modified for other topics.

 

Evaluation Toolkit for Smoke-Free Policies

Centers for Disease Control and Prevention

Describes how to conduct an evaluation to assess the impact of local smoke-free laws.

Audience: Designed for community tobacco control programs and coalitions, but may be useful for other audiences.

Introduction to Program Evaluation for Comprehensive Tobacco Control Programs

Centers for Disease Control and Prevention

"How to" guide for planning and implementing evaluation activities.

Audience: Specifically geared towards tobacco, but could be adapted for other health topics.

 

Measuring Policy and Legislative Changes

National Cancer Institute

Describes measures and methods developed for the evaluation of the American Stop Smoking Intervention Study (ASSIST) to assess state and local legislative changes in youth access laws and clean indoor air laws.

Audience: Specifically for those working on tobacco policy

 

Indicators and Outcome Measures

These resources include information on selecting indicators and outcome measures for an evaluation.

 

Key Outcome Indicators for Evaluating Comprehensive Tobacco Control Programs

Centers for Disease Control and Prevention

Provides information on selecting indicators and linking them to outcomes. Discusses in detail 120 evidence-based key indicators that have been scientifically linked to program outcomes. Documents the evidence that shows the value of using these indicators to measure the progress of a state tobacco control program.

Audience: Specifically geared towards tobacco, but could be adapted for other health topics.

Measuring Population Health Outcomes

R. Gibson Parrish, MD

Reviews common outcome metrics for population health, such as disease-specific mortality and the percentage of adults reporting fair or poor health. Discusses the attributes of a good metric.

Audience: Anyone needing to develop outcome metrics to measure a particular policy or program.

Outcome Indicators

Urban Institute

Provides a table of common outcomes and indicators and a chart detailing how one outcome leads to the next.

Audience: Anyone involved in policy work.

 

Using Indicators for Program Planning and Evaluation

Centers for Disease Control and Prevention

Focuses primarily on the use of outcome indicators for evaluation of policy, systems, and environmental change approaches to heart disease and stroke prevention.

Audience: Intended for use by National Heart Disease and Stroke Prevention program grantees, state health departments, advocates, evaluators, and researchers in program and evaluation planning that are using the DHDSP Outcome Indicators for Policy and Systems Change.

 

The Centers for Disease Control and Prevention also maintains a list of evaluation resources.

Download this list. (PDF)