This 13-page academic paper presents the Program and Policy Change (PPC) framework, a new tool for measuring the use and impact of research in low-and middle-income countries. It was developed by the Center for Development Research, part of the Global Development Lab at the United States Agency for International Development (USAID). The PPC framework is a method specifically designed for tracking the influence of research on program and policy change in international development.
In developing the framework, the authors reviewed three different measurement methods for which they include a comprehensive literature review:
- Metrics of academic impact—peer review and bibliometrics (citations and publication impact factors)
- Innovation metrics—patents, licensing income, and investment mobilization
- Conceptual frameworks and narratives that assess research use and uptake of recommendations
The resulting framework combines both quantitative and qualitative measurements of research impact. It assigns scores to projects based on the extent of use of research and the influence level (how many people might be impacted).
The extent of use is described in the framework as the implementation level. It has four categories with a rising score for increasing research uptake:
First level: Evidence gathering … zero points
- Research is ongoing, findings are not yet produced
Second level: Scientific solution produced … one point
- Research activity is completed, evidence is distributed to stakeholders
Third level: Stakeholder commitment … two points
- Research findings are integrated into policy
Fourth level: Scientific solution implemented … three points
- Research-informed policy has been brought to practise
The second dimension – the influence level – evaluates the reach of the research-informed policy or program. It has three levels that serve as proxies for the number of reached beneficiaries:
First level: local … one point
- Local governments, a few villages, smaller NGOs and civil society organisations are using research findings
Second level: regional … two points
- Provincial governments, cities using research evidence
Third level: national … three points
- National entities are using research findings
All projects that are evaluated under the PPC framework earn zero to three points in the implementation dimension and one to three points in the influence dimension.
The main benefits of the PPC framework are its applicability across various research sectors, its focus on evidence-informed policy at different geographical levels and a scoring system that enables the quantification of outcomes. Additionally, it focuses on intermediate results that have a clear connection to the initial research, which might solve common issues of attribution and time lag of other frameworks.
The paper applies the PPC framework to USAID case studies from Brazil, Kenya and Tanzania, and a program in Nigeria and Cameroon.
However, the framework does not shed much light on capturing either the indirect or conceptual uses of research. It also doesn’t attempt to measure the effectiveness or quality of the policy that is influenced by research. Lastly, the scoring system might be easily misinterpreted. For example, although the national-level influence receives the highest score, some might argue that impact on the local level is just as important and valuable.
In conclusion, the PPC framework is another tool you can consider using if you are looking for a straightforward evaluation method for research for development. It is best suited for applied research projects that aim to address specific problems through policies and programs.
This article is part of our initiative, R2A Impact Practitioners. To find out more, please click here.