It’s amazing how the process of answering simple questions can help you uncover so much complexity. It’s rather like a child lifting a large stone to reveal all kinds of creep crawly things that they never knew existed. Unfortunately, when developing a Theory of Change (ToC) we don’t have the choice, like children, to leave those things alone we don’t like the look of. Developing a ToC gives us the opportunity to address problems, complexity, and opportunities that exist within a programme and to think about how these issues can be monitored and evaluated.

Sometimes it’s difficult to find the right questions at the right time, and this is very true of when it comes to developing a ToC. Often people are thrown into a room together and quite simply asked to get going (without little facilitation or support). The following questions are intended to help guide people in that exact scenario. The questions can be largely treated as sequential, but there are no hard and fast rules in developing a ToC (sometimes it’s necessary to come from a different angle), so if you need to follow a different path these questions are interchangeable.

The majority of these questions are inspired by work that has been done within the advocacy world where pre-determined changes are sought using objective evidence. As such, this provides useful insights that can be applied to the research uptake process.

Figure 1. Initial questions for Research Uptake ToC

1. Are you clear what you want to achieve?

This is the most important question of all because it gets to the heart of why we’re doing research in the first place. It is, ironically, the question that we spend a lot of time thinking about, but the least time articulating in our research strategies.

ToC are valuable for lots of reasons, but we’ll focus on three in particular here. The first is because they force us to be explicit about the kinds of changes we believe (and hope!) our research will bring about. The second is that they help us to think through what kinds of communication and engagement activities we need to do in order to maximise the chances of our research bringing about change. And third, they ask us to think about (and work towards) the wider changes in other peoples’ behaviours and systems which need to happen in order for our research to catalyse change that is sustainable (sometimes called ‘the enabling environment’ for research uptake).

ToC brings together the process implementation elements of a programme and its outcomes. This makes us think squarely about whether the ‘process’ will deliver the ‘outcomes’ and vice versa.

Defining your vision is particularly useful from an Monitoring and Evaluation (M&E) perspective, as it allows you to think more tangibly about what success will look like; what can and should be monitored; and what indicators might be relevant. Donors, like DFID, have asked grantees to develop a ToC in relation to the research uptake strategies of programmes, because in the past these have proved difficult to monitor and evaluate. A ToC ought to improve the balance between ‘activity’ indicators and ‘outcome’ indicators within your programme’s M&E plan and make you think about the tangible ways you get from one to the other.

For example, if your research is looking at the cost effectiveness of teacher training programmes in Ghana, your Theory of Change should make explicit how the findings might be messaged and communicated to stakeholders as diverse as teacher training institutes (to influence their programmes and policies), Education Ministries (e.g. to influence the way budgets are spent on different kinds of teacher training); and schools themselves. Process indicators would look for evidence that the activities have taken place to engage, understand and reach target audiences. Outcome indicators would look for evidence that the desired changes had indeed taken place (e.g. teacher training budgets modified to follow research recommendations; teachers displaying new effective teaching patterns that result in improved child learning etc. However, if you are not clear on what you want to achieve, if the vision is not fully worked, then your indicators are likely to be out of sync from the very start.

Related questions:

• Is your vision plausible?
• Is the timeframe realistic?
• Will the ‘process’ activities deliver the specified outcomes?

2. Do you understand how change [might] happen within the context you are working?

Often there a number of change processes that need to come together in order to bring about change. This can include change processes attached to some of the following things: the policy process; the knowledge/science community; society/behaviour; political forces; and the management/organisation/capacity of the programme. It is unlikely that you will have a grasp of how change happens, without first doing your research.

No one can really tell you how change will in fact come about, but having some intelligence up your sleeve is valuable because it helps shape an approach, and the approach becomes key. This is what you will be asked to monitor and evaluate yourself against so the more you know about how this process might play out the better!

“The process will go more smoothly and produce better results if the stakeholders and facilitators have access to information that allows them to draw on the existing body of research and current literature from a range of domains and disciplines, and to think more systematically about what it will take to promote and sustain the changes they want to bring about.” (Guthrie et al)

The need to understand this process leads on to a series of other questions that relate to the evidence gathered and outcomes captured by other programmes as part of the change process. Context is very important here, so be careful not to assume that the change process will work in your own context, as it has done in others. There are always variables that you need to take account of, when thinking about how change and influence occur in your own context.

The alignment of actors is one area that can be better understood by looking at how this process has been planned for and then how it has actually played out within other programmes. When actors align towards a shared outcome the end result can be stronger and more sustainable, it can also help ensure change is more likely to come about in the first place. As Keystone point out, “Bringing about greater alignment in a system can itself be a crucial pre-condition for success. Relationship building is, therefore, often a strategic objective and indicator that you are indeed making progress towards your long term, sustainable outcomes.”

Related questions:

• Have you looked to see which other actors are working in the same area as your own programme, and assessed how they might influence your outcomes? Could you work together? Or are they opposed to what you are doing?
• Do you have a strategy for dealing with opposing or dissenting voices?
• What change processes are already underway in your ‘ecosystem’, and how do they influence the outcomes that your programme wants to achieve?
• Have you considered relationship building as a key element of success/indicator of change?
• Do you understand how your specific context might shape your outcomes?

3. What strategy are you going to use to make change happen?

Ok, so you have your vision, you have done your homework, it’s time to think of the strategy you are going to employ to bring about the desired change. If you think you understand how change ‘might’ happen then there’s a good chance you will have already started to think about the strategy or strategies that might be employed to bring about change.

Every element of the vision is underpinned by a set of changes that need to take place in order to move towards this end goal. Defining the changes that need to come about and the strategies for dealing with these can be challenging (like changes in confidence, skills, capabilities, relationships, attitudes and behaviour). Some elements of change are tangible changes (such as policy change), but many will be intangible process outcomes that are more difficult to see and document (i.e. changes in confidence, skills, capabilities, relationships, attitudes etc.) (Keystone, pg. 15)

Related questions:

• What needs to happen to make your vision a reality?
• Are all your outcomes necessary?
• Do you understand the pre-conditions that will ensure change is sustainable and long-term?
• What helps you accelerate your efforts?
• What gets in your way or inhibits your progress?

4. Do you have a good understanding of potential benchmarks and indicators?

Policy change goals are often long term and can take a number of years to achieve. This means that the indicators and benchmarks you set a programme are very important means of helping you understand your progress, while also ensuring that the programme can be evaluated effectively. Benchmarks should be set for key milestones within your programme using baseline data.

For policy and advocacy based programmes there are already a number of frameworks in existence that can be used to help you think about relevant indicators. These frameworks provide useful examples of activities, strategies and types of outcomes associated with the policy process. The following framework (see Figure 2 below), taken from a scoping study on Monitoring and Evaluating Advocacy by Jennifer Chapman and Amboka Wameyo for ActionAid, provides a useful example of the kinds of indicators that can accompany a ToC in the area of research uptake.

Figure 2. Framework for understanding possible outcomes and impact of advocacy and campaigning work

Related questions:

• Have you included benchmark indicators that take into account your own capacity development?

Read part 2.

Both parts of this post are available as a PDF download: Key questions to ask when putting together a Theory of Change for Research Uptake by Andrew Clappison

Acknowledgments:

Thanks to IDS for allowing me to work with them on part of the process of developing a ToC for the Transform Nutrition RPC.

Thanks to Megan Lloyd-Laney and Betty Allen for their comments and thoughts on the initial draft.

References:

Kendall Guthrie, Justin Louie, Tom David and Catherine Crystal Foster (2005). The Challenge of Assessing Policy and Advocacy Activities: Strategies for a Prospective Evaluation Approach, The California Endowment

Jennifer Chapman and Amboka Wameyo (2001). Monitoring and Evaluating Advocacy: A Scoping Study, ActionAid

Keystone Accountability (2008). Developing a theory of change: A guide to developing a theory of change as a framework for inclusive dialogue, learning and accountability for social impact,