Monitoring and evaluation

Participatory approaches to impact: Learning from the STEPS Centre at Sussex Impact Day

By 22 June 2015

At the Sussex impact Day on 16 June, staff from the University of Sussex, plus a few visitors, gathered to share methods, experiences and questions on research impact. My colleague, Adrian Ely, and I ran a workshop to share our learning from using a participatory impact method within the STEPS Centre. We wrote about it in a working paper last year, which is available on the STEPS website.

STEPS Centre researchers sometimes describe themselves as ‘engaged’: they aim to confront real and serious human-environment problems, and open up possibilities to benefit poor and marginalised people. The research is informed by values and seeks to promote positive change, and much of our field research is done with local partners in various places in Asia, Latin America and Africa.

At the same time, we recognise that STEPS is only one actor in a vast, complex and dynamic system of policy processes and socio-ecological change, with many kinds of knowledge and evidence being generated and influencing each other, and used in different ways.

So if a change happens, how do you know it was because of you? It’s rarely obvious, and often you have to use secondary or indirect ways to indicate it – looking at intermediate ‘outcomes’, positive interactions with key players, or even changes in the language used in reports or meetings.

Values, choices, strategies

There are other problems too. How do you decide what impact you want, and choose who to target or engage with? What values inform these decisions? How do you keep track of wider changes?

Participatory methods respond to these questions by drawing in more voices and perspectives into the process of articulating a project’s vision for impact, which networks it should engage with and how it will do so.

In the last 5 years, STEPS has used a version of Participatory Impact Pathways Analysis (PIPA) – (loosely) adapted from an approach developed by Boru Douthwaite and colleagues and pioneered at the International Centre for Tropical Agriculture (CIAT). It was applied in the Challenge Programme on Water and Food, which spent $12 million per year over 10 years and aspired to have a measurable impact on the MDGs (Douthwaite 2013).

Keeping things in proportion

In comparison, STEPS projects are tiny, so we adapted PIPA to focus on parts of it that could be worked through quickly – typically in an afternoon. Our scaled down version includes formulating a shared problem and vision, creating a network map of relevant actors, and using the vision and map to discuss communications and engagement strategies.

Collective network mapping lets us talk about who the people/actors are in the system, their power and influence, how they relate, and whether they might be helpful, indifferent or resistant to our aims. With PIPA, it’s important for this discussion to bring in different perspectives, knowledge and values. In theory, a workshop should include ‘end-users’ of the project as well as various stakeholders in it, not just the core project team. STEPS hasn’t always had the resources to do that, but we’ve always involved local partners – including non-researchers who have worked on the same issues over the long-term – and tried to hold the workshops close to where the field work actually happens.

diagram2Impact Day

What have we learned? STEPS project leaders noted that the method helped to air differences and promote team-building. It also foregrounded the need for communications and engagement early on. And it’s allowed us to be more realistic, prioritising who to engage with.
We’ve also found that repeating and reviewing the process can help to refocus, aid learning and allow people to appreciate how the context changes over time. And we’ve also had some initial success in bringing different STEPS projects together and comparing or linking their impact work across the Centre’s portfolio of research.

There are challenges too. One is persuading people that doing a PIPA workshop is worthwhile, and allocating the time and money for a workshop – but also to track and analyse impact – is really challenging in small projects. Monitoring and evaluation is also expensive and needs its own systems and methods.

Academics also find themselves under massive pressure to publish papers, leaving less time for other forms of outreach. Funders do require us to document impact-related material, but the collection system is intensely structured according to categories and discrete events, so doesn’t allow us to discuss the dynamics, politics and intersections we encounter.

Questions and ideas

Our Sussex Impact Day group came from diverse disciplines – social sciences, development studies, English, law, business and education – and threw up some great ideas and questions.

Participation: Are the STEPS workshops participatory enough, and what would it mean to involve more perspectives? One attendee discussed a project involving children, and wondered how they could be meaningfully and safely included in such a process.

Articulation: Others thought a participatory impact method might help to reflect on, and be more explicit about, the benefits of inclusive project design – for example where diverse steering committees are used. Thinking properly about participatory impact methods could link to ESRC’s interest in the ‘co-production’ of research.

Another attendee compared our method to business modelling, where dynamic mapping is increasingly used to imagine how relationships and interactions could develop over time.

Legacy was another question – once we’ve finished a project, we need to allow local partners to take forward impact methods and use or adapt them, if they wish, in ways that meet their needs.

We were very grateful for the discussion and ideas generated by our Sussex Impact Day session – we just wish there had been more time! We’d like to hear comments, questions or reactions from you too, as we continue our journey into applying and adapting impact methods with the STEPS Centre and our partners.

Topics: , , , ,

Nathan Oxley

Nathan Oxley has worked on communications and impact at the Institute of Development Studies since 2009. He is a member of the STEPS Centre's Impact, Communications and Engagement Unit, which seeks to apply participatory impact methods to research projects, and understand the advantages and disadvantages of using these methods. Nathan also works as the website editor and acting communications manager for the Future Agricultures Consortium, an Africa-based alliance working on the political economy of African agricultural policy.

  • Tricia Ong

    Really nice little article, especially for those of us who have adopted participatory approaches to research in Asia (Nepal) and who are developing new (participatory) research methods.