In this 13-page study, Lai Ma and Rachael Agnew introduce a framework for evaluating impact in grant applications. This framework zeroes in on assessing the real-world, tangible results of your research. The framework is useful for anyone looking to conduct impact evaluations in grant applications for funding programmes supporting the following kinds of research: basic and curiosity-driven, societal and global challenges, academic–industry or academic–non-governmental organisation (NGO) collaboration, and public engagement and science communication.

Most research grants require applicants to describe what impact their research is likely to have in the real world. Not only is impact very difficult to predict, the guidelines are also often unclear on what actually counts as impact.

Ma and Agnew’s study reviews 100 impact case studies submitted as part of the UK’s Research Excellence Framework (REF) 2014, and uses the findings to create a simple framework for evaluating impact. The REF has been a rich source of information on the different ways that academics describe research impact: 3,709 unique impact pathways have been identified across the impact case studies considered.

There are two main types of impact. The first they describe as ‘use-based’ and the other ‘experience-based; you can find a table here that describes each type. The study concludes that most of the case studies under review have not achieved long-term impact (acknowledging that it would be difficult to do so in the short timeframe of most projects) but can show outcomes and medium-term impacts.      

The framework they present, based on their analysis, helpfully considers different types of funding programmes: basic, societal challenges, collaboration, and public engagement.

For basic programmes, the primary aim is to provide space and resources for exploring important questions. So, when you’re applying, remember they’re more interested in the quality of the research proposal, rather than its long-term impacts.

Societal challenges programmes have predefined long-term impacts – the funding agency already knows what kind of long-term impacts they are looking for. Evaluation should assess how well the research project addresses these challenges and national priorities.

Collaboration programmes aim to foster partnerships between academic research and NGOs through co-creation. So, the evaluation here looks at project outputs, expected outcomes, and the potential for long-lasting partnerships. A good way to think about it is assessing compatibility.

Finally, for public engagement programmes, it’s all about enriching cultural understanding and science communication. The evaluation focuses on the exciting activities, the journey, the outcomes, and everyone who got on board.

In conclusion, this framework provides useful insights for anyone applying for a research grant who wants to describe – and have – societal impact. The study warns that the impact case studies method may foster competition between researchers, rather than collaboration, by asking them to ‘claim’ societal impacts. This would limit the overall societal impacts and, in some cases, may stifle contributions that are responsive to society, as well as the progress of research.This article is part of our initiative, R2A Impact Practitioners. To find out more, please click here.