This 17-page academic paper by David Budtz Pedersen, Jonas Følsgaard Grønvad and Rolf Hvidtfeldt explores the current literature on research impact in social sciences and humanities (SSH). The paper studies different impact assessment frameworks and methods, drawn from 158 academic papers and 125 policy documents. It is a great resource if you’re interested in learning about the range of different frameworks and methods that exist for evaluating societal impact, and the advantages and disadvantages of each one.

In the first section of the paper, the authors describe the major frameworks to assess SSH societal impact, including the Research Excellence Framework (REF) in the UK, the standard evaluation protocol in the Netherlands, IMPACT-EV model funded by the European Commission, the SIAMPI model based around the idea of productive interactions, the Research Contribution Framework (RCF), contribution mapping and the RAPID outcome mapping approach. 

The second section describes the different methods and tools for tracking impact and lists their advantages and disadvantages. The most frequently used methods across the frameworks are interviews, case studies, surveys, expert or peer reviews and statistical databases. 

Overall, interviews are the most preferred method for studying research impact. They are often highlighted as one of the most useful sources of information. Similarly, case studies are mentioned as the central method in almost all assessment frameworks. They can be complex and provide descriptions of pathways that lead to the use, uptake and impact of research in practice. However, they are often criticised for being subjective, not providing clear evidence to back up claims and very labour-intensive. 

The third chapter of the literature review looks at the main distinctions between the assessment models and what it means for discussions about research impact. For instance, the academic paper mentions the tensions between positive and negative impact and the case-based versus metric-based evaluations. 

The authors stress that societal impact does not happen in a vacuum but takes place in a complex environment of interacting actors, interests and values. Additionally, ‘impact’ does not mean the same thing across institutions, research cultures and geographies, so it makes sense that a range of different models and methods are used for measuring different aspects of impact. There are no gold standards or magical indicators of societal impact. 

Overall, the authors recommend that the assessments of impacts should be based on a) contributions of research to societal impact instead of attributing impact to specific projects, b) a clear understanding of values and scales such as interests, locality and temporality, and c) acknowledgement that impact takes place in a complex ecosystem. 

The academic paper is an accessible resource for those looking for a nuanced understanding of non-academic impact, a new framework or method to try and/or those interested in assessing the societal impact of SSH research.

This article is part of our initiative, R2A Impact Practitioners. To find out more, please click here.