This 11-page academic paper written by a team of Australian researchers aims to better understand how research influences policy. The authors study the impacts of childhood obesity prevention research in Australia from 2000 to 2015. They trace the impacts in two directions: forwards from the project by examining research impact assessments and backwards from policy by conducting policy case analyses. This approach allows the team to track the direct impacts of research as well as further, often indirect uses of research such as its influence on knowledge, changing ways of thinking or alerting policymakers to problems.
The academic paper is therefore a great resource for understanding how policymakers use research evidence, how research achieves impact and what might be the best way for assessing research impact on policy. The authors also give examples of the different impacts and corroborate them with comments obtained from policymakers. For example, one research project shows policymakers that the planned initiative will be feasible:
“When XX came in and told us the results, we thought, well, we’re on the right track here. We’ll pursue it.”
In a different scenario, research evidence is used to support intervention choices:
“The fundamental movement skills, that was based on that research by XX. That had already been in place when we ran the obesity summit, but they were getting evidence, and so it really got legs to keep going.”
These comments are useful for understanding how policymakers engage with the evidence and how they implement it in practice. However, out of the studied 148 research projects, only 16% had an impact on policy and a further 19% reached policymakers but weren’t used to influence their decisions. The rest went unnoticed and without any policy impact.
The authors also trace the impacts backwards from policy by analysing three policy initiatives to evaluate how research evidence is used across different development stages. In the first case study, research serves as an important source of information, and it influences decisions throughout the policy process. The research findings work as a catalyst for action, with further studies influencing the design and content of the intervention. In the second example, the evidence is used indirectly. The influence is achieved by applying best practice frameworks and capacity building. In the third case study, research helps to improve decisions and it supports the chosen policy approach.
Overall, the interviews with policymakers show that oftentimes, multiple sources are used to form a decision:
“The X one, we did that ourselves, internally. The other one, the incentives trial, we were worked with X an academic from the X. So, we collaborated there and with other data analysis of X, we worked with X University on further data analyses, looking at what the data were telling us about program performance and issues like that.”
In other words, the academic paper supports the view that researchers alone cannot be held accountable for whether and how their research is used, once policymakers are aware of the findings. The impact often results from an unpredictable merge of events during the policy processes rather than from the research itself or the researcher’s activities. The circumstances surrounding the impact are often specific to the context, difficult to predict and out of the researcher’s control.
In terms of measuring and evaluating impact, the paper demonstrates that it is not always possible to find direct and traceable evidence. Especially difficult is the evaluation of the conceptual use of research: to identify it, you need the input of policymakers who were influenced by the research in that way. Unfortunately, policymakers might not recall all the nuances of how evidence was used.
To conclude, this academic paper is a great resource for those who wish to better understand policy research impact. Measuring research impact from both directions (tracing it forwards from impact assessments and backwards from the policy) captures the different ways in which research is used in policymaking. It is necessary that we examine how policies evolve and develop over time, paying attention to the whole process. Additionally, we need to acknowledge that research isn’t the primary driver for policy change and that knowledge is often accumulated from various sources.
This article is part of our initiative, R2A Impact Practitioners. To find out more, please click here.