Monitoring and evaluation

Are we losing sight of the true value of research ‘impact’?

By 11 April 2013

REF 2014 arrived on the UK higher education scene with the promise of changing the way both the academic community and wider society think about the value and ‘impact’ of research. As we have dived deeper and deeper into financial turmoil the emphasis on impact and the impact agenda has grown stronger, and the REF has most certainly become very much entwined with that.

Yesterday, at the Future of Impact Conference, which marked the end of the University of Exeter’s DESCRIBE Project, we heard a great deal about research impact being a means of both accountability and advocacy – A way of showing to donors and tax payers that research funding is value for money, makes a big difference to society and gives a significant return on the investment. We also heard from David Sweeney (HEFCE) how performance based funding has time and time again proved to improve research outputs in their broadest possible sense. He went on to add that “the REF is not about assessing the impact of all academic research, but to present a sample of the best research produced – a means of ensuring that research funding is distributed more wisely in the future”.

The REF has come to act as a pseudonym for ‘research impact’ across the HE sector, and although this is not unsurprising, it does pose a danger to the wider impact agenda and the way impact is perceived among researchers. I’m not suggesting that we should separate impact from the REF, but universities must not lose sight of the need for a more balanced approach, and get away from ‘dirtying’ the waters of the impact agenda by treating it as a bureaucratic exercise underpinned by fierce competition between different academic institutions. Paul Manners (NCCPE)  touched upon this when he encouraged delegates to consider “What does quality engagement and impact mean to you?”

In my own work, which focuses on how to support, track and report the impact of public-goods research related to international development, I perceived the emergence of the impact agenda as continuing recognition of the potential research can play in bringing about social and environmental change, and the need to ensure that valuable research finds its way into the hands of decision-makers and policy actors –heralding a ‘golden age’ of research engagement. Most people will have a different response to Paul Manners question, but the importance of trying to answer it lies in the need to highlight that impact is not just about the REF.

I’m not yet prepared to let go of what to some may appear as a romanticised view of research impact. We heard a great deal yesterday about how the accountability argument dominates conversations around the importance of the REF and research impact, without giving enough air-time to selling the benefits of impact in making research better, building demand into the research cycle, enhancing personal development and career progression.

Michael Wykes (University of Exeter) opened the conference by asking “How do we make impact a cultural norm?” – part and parcel of the academic researcher’s everyday.  There are of course a number of different possible options here, centred on a number of different approaches to embedding impact into academic culture. One approach that came up again and again, especially in light of REF 2014, was the need to systemise the way impact data is collected, to start thinking about the next round of the REF and setting up institution wide systems that start to collect data now rather than retrospectively.

Sarah Morton (University of Edinburgh) talked about the potential of a logical approach, already espoused by major multilateral and bilateral donors and very familiar to me in my own work. According to Morton “this approach allows for the identification of relevant potential users of research, how they would be or have been reached, and what other factors are influencing their behaviours and practices. It creates clear links between inputs, outputs and outcomes which are logical and tenable”. This all sounds very good, and it does hold a great deal of potential for getting researchers to engage with impact in a more holistic way, but there is need for caution. Logical approaches, such as the Logical Framework Approach (LFA) are intensely disliked, often poorly applied and understood by both donors and research organisations alike. Nevertheless, if done well, logical approaches can be highly valuable.

There are of course massive risks here in over bureaucratising institutional approaches to the REF and stalling innovation, as researchers begin to get lost in what Geoff Rogers called “chasing impact” and focusing on supply over demand.

My colleague, James Harvey, has coined the term “impact literacy” to describe an approach that seeks to embed a broader understanding of impact across academia. This rests on placing impact firmly within the research cycle or journey, not as a means to tick boxes, but to unlock potential, shine a light on the value of research engagement and to give impact a chance – empowering researchers to be active and engaged citizens.

The Future of Impact Conference made one thing very clear to me: an embedded culture of research impact will not emerge unless universities think about impact in broader terms beyond the REF.


  • ResearchImpact

    Thanks for posting this Andrew. I have meeting envy as the Future of Impact Conference sounds really energizing especially with folks like Sarah Morton and Paul Manners presenting, both of whom I know and have tons of respect for.

    One person I don’t know if James Harvey. I love what you write about his impact literacy – “This rests on placing impact firmly within the research cycle or
    journey, not as a means to tick boxes, but to unlock potential, shine a
    light on the value of research engagement and to give impact a chance”

    Impact does need to be something that researchers and their partners practice (and budget for in funding applications) as a matter of course, not a special activity added on at the end. Impact also needs to be something institutions support. Not just in a REF retrospective (a REFrospective?) exercise but in a prospective manner, creating the conditions that maximize the chances for impacts to occur.

    We will be exploring this at the UK Knowledge Mobilisation Forum ( and with Association of Research Managers and Administrators the day after the Forum exploring the role of research administration in the impact agenda.

    Tons of fun!

    David Phipps
    York University (Toronto, Canada)

  • Pingback: Angela Vanlods()