Monitoring and evaluation

Whose knowledge counts?

By 17 June 2014

There are many (brilliant and essential) initiatives around better linking academic research and policy such as 3ie*, DRUSSA and Research Impact. However it is not just research-generated ‘knowledge’ that does and should inform development policy.

According to Harry Jones et al there are 3 types of knowledge: research-based; practice-informed; and citizen knowledge.

Most people think about policy development as typically coming from ‘evidence’ which they in turn understand to be derived from research-based knowledge. Because of the power dynamics around the different types of knowledge, together with a multitude of practical issues facing each, this is not surprising. But practice-informed (and citizen or ‘local’) knowledge is equally valid, relevant and critical to good policymaking. And there is so much practice-informed knowledge out there.

Practice-informed knowledge

Practice-informed knowledge is generally knowledge gained from the experience of ‘doing’. It can derive from planned and formalised knowledge-generating activities through, for example, undertaking and reporting on monitoring and evaluation activities (practice-informed knowledge made explicit); or from unplanned and un-codified knowledge-generating activities, through for example questions raised in a simple routine staff meeting (leading to tacit knowledge).

As a former overseas Project Manager, some of the most relevant and valuable learning I gained (and tried to pass on) from my experience related to, for instance, how best to approach and communicate with key stakeholders: from staff in ministries to local authorities, local NGOs, beneficiaries and even to my particular project implementation team.

So practice-informed knowledge can range from specific learning on a particular issue in a particular sector- such as on Diabetes prevention and control projects in countries with limited resources– to more general and transferable insights into, for example, relationship building in a particular context. And as we all (should!) know, relationships are key to influence, whether for policymaking at the global level or decision-making at the local level.

The more effort spent on monitoring and evaluation activities the more learning or knowledge will be generated. And of course the longer the operating presence ‘in-country’ the more exposure and opportunity for generating and absorbing practice-informed knowledge (whether intended or unintended).

Practice-informed knowledge and policymaking

But whether explicit or tacit, accessing and harnessing this experiential knowledge can prove fundamental to the success or failure of policymaking and of implementation models. Practice-informed knowledge can effectively provide direct and crucial feedback on current or previously implemented policies: in other words, it allows for a testing ground or reality check.

Everyone’s heard of examples of mosquito nets being used as football goals, as fishing nets rather than for the intended purpose of malaria prevention… Generating, capturing and using such practical knowledge on unintended outcomes of policies (as well as of course investing in understanding the operating context in the first place!) is crucial to ensuring that future intervention and policy models are effectively adapted to take into account the ‘reality on the ground’ – and so achieve more positive impact (i.e. the intended outcomes).

Some of the main challenges of recognising and utilising practice-informed knowledge in policymaking however (aside from the power dynamics) are the actual processes surrounding the capture, management and mobilisation of it.

On the collective, global scale there have been many positive initiatives and advancements in knowledge management and mobilisation in the development and humanitarian sector. These include the creation of clusters, working groups and mechanisms for better coordination and knowledge management; the move towards greater monitoring, evaluation learning and accountability with the associated introduction of international standards, bodies and codes of conduct such as SPHERE, HAP and ALNAP; and of course the evolution towards greater advocacy and engagement in policy.

But although some degree of formal, codified practice-informed knowledge may and can be captured and shared externally, more often than not it is held in internal organisational documents such as in routine reports, strategic plans, project logframes and in handovers. Some of the richest knowledge is less tangible and ‘smaller scale’: knowledge that isn’t big or directly relevant enough to share in an inter-agency meeting; that isn’t ‘data’, and isn’t explicitly policy relevant.  It is this kind of rich knowledge that often falls between the gaps.

How to capture, manage and mobilise practice-informed knowledge

So (and quite aside from the multiple other challenges facing the capture and publication of practice-informed knowledge such as the competitive environment, funding systems and confidentiality issues…), what can development practitioners do?

At the individual level:

  • Recognise that you (probably) have generated and hold valuable knowledge that even your colleagues, organisation, counterparts in other agencies/in the sector may not have.
  • Audit your own experience and lessons learned and try to capture it in a way it can be made easily explicit
  • Make a strategic plan to share it with targeted audiences; whether through discussions in meetings with relevant stakeholders; in ad-hoc e-mails to relevant parties; through handovers (always critical in protecting and safeguarding institutional knowledge), which contain such key experiential knowledge; or through other relevant online groups/forums/initiatives such as KM4Dev , Knowledge for Development, or Development Aid Support.

At the organisational level it is more difficult to draw generic tips but some interesting resources and recommendations from people who have previously considered and started to tackle knowledge management and mobilisation issues for practice-informed knowledge can be found via:

  • INTRAC who address the need for NGOs to create the motive, means and opportunity for organisational learning and suggest how to combine these elements into planned and emergent organisational strategies
  • This case study of a particular international NGO network that concludes with recommendations to enhance technical and advisory services, to capacity build, to create greater awareness of knowledge management, to decentralise knowledge management processes, to implement a knowledge management strategy, and to improve relationships between centres.
  • ODI RAPID who set out a Knowledge Strategies Framework which they use to analyse data collected on current knowledge and learning practices across 13 case study organisations. They comment that a learning organisation is normally something that one aspires to; that “learning is a continuous process of becoming, rather than attainment.”  However, against that aspiration they argue that there is a clear need to accept the resource constraints that must inevitably be placed on these initiatives, and work towards goals that are, as far as possible, specific, measurable, achievable, realistic, time-bound and appropriate to a given organisation”.
  • Climate Change Social Learning and Communication which offers an example of a current project that is genuinely trying to incorporate different kinds of knowledge

And what can policymakers do? First, they can recognise that there are these (at least) three distinct types of knowledge (research-based, practice-informed and citizen); and that they will have different things to offer depending on the decisions being made. Second, is that they can strive for making better policies by strengthening the links between these different types of knowledge and deploying them as appropriate in their specific context.

This post was first published on the CommsConsult Ltd. website.

  • Andrew Clappison

    Hi Jo, this is a very rich and valuable post that raises some important issues at a time when evidence in the form of systematic reviews and RCTs (among others) is becoming the new norm in terms of framing and guiding policies. Donors like DFID are placing massive emphasis on evidence-informed policy, and while I agree that this emphasis is important, there is a massive risk that this emphasis overlooks reality and builds a policy process that is disconnected from people and their situated and specific knowledge. Moving forward we need to find a better balance that recognises, values and understands how different forms of knowledge are often interconnected and mutually dependent.

    • Jo Reid

      Thanks for your thoughtful comment Andrew. Think you’ve hit the nail on the head. Whilst research-based knowledge, gained through RCTs and the like, are massively important and valuable, they are often so large scale and ‘high level’ that they can risk becoming disconnected from the nuanced reality on the ground. Triangulating such an approach with other forms of knowledge however leads to much more robust ‘knowledge’ and a fuller picture. This is of course not rocket science but beyond this critical recognition of other forms of knowledge comes the challenge in accessing and harnessing them. But I bet most researchers know/have links to practitioners (and vice versa of course)… Why not start – at the micro-level- by just having an informal chat/dialogue with them about the research, it’s design and findings? One small, easy step that can be achieved by all along this path. And similar for citizen-informed knowledge- try to just speak informally to local friends, colleagues, CBOs…and see what expected and (critically) unexpected information emerges!

  • Jo Reid

    Just came across this – http://siid.group.shef.ac.uk/id100/ – a great example of a project that actively aims to bring different types of ‘knowledge’ together to influence and inform the policy arena. Good luck Project ID100!

  • Another really interesting and related article I’ve just come across about this to add to the list of people already tackling issues surrounding practice-informed knowledge:

    Mike Powell (2006), Which knowledge? Whose reality? An overview of knowledge used in the development sector, Development in Practice, 16:6, 518-532, DOI: 10.1080/09614520600957951

  • Jo Reid

    Another really interesting and related article I’ve just come across about this to add to the list of people already tackling issues surrounding practice-informed knowledge:

    Mike Powell (2006), Which knowledge? Whose reality? An overview of knowledge used in the development sector, Development in Practice, 16:6, 518-532, DOI: 10.1080/09614520600957951

  • ResearchImpact

    Thanks for this insightful post. One group worth considering is the new Knowledge To Practice Learning Network (@KIPLNetwork; also search for their Linked In group). The group explicitly chose “knowledge” into practice, not “research” into practice to embrace the knowledge of non-academic/lived experience/practice based experts. Vicky Ward (Leeds Uni) is a champion of the network and of practice based knowledge.

    One possible joining up of academic and non-academic knowledge is the increasing recognition of the need to inform academic research with the knowledge and needs of stakeholders. How does stakeholder engagement compare with sharing of practice based knowledge? Can stakeholder engagement join up the knowledges of academic and non-academic participants?