Monitoring and evaluation

Looking for a tool to analyse and ‘compare’ policies? Check out lessons learned from conducting a QDA

By 12 November 2013

by Elise Wach

As part of our team efforts to maintain a reflective practice and share learning to others, one of our latest ‘Practice Papers in Brief’provides some insights from conducting a Qualitative Document Analysis (QDA) on policy documents for the rural water sector.

The QDA was undertaken as part of the Triple-S (Sustainable Services at Scale) initiative, for which the Impact and Learning Team (ILT) at IDS facilitates learning.

Qualitative Document Analysis (QDA) is a research method for systematically analysing the contents of written documents.  The approach is used in political science research to facilitate impartial and consistent analysis of written policies.

Given that Triple-S is aiming to change policies and practices in the rural water sector, the initiative decided to undertake a QDA on policy documents at the international level in order to understand trends and progress in the sector and also to engage development partners in identifying possible changes to policies and practices to move the sector closer to achieving ‘sustainable services at scale.’  Later, we decided toexpand this to ‘practice’ documents as well.

Consistent with Triple S’s ‘theory of change’, generating discussion on these issues and catalysing change was just as much of a priority as generating reliable evidence about policy trends.

In the paper, we discuss the strengths and weaknesses of the methodology, and provide some pointers that might be helpful if it is a tool you might consider using.

Overall, we found that the QDA exercise provided useful information about trends and gaps in the rural water sector, helped to refine the Triple-S engagement strategy, and served as a useful platform for engagement with partner organisations.

Some of our lessons related to issues of defining our ‘themes’ and scoring, inclusion criteria for documents, unclear or zero scoring, and the relationships between the research team and the organisations included in the review.

Next week, Triple-S will be kicking off another QDA for the Ghana Workstream, to analyse government rural water policies, and will incorporate many of the lessons that we’ve learned on QDA so far.  We’ll also be conducting another round of QDA at the international level next year to analyse the ways in which the rural water sector policies have shifted over the course of the Triple-S project, and to understand what to focus on moving forward.

Title: Learning about Qualitative Document Analysis Author: Elise Wach, Richard Ward and with contribution from Ruzica Jacimovic Year: 2013
*Monitoring, Evaluation & Learning Advisor, Impact and Learning Team, Institute of Development Studies IDS.
This post originally appeared on the IDS  blog, we are grateful for them to allow us to share it here,