This 23-page guide by Zenda Ofir, Thomas Schwandt, Colleen Duggan and Robert McLean presents the RQ+ Assessment Framework, an approach to defining and evaluating the quality of research. It was developed and tested in practice by the Canadian International Development Research Centre (IDRC).

The framework is based on the premise that an assessment of research quality should not only include an evaluation of research outputs but also consider important aspects of the context in which the research was conducted and the manner in which it was managed. 

You can tailor the RQ+ framework to your context and purpose. It can support your planning, management and learning at any stage.

There are three main components forming the RQ+ Assessment Framework:
I/ Key influences

This component highlights the factors that are most likely to affect research quality. The document identifies five main influences on research quality: 

• a) Maturity of the research field,
• b) Research capacity strengthening, 
• c) Risk in the data environment, 
• d) Risk in the research environment, and 
• e) Risk in the political environment. 

II/ Dimensions and sub-dimensions

There are four key quality dimensions in the RQ+ framework. These are:
1. Research integrity … looking at the technical quality, appropriateness and rigour of research design and execution.

2. Research legitimacy … evaluating the extent to which the research took account of the concerns, values and perspectives of stakeholders. 

This dimension includes four sub-dimensions that ask assessors to consider:

• 2.1 the potentially negative consequences of research, 
• 2.2 gender responsiveness,
• 2.3 inclusiveness of vulnerable populations, and,
• 2.4 the engagement with local knowledge.

3. Research importance … considering the importance and value to key users and the understanding generated by the research. 

There are two sub-dimensions: 

• 3.1 research originality,
• 3.2 research relevance.

4. Positioning for use … the extent to which the research outputs are prepared in a way that improves the probability of use, influence and impact. 

Two sub-dimensions include: 

• 4.1 knowledge accessibility and sharing, and 
• 4.2 actionability and timeliness.

The guide highlights the adaptability of this part of the framework. While you can use these pre-defined dimensions and key influences, you are also free to develop new ones which would reflect your needs and context more accurately. 

III/ Evaluative rubrics 

The performance of the research project is assessed by customisable rubrics that combine quantitative and qualitative measures. The key influences are rated on a rubric with a three-point scale (low, medium, high) that establishes a risk profile. For the quality dimensions and their sub-dimensions, the rubrics have a scale from one to eight across four levels (unacceptable, less than acceptable, acceptable to good, and very good). The document provides examples for several dimensions, explaining what it means that the research is “good” or “unacceptable”.

Overall, there are four steps in the application of the RQ+ framework:
Step 1: Selecting the sample … selection of projects and project outputs for assessment

Step 2: Characterising the selected projects based on key influencing factors … examining the research context and applying the relevant rubric

Step 3: Rating the quality of the research … rating the quality of outputs in the project sample using the quality dimensions and sub-dimensions 

Step 4: Synthesising the ratings … aggregating the ratings and rolling them up across projects, programs and portfolios to reach the desired level of evaluation.

The guide offers a visualisation of the framework’s roadmap and shows an example of a fictional synthesis of results to demonstrate the assessment process. Additionally, the resource describes the experience of IDRC, which applied the framework during its external reviews in 2014/2015. It identifies both positive and negative aspects of using the RQ+ framework in practice, such as its customisability, consistency and ability to compare different types of research. The resource also highlights several pitfalls of the framework, such as its time-consuming aspect.

In summary, the RQ+ Assessment Framework is a flexible and adaptable evaluation tool for assessing research quality. The guide is written in a very accessible and comprehensible way, explaining and illustrating the thinking behind the framework, its strengths and weaknesses and possible uses.

This article is part of our initiative, R2A Impact Practitioners. To find out more, please click here.