Skip to main content

Research Quality Plus

June 14, 2018

Research can change the world

Since its inception in 1970, IDRC has learned a great deal about how research can make a difference in solving complex development challenges. Research encourages open inquiry and debate, empowers people with new knowledge, and broadens the array of options and solutions available to policy and development challenges.

We want the best research to underpin social and natural progress, but how do we know when research is of high quality? And if we can determine what quality research is made of, how can quality research be cultivated?

In conversation with the researchers and research users we work with, IDRC is advancing how research for development is defined, monitored, and assessed. These efforts will strengthen the quality of the research we fund and contribute to the evolving discussion on research quality in the broader research community.

Research Quality Plus: An approach for achieving quality research for development

IDRC has developed a flexible and holistic approach for evaluating the quality of research for development: Research Quality Plus (RQ+). This approach embraces a broad definition of research quality that includes scientific rigor but also recognizes other critical dimensions. RQ+ takes contextual factors into consideration, includes customizable assessment rubrics, and promotes the use of empirical evidence to inform expert evaluations of research quality.

The RQ+ approach can support planning, management, and learning processes of a research project, program, or grant portfolio. Read more here, in brief: Research Quality Plus. Also available in Spanish.

A full introduction to RQ+ is provided in our position paper:

wireframe icosahedron

Research Quality Plus: A Holistic Approach to Evaluating Research

by Zenda Ofir, Thomas Schwandt, Colleen Duggan, and Robert McLean (IDRC, 2016) 

Read our article in Nature

“A better measure of research from the Global South”

by Jean Lebel and Robert McLean. (Nature, vol. 559, no. 7712, 2018, pp. 23-26)

What have we learned from RQ+?

IDRC has experimented with RQ+ in a variety of ways. For example, as a learning tool during a formative evaluation, for bridge-building while forming partnerships with other organizations, and as a post hoc summative evaluation framework.

Recently, we endeavored to bring this learning to a higher level. To do so, we conducted a retrospective meta-analysis of the quality of the research we had supported by critically examining research from across disciplines; within academic, public, and private institutions; and around the world. The RQ+ approach underpinned this work. It was the common framework that allowed multiple study aggregation and the subsequent meta-analysis to work.

The findings of this meta-analysis challenge assumptions and demonstrate the value of research and innovation in the Global South. Read the results and their implications in our brief:
 

Why southern research?  

cover for oxford academic research evaluation

Making a difference in the real world? A meta-analysis of the quality of use-oriented research using the Research Quality Plus approach

by Robert McLean and Kunal Sen (Research Evaluation, Volume 27, Issue 4, 2018)

Man walking between the trees of a forest

Making a Difference in the Real World? A Meta-Analysis of Research for Development

by Robert McLean and Kunal Sen (IDRC, 2018)

External Program Reviews

In 2015, IDRC completed seven external program reviews to assess program strategies, outcomes, and research quality using the RQ+ approach. Read more about implementing the RQ+ framework and its application at IDRC.

Dr Munir checking the flow from the solar powered water pumping system allows Farmers to irrigate their fields on the SMF farm nr Talagang,Pakistan.

RQ+ Framework at a Glance

Female researcher looking in microscope

IDRC’S Research Quality Plus Assessment Instrument

RQ+ Assessment Instrument (IDRC, updated 2020, PDF, 1141KB )

What informs IDRC’s Research Quality Plus approach?

In July 2018, in partnership with Africa’s Science Granting Councils Initiative (SGCI), and co-hosted by the University of Johannesburg and Stellenbosch University, IDRC joined practitioners and leading scholars from across the globe to discuss emerging ideas and practices for improving research quality evaluation. One result of this gathering is a co-edited volume of essays and empirical research, published in 2020 and available in open access.

transforming research excellence book cover

Transforming Research Excellence: New Ideas from the Global South

Edited by Erika Kraemer-Mbula, Robert Tijssen, Matthew Wallace & Robert McLean (African Minds 2020)

In 2011, IDRC launched a study to identify ways to improve evaluation and strengthen research quality. Read more in the brief Evaluating Excellence in International Development Research (PDF, 449.2KB).

Specific results of this study have been communicated in a series of papers: