A worker from Caritas Homabay is showing local residents beans in Africa.

Scaling Science

A new model for optimizing the impact of research and innovation
February 23, 2018

The scaling of research and innovation that creates social impact is a priority for IDRC and the development community broadly, but how best to achieve impact at scale is far from straightforward.

While we can learn a great deal from the private sector models that have grown increasingly popular amongst innovation and development agencies, these paradigms are designed to achieve growth and maximize corporate profit rather than improve social outcomes.

When the end goal is to enhance the public good, these models, drawn from industrial expansion, pharmaceutical regulation, or the high-tech sector, are not sufficient for the diverse and complex contexts in which development organizations operate. We must adapt and build on these models by drawing on the hands-on experience of designing and implementing innovative solutions across the Global South.

Seeking a more nuanced and systematic approach, IDRC conducted a review of more than 100 of the research projects it has funded. The result is a new approach for achieving impact at scale we call scaling science. Scaling science emphasizes four principles to guide development agencies in their decision-making: moral justification, coordination, optimal scale, and dynamic evaluation.

Together, these principles suggest a concept of scaling grounded in the vast and eclectic experience of the Global South. From this perspective, scaling impact is a coordinated effort to achieve a collection of impacts at optimal scale that is only undertaken if it is both morally justified and warranted by the dynamic evaluation of evidence.

Moral justification

The desire to achieve broad social impact creates a bias toward taking programs, policies, or products to large scale, but scaling social impact is not necessarily synonymous with growth. Viewed from this perspective, questions such as how to scale appropriately and responsibly, under what circumstances to scale, and most importantly  whether to scale, become central. The decision to scale a development effort must be based not only on the values of the organizations leading it, but also on those of the people affected by the innovation. After all, it is their lives that are materially impacted by the outcomes.

To morally justify decisions about scale, organizations need to consider the impact risk borne by those affected — that is, the likelihood of desired positive outcomes as well as of unintended negative outcomes.

The response to the West African Ebola outbreak, one of the projects reviewed by IDRC, stands as an example. During the crisis, no fully tested vaccine was available. Given the urgency of the situation and the dramatic impact risk faced by those on the ground, a standard clinical trial was not realistic. Instead, a promising vaccine was used in conjunction with an innovative vaccination strategy. This approach was less certain, but the level of risk was acceptable under the urgent circumstances with thousands of lives on the line.

Coordination

Innovators must develop relationships with those that make scale possible and those affected by innovations. This principle of coordination goes hand-in-hand with moral justification. While the participation of financial and government players is critical for the resources they provide and the power they possess to clear policy and regulatory hurdles, it is the people affected by innovation that are best positioned to judge whether the impacts achieved constitute success. Their participation increases the likelihood that an innovation will scale successfully and appropriately.

Coordination among these diverse stakeholders can be directed or undirected. With directed coordination, participants agree on a plan of action and a subset of them oversee implementation. Throughout the scaling effort, various actors play more or less prominent roles at different stages of the process.

Undirected coordination follows a more organic approach. Players establish networks of interaction out of which activity becomes self-organizing based on shared priorities, hands-on situational awareness and grassroots initiatives. At the 2016 American Evaluation Association (AEA) conference,  IDRC hosted a formative meeting on the topic of  scaling science design and evaluation.  Mallika Samaranayake, president of the Community of Evaluators South Asia (CoE SA), made the point that scaling does not always have to be top-down. The people themselves can become change agents if they buy into and support an innovation, generating lateral spread and uptake. Her organization has seen successful outcomes in several countries arising from undirected interactions among members of the evaluation systems that CoE SA helped to establish.

Optimal scale

“Bigger is better” is the standard mantra of the business world where industry seeks to achieve economies of scale to increase profit. In the development context, “better” doesn’t necessarily equate with bigger.

The concept of optimal scale offers a way to think critically about how to define impact at scale and how to measure it. Success is not purely quantitative. Qualitative indicators such as sustainability, satisfaction, and quality of life are also key metrics. As programs are scaled, we should be aware that impacts will not grow in a one-to-one relationship and we should be wary of unintended consequences such as degrading positive impacts, amplifying negative ones, or displacing more effective alternatives.

In Tanzania, vitamin A deficiency is a significant health concern, particularly in rural areas. Large commercial producers sell refined sunflower oil that is fortified with vitamin A, but rural areas rely on unrefined and unfortified sunflower oil produced by small and medium enterprises (SMEs). A project to address the vitamin deficiency in the rural populace highlights the importance of appropriate scale. Working with SMEs to provide fortified unrefined sunflower oil necessarily limits the potential scale of the project, especially given that most of the existing producers are too small to cost-effectively adopt the fortification approach. At the same time, by working with SMEs, the project supports local businesses while successfully reaching the people most in need of additional vitamin A. In this case, a targeted intermediary scale makes sense — it meets an immediate need and is matched to the existing context.

Dynamic evaluation

Impacts can increase, decrease, or become qualitatively different as interventions are scaled up. Dynamic evaluation enables programs to stay attuned to changing impact by continuously gathering, assessing, and integrating data as scale, circumstances, and outcomes change. Methods of measurement and analysis can be adapted as necessary throughout the scaling process.

The RAHAT initiative, a survivor-centric approach to providing social and legal support for survivors of sexual violence, developed because of the rape of a four-year-old girl in Mumbai, India. The effective program has since scaled both horizontally to other communities and vertically through institutional layers. However, because the success of RAHAT is highly context-specific and reliant on the expertise of the lead organization, they have employed a dual approach of a targeted external evaluation and continuous internal assessment. The external review looked at the risks of and strategies for scaling in order to ensure the ongoing quality of the intervention. The internal assessment process monitors the changing circumstances (legal, social, institutional, etc.) in both existing implementations and potential localities for expansion so that the program can be adapted as needed.

Panelists at the AEA conference were in agreement that dynamic evaluation should be undertaken with a preference toward action and rapid learning applied to practice so that organizations can be innovative and try new things in the field. However, they stressed that rigorous evaluation is essential when addressing questions of scale.

Putting the principles into action

In conducting the review that led to scaling science, IDRC found the projects that best handled scaling tended to apply the paradigm’s four principles to the process. Producing three planning elements can aid in putting the principles into action: a path to scale, which specifies the stages through which an innovation is expected to pass as it’s scaled; a response to scale, which lays out how impacts are expected to change at these different stages; and partners for scale, which identifies the players involved and their interactions throughout. Together, these elements constitute the starting place for a scaling theory of change.

Scaling science is not intended as a prescriptive guide for successfully scaling projects. Rather, it is proposed as a map to make the complex waters of scaling appropriately easier to navigate. The hope is that it will also generate further discussion and contributions among those involved in development research and innovation.

 

Photo: IDRC / Nichole Sobecki