Applying lessons in evaluation

February 01, 2011
Keane J. Shore

The International Development Research Centre (IDRC) sponsored 11 people from organizations in the West Bank and Gaza, Jamaica, Senegal, Mexico, Ecuador, and Uruguay to attend the third annual International Program for Development Evaluation Training (IPDET) in Ottawa. Researchers are now applying the skills they learned to their work in the field.

“The idea was to help give them the ability to evaluate their own projects and improve their approaches,” says Sarah Earl of IDRC’s Evaluation Unit. “It may help them meet the Centre’s evaluation needs, but the real aim is for groups to run their own evaluations in any situation.”

Hosted by the World Bank’s Operations Evaluation Department and Carleton University’s Faculty of Public Affairs and Management, IPDET attracted 187 participants from 57 developed and developing countries — an increase of 15% from the previous year.

Diverse participants, already experts in their research fields, come to learn evaluation from international authorities and, as important, from each other. After the course, they stay connected through an Internet peer network where they share challenges and solutions related to evaluation.

“They work under similar constraints and they’re all trying to meet the same test of excellence,” says Karen Ginsberg, IPDET’s manager. “Good evaluation leads to good decision-making on programs and projects that are undertaken by governments or other organizations. It leads to good governance. You don’t know what you have until you look back and do an evaluation.”

When IPDET looked back recently on its own summer program, it found high satisfaction rates. Over 80% of participants in 2002 and 2003 believed that the program helped develop knowledge and skills that they would use in the field. Those satisfied customers included three IDRC-sponsored participants: Alberto Narváez, Arturo Campaña, and Clara Píriz, who each took away insights to help them in their work.

Mining for answers

In a remote region of Ecuador near the border of Peru, mines have contaminated a river that spans two municipalities. Alberto Narváez, who works in Quito for the Fundación Salud Ambiente y Desarrollo (FUNSAD), a foundation for health and environmental development, is trying to find out how decades of gold mining have affected the environment and surrounding residents.

Narváez wants competing interests to agree on how to clean up the problem. A half-dozen previous projects had varying success, and — with new skills developed at IPDET — Narváez is helping FUNSAD fine-tune its approach to evaluating those earlier hits and misses.

“We’ve tried to build a partnership approach. This was our most important activity in the last six months,” Narváez says. “For us, the most important thing is to link strategies with outcomes.”

The deadly scent of flowers

In Ecuador’s Cayambe region, vast plantations produce cut flowers for export. The beauty of the product, however, is offset by the harmful impact of the work on labourers, who are exposed to farm chemicals.

Arturo Campaña, who works for the nongovernmental organization Centro de Estudios y Asesoría en Salud (CEAS — Centre for Health Study and Assessment) is trying to persuade flower plantation labourers to monitor their farm chemical use to reduce pollution. CEAS needs to evaluate its efforts in the context of its wider aim to improve the overall health of the region’s population.

In recent qualitative research into how cut-flower workers relate to chemicals, researchers were unsure how to analyze answers to open-ended questions. Campaña found his own answers in Ottawa.

“That’s one beautiful thing I’m taking home to my work,” Campaña says.

Finding a balance

The Río de la Plata region along the coast of Uruguay is home to most of the country’s population, as well as to intensive agricultural, industrial, shipping, touristic, and fishing activities. Researchers are developing integrated coastal zone management (ICZM) policies that will help ensure current activities are more balanced and sustainable.

As geographers and biologists try to improve coastal ecology by integrating approaches of sometimes-at-odds government ministries and communities, Clara Píriz tries to find how all the personalities relate. IPDET has given her ideas about evaluating how players cooperate and communicate. Evaluation can help her stay focused on results in a complex and shifting situation.

“You can’t try to understand everything in this kind of project because it’s not simple,” Píriz says. “You have to keep thinking about the next step, and what to develop in the project and the activities. Above all, you’re always trying to improve the methodology and the way you are working.”

Transferring lessons into action

Narváez, Campaña, and Píriz agree the evaluation skills they’re taking back to Latin America will help them and their groups. They also can see their new skills applying to other areas. Píriz, for instance, thinks evaluation could well bear on her new university posting as academic supervisor for extension education.

Earl says quality evaluations aren’t easy, and anything that helps IDRC’s partners do better jobs is welcome. “The projects they do with us and others will be better, because they’ll use evaluation to question what they’re doing and get better at what they’re doing. They'll also be able to articulate what they’ve done and how well they’re doing it,” Earl says. “People who work in our research projects are extremely passionate about what they do. They’re so invested, when you come with evaluation as a way to get better at what they’re so passionate about… they jump on it.”

Keane J. Shore is a freelance writer based in Ottawa.