Document(s) 1 of 12

Imagine a map . . . drawn from your memory instead of from the atlas. It is made of strong places stitched together by the vivid threads of transforming journeys. It contains all the things you learned from the land and shows where you learned them . . . . Think of this map as a living thing, not a chart but a tissue of stories that grows half-consciously with each experience. It tells where and who you are with respect to the earth, and in times of stress or disorientation it gives you the bearings you need in order to move on. We all carry such maps within us as sentient and reflective beings, and we depend upon them unthinkingly, as we do upon language or thought . . . . And it is part of wisdom, to consider this ecological aspect of our identity.

– John Tallmadge, Meeting the Tree of Life (1997: IX)

Maps are cognitive guides. They locate us, helping us to figure out where we are now in relation to where we’ve been, and to plan where we’re going. It is altogether appropriate, then, that IDRC’s Evaluation Unit has chosen the metaphor of mapping to guide those interested in development on the sometimes confusing, even frightening journey through the hazardous territory of outcomes.

The language can be daunting: outcomes, impacts, goals, objectives, purposes, mission, and outputs — and these terms just scratch the surface. The questions can overwhelm. What’s the difference between evaluation and monitoring? How do short-term changes relate to intermediate changes and long-term results? What kinds of results count as outcomes? How can the need for accountability be balanced against the need for learning? Then there’s the attribution problem. To what extent and in what ways can one establish a causal linkage between activities, outputs, outcomes, and impacts? Who gets credit for results? What kinds of evidence are credible? What’s the unit of analysis? What role does stakeholder involvement play in all this?

The territory of developmental change and evaluation is vast, complex, and ever-changing. Trying to manoeuver through that territory one is likely to encounter deep ravines of uncertainty, mountains of data, and side-tracks that lead nowhere. It sure would help to have a map to the

territory. This manual on outcome mapping can’t provide the specific map you need for your own territory, for each territory is unique and presents its own challenges, but this manual will tell you how to create your own map. It will guide you through the language forest of terminology. It will show you how to navigate the winding river of a results chain. It will help you figure out where the boundaries are of the territory you are exploring and provide assistance in identifying “boundary partners” to accompany you on the outcomes journey. It will show you how to construct a strategy map and figure out progress markers.

A vision of useful and meaningful evaluation — evaluation in support of learning — undergirds this manual and is essential to understanding its important contributions. A sophisticated understanding of contemporary evaluation issues informs what may appear as simple mapping exercises provided here. One of the strengths of the manual is that it cuts through the complex evaluation literature, extracting deceptively straightforward and commonsensical wisdom from the many divisions and debates within the evaluation community based on a deep knowledge of, and explicit value premises related to, development. The staff of IDRC’s Evaluation Unit have long been working to support learning as a primary outcome of development program evaluation. They have observed that longer term outcomes and impacts often occur a long way downstream from program implementation and may not take the form anticipated. These longer term outcomes depend on responsiveness to context-specific factors, creating diversity across initiatives. The outcomes examined include the depth and breadth of involvement by many stakeholders, processes that become results in and of themselves when done in ways that are sustainable. These characteristics make it difficult for external agencies to identify and attribute specific outcomes to specific components of their programs or to aggregate and compare results across initiatives.

Outcome Mapping offers a methodology that can be used to create planning, monitoring, and evaluation mechanisms enabling organizations to document, learn from, and report on their achievements. It is designed to assist in understanding an organization’s results, while recognizing that contributions by other actors are essential to achieving the kinds of sustainable, large-scale improvements in human and ecological well-being toward which the organization is working. The innovations introduced in Outcome Mapping provide ways of overcoming some of the barriers to

learning faced by evaluators and development partners. Attribution and measuring downstream results are dealt with through a more direct focus on transformations in the actions of the main actors. The methodology has also shown promise for across-portfolio learning in that it facilitates standardization of indicators without losing the richness in each case’s story, thus combining quantitative and qualitative approaches.

An old hiking adage warns that “the map is not the territory.” True enough. You need to keep your eyes open and watch for unexpected outcroppings and changes in the terrain. But without a map, you can get so lost in the territory that it’s hard to even figure out where you started much less find your way to an outcome. Outcome Mapping provides not only a guide to essential evaluation map-making, but also a guide to learning and increased effectiveness, and affirmation that being attentive along the journey is as important as, and critical to, arriving at a destination.

Michael Quinn Patton

17 September 2001

Michael Quinn Patton is an independent organizational development and evaluation consultant. He is the author of five books on program evaluation including a new edition of Utilization-Focused Evaluation: The New Century Text (1997). The two previous editions of that book have been used in over 300 universities around the world. His other books are Qualitative Evaluation and Research Methods (1990, 2nd edition); Creative Evaluation (1987); Practical Evaluation (1982); and Culture and Evaluation (1985). Patton is former President of the American Evaluation Association. He is the only recipient of both the Alva and Gunner Myrdal Award from the Evaluation Research Society for “outstanding contributions to evaluation use and practice” and the Paul F. Lazarsfeld Award for lifetime contributions to evaluation theory from the American Evaluation Association. He has a vast academic background and is a faculty member of the Union Institute Graduate School, which specializes in individually designed, nonresidential, nontraditional, and interdisciplinary doctoral programs. He also has been involved with the development of the African Evaluation Association.

Document(s) 1 of 12