International Development Research Centre (IDRC) Canada     
idrc.ca HOME > Publications > IDRC Books > All our books > LEARNING FROM CHANGE >
 Topic Explorer  
IDRC Books
     New
     in_focus
     Development/evaluation
     Economics
     Environment/biodiversity
     Food/agriculture
     Health
     IT/communication
     Natural resources
     Science/technology
     Social/political sciences
    All our books

IDRC's 40th anniversary

Subscribe

Free Online Books
 People
Rodrigo Bonilla

ID: 114837
Added: 2007-08-18 0:05
Modified: 2009-04-01 23:30
Refreshed: 2010-03-14 06:36

Click here to get the URL for the RSS format file RSS format file

Chapter 11. Getting the right end of the stick: Participatory monitoring and evaluation in an organizational context
Prev Document(s) 10 of 24 Next
Penelope Ward

Introduction

CARE INTERNATIONAL is a non-sectarian, non-profit development and relief organization. It operates on four continents and in over 50 countries. CARE began operations in Zambia in January 1992 upon the invitation of the Zambian Government. Their activities initially included those related to responding to the severe drought of the early 1990s, and interventions to mitigate the effects of escalating inflation and extreme poverty in urban areas.

By 1994, it became clear that conventional development approaches, such as food for work (FFW) activities, created dependency. Project participants were abandoning marginal income-earning activities for the perceived security of FFW. As a consequence, CARE Zambia decided to reorientate its development strategy. The household livelihood security (HLS) approach was adopted, which provided a more holistic perspective on factors that affect people's livelihoods (Drinkwater, 1994). Fundamental to this reorientation process was a shift from physical development projects towards a more human development emphasis, aimed at building individual and organizational capacities.

Using the HLS approach as a framework, CARE Zambia then began to encourage a more learning-oriented approach within the organization. Becoming a learning organization was perceived as a critical step towards better understanding of HLS, changes experienced by households over time, the impact of project activities, and areas for future intervention. Organizational learning entails learning along with local communities and ensuring that villagers are 'getting the right end of the stick'. It is a long process that aims not only towards organizational learning within CARE Zambia, but also towards building local capacities to improve programme planning and intervention.

In this chapter we outline the process that CARE Zambia underwent to achieve the shift towards a more learning-oriented and participatory approach to livelihood development. The Livingstone Food Security Project (LFSP) is used here as a case study to illustrate the steps and strategies CARE has undertaken to improve institutional learning and become more responsive to people's needs and priorities. Our experiences in CARE Zambia reveal the following lessons (which are considered in more detail at the end of the chapter):

  • participation is a process, not just an activity

  • learning from the project context does not happen automatically; it needs to be integrated as part of project activities

  • really 'handing over the right end of the stick' involves more than having villagers collect information for the project

  • ownership of the learning process is a vital component of capacity building, which occurs at two levels:

  • the project organizational level

  • the community or village level.

Building a learning organization within CARE Zambia

The HLS approach was promoted within CARE Zambia in order to (Drinkwater, 1997):

  • improve CARE's ability to target poor and vulnerable households in its programmes

  • monitor and develop a deeper understanding of trends in the improvement or decline of HLS in communities over time

  • ensure that project activities address livelihood and food security concerns of households

  • create synergistic relationships between projects with the same geographical coverage, so that activities of different projects complement each other and help to address the overall needs of vulnerable households

  • create coherent country office information and monitoring systems that are able to measure project impact at different levels within a community/project area.

One significant development in adopting the HLS approach within CARE Zambia has been the need to establish information feedback and review systems. CARE Zambia wanted to monitor trends and changes in HLS, in order to use this information directly to improve its programming and interventions. This then required developing a more learning-oriented approach within the organization that would encourage project staff and local partners to learn from their experiences and to respond more effectively to changing needs and priorities. The following sections briefly describe the HLS framework and the concept of a 'learning organization'.

What is HLS?

Chambers and Conway (1992: 6) define livelihoods as:

'the capabilities, assets (stores, resources, claims and access) and activities required for a means of living. A "sustainable livelihood" is one which can cope with and recover from shocks (e.g. one-off events such as a death, illness or retrenchment) or stress (e.g. long-term events such as prolonged drought, continued unemployment or illness of a breadwinner); maintain and enhance its capabilities and assets, and provide sustainable livelihood opportunities for the next generation; and which contributes net benefits to other livelihoods at the local and global levels over the long and short term.'

Figure 11.1 describes the HLS framework. It illustrates the relationships between capabilities, assets and economic activities. Assets include the capabilities and skills of household members, their physical assets and resources, their access to information and influential people, and their ability to claim from relatives, the state or other agents in times of stress. Using these assets, a livelihood is able to undertake various production (e.g. agriculture, fishing), processing and exchange activities (e.g. trading, manufacturing). These collectively contribute to the household's consumption of food and other commodities and services (health, education, recreation), and, ideally, to investment in strengthening the asset base of the household.

Image

Figure 11.1: The household livelihood model Adapted from: Swift (1989); Drinkwater (1994); and Chambers and Conway (1992)

The 'learning organization' as a concept

Adopting the HLS framework within CARE Zambia has emphasized the importance of building a learning-oriented organization in order to improve its livelihood development activities. The 'learning organization' as a concept is increasingly recognized as a valuable tool for strengthening people's capacities, establishing effective feedback mechanisms, and improving performance (see Box 11.1).

The key features of a learning organization include:

  • adapting to the environments in which it operates

  • continually enhancing its capability to change and adapt

  • developing collective as well as individual learning

  • using the results of learning to achieve better results.

Box 11.1: What is a 'learning organization'?

'A Learning Organization is one in which people at all levels, individuals and collectively, are continually increasing their capacity to produce results they really care about.'

(Karash, 1997:1)

'A Learning Organization is an organization skilled at creating, acquiring, and transferring knowledge and at modifying its behaviour to reflect new knowledge and insights.'

(Garvin, 1993:78)

 

Four levels of learning

The process of becoming a learning organization involves different levels of learning. Hamel and Prahaled (1994) have developed a model that defines four levels of learning to show how organizations evolve and develop new knowledge and skills (see Box 11.2).

According to this model, learning processes at Levels 1 and 2 can be accomplished relatively quickly and easily within an organization. The most critical challenges to higher learning begin when organizations develop from Level 2 to Level 3. At this stage of learning, staff must learn how to adapt their new skills to different situations in the field. This requires creativity and adaptive thinking. Institutionalizing this adaptive behaviour requires time, experience and skilled facilitation. Organizations that have attempted to progress beyond Level 2 often need to develop a number of strategies to encourage creativity and innovation amongst staff. Some of the strategies that CARE Zambia has undertaken to promote institutional learning are described in the following sections.

Box 11.2: Four levels of learning

Level 1

  • Learning facts, knowledge, processes and procedures
  • Applies knowledge to familiar situations where changes are minor

Level 2

  • Learning new job skills that are transferable to other situations
  • Applies knowledge to new situations where existing responses need to be changed
  • Bringing in outside expertise as a useful learning strategy

Level 3

  • Learning to adapt
  • Applies knowledge to more dynamic situations where the solutions need to be developed
  • Experimentation and deriving lessons from success and failure

Level 4

  • Being innovative and creative – designing the future rather than merely adapting to it
  • Assumptions are challenged and knowledge is reframed.

Institutionalizing learning within CARE Zambia

Seven strategies were adopted by CARE to encourage and institutionalize learning, particularly amongst staff at all levels within the organization (see Box 11.3). Providing staff with training and relevant skills was an important step in this process. These strategies or 'building blocks' may be considered benchmarks in helping CARE staff become more learning oriented.

CARE staff soon found that building a learning organization was not so straightforward and clear cut as they had anticipated. Most of the time, we were unsure of what we were doing and what skills we were learning. We all started out at different levels of understanding and learning, and many of us were completely new in the field. We would try out different approaches simultaneously, and many times we failed miserably. But one important element of CARE's transition was the learning environment it created: staff freely shared their ideas and experiences amongst each other, and were encouraged by higher-level staff. Institutional learning did not always take place formally (i.e. through trainings); more often, staff exchanged ideas during their day-to-day activities, i.e. in corridors, over coffee, during meetings, etc. Hence, learning became part of a larger, more fluid process.

Adaptive and creative learning in the community

Developing community-based monitoring systems

The LFSP is used here to show how CARE Zambia staff attempted to apply a learning-oriented approach to programme implementation at the community level. The case study illustrates the process through which CARE staff learned how to apply new knowledge and skills by using participatory approaches and methodologies. It further describes how CARE staff involved villagers in establishing a community-based monitoring system, not only to improve programming of activities but also to strengthen local institutions in sustaining livelihoods and household food security. Box 11.4 provides a general overview of the LFSP, describing project objectives, activities, and partners involved.

Establishing a community monitoring system

The LFSP staff designed a monitoring system centred around the use of the HLS model. The model was used to structure interviews with individual village households. This information was later supplemented with a wealth ranking exercise conducted together with participating villagers, who specified their own categories of wealth and identified sustainable livelihood indicators for future monitoring (Box 11.5). By using the HLS model to structure household interviews and then applying the wealth-ranking exercise, CARE Zambia project staff and participants gained a better understanding of people's livelihoods status, their coping strategies in times of stress and shock, and their needs. This initial assessment provided the basis for developing a monitoring framework.

Box 11.3: Strategies to build a learning organization within CARE Zambia

1 Thriving on change

  • Senior staff and external consultants help to introduce the concept of household livelihood security and participatory learning and action (PLA) techniques into mission programming
  • Existing projects encouraged to make the shift from conventional service delivery activities to a more holistic livelihood approach to development
  • Experienced senior staff able to provide guidance, support and vision on an ongoing basis

2 Facilitating learning from the surrounding environment

  • New participatory methods are developed and applied in the field - resulting in more solid community ownership of project activities
  • Projects are redesigned so that beneficiaries participate more in design and implementation
  • Staff establish and train community-based teams responsible for monitoring and planning project activities

3 Facilitating learning from staff

  • Long-range strategic planning sessions held during which core values and three-year strategic thrusts are drafted for the mission
  • All projects encouraged to produce logframes, monitoring frameworks and annual work plans through team work and discussions
  • Through team work staff are able to demonstrate an understanding of the larger participatory programming framework within which their individual roles lie

4 Encouraging experimentation

  • Appropriate and experienced external consultants employed to design and conduct training and fieldwork to expose staff to new methods; project staff benefit from continuous contact and follow-up by experienced senior staff and consultants. This provides staff with access to necessary skills and resources to practise participatory learning in their work
  • Staff and project participants begin to develop an effective array of their own participatory tools, e.g. household livelihood monitoring systems

5 Communicating successes and failures

  • Projects develop methods to document case studies and share experiences in the field, such as newsletters, inter-project discussions and staff sharing
  • Staff and participants learn to monitor progress, analyse results and to use this information to modify activities
  • Experienced senior staff provide ongoing guidance and support

6 Rewarding learning

  • The mission's core values are incorporated into a revised annual staff performance appraisal. Recognition is awarded to those who demonstrate these values most effectively
  • Project successes and experiences are shared across the organization through newsletters and inter-project meetings
  • Staff are encouraged to publish papers, give presentations at international conferences and attend international training courses

7 Promoting a sense of caring

  • Annual staff appraisals recognize performance and outputs regarding core values
  • A staff tuition reimbursement scheme is developed

Because of the large project area and relatively few staff, it became imperative that the implementation of the monitoring system be conducted mainly by the villagers or groups within each community. A pilot community self-monitoring system (CSM) was introduced in 45 villages. Participants included members from village management committees (VMCs), who are elected by villages and who were initially mobilized by CARE Zambia to manage food relief activities. CARE staff provided training to build the capacity of these local institutions, including training selected members in monitoring, collecting and evaluating information.1 CARE staff then held follow-up meetings and workshops to help VMC members analyse information, especially in drawing out trends across the project area and by comparing data collected by the various VMCs.

Data collected by the VMCs covered HLS trends during the first two agricultural seasons of the project, 1994-5 and 1995-6. (Between 1996 and 1997 data for the last season was still being collected and had not yet been completed in most pilot villages.) By repeating interviews with the same households each season,2 the HLS framework was used to monitor trends and changes in livelihoods over the seasons. Distinct trends have begun to emerge from the CSM data, and it has been possible to see marked improvements in peoples' livelihoods over these two seasons. One of the most striking trends noticed was the increase in food availability across the different household wealth categories. CARE staff and villagers related this trend partly to the amount and types of crops that have been cultivated, the project's seed distribution activities, and the promotion efforts led by the VMCs and village seed groups. Reasons given by villagers to explain low agricultural productivity especially in poorer households - namely inappropriate choice of crop, soil type, poor farming practices, pest attacks – provided important lessons on how to improve household food security and suggested new or revised project strategies.

Box 11.4: The Livingstone Food Security Project (LFSP)

Project objectives

  • To develop a community-based seed multiplication and distribution system
  • To build the capacity of community institutions to plan, manage and maintain activities crucial to drought mitigation and ensuring household food security
  • To develop sustainable farming systems
  • To improve water harvesting methods
  • To raise incomes by developing market linkages and improving income-earning opportunities

Project area

  • Southern Province in Zambia
  • 9,600 participating farmers

Project activities

  • In the first two growing seasons (1994-5 and 1995-6), the main agricultural activity was the introduction of drought-tolerant crops through a community-based seed distribution and bulking-up scheme. Information on crop and soil agronomy, seed handling and post-harvest storage was provided
  • In the first season of operation (1994-5) a pilot seed scheme involved 330 farmers on an individual basis. During the 1995-6 season, the scheme was institutionalized. VMCs were established. These registered a number of seed groups, each consisting of four to seven households. 180 VMCs were established with over 6,800 participating farmers
  • In 1996-7 the project area expanded and the number of participating farmers increased to 9,600

Project partners

The community participatory planning process involved three sets of actors:

  • project field staff
  • district and field staff from the Ministry of Agriculture and Forestry (MAF)
  • the villagers living in Southern Province

CARE staff, who were now quite familiar with using participatory methods in the field, grew more confident and began applying the different tools more consistently throughout the project cycle. More importantly, they soon developed a better understanding of the principles of participation and adapting what they had learned in training to actual field situations. However, developing a CSM system still posed a challenge to CARE staff and revealed areas of their work that needed further improvement. For instance, monitoring workshops were still initiated and designed mainly by LSFP staff, who played a greater role than the local community in identifying indicators for monitoring project impact. Little input was sought from the community regarding their data needs and expectations, and their main role was limited to that of data collectors. As a result, villagers still perceived the CSM largely as a CARE-driven process. Villagers have not been sufficiently involved in analysing information and using data to regard the monitoring process as fully their own.4

Box 11.5: Wealth ranking with communities

'Wealth' or 'well-being' ranking is a common tool used to establish and define local terms and definitions of wealth status. It is useful for obtaining a quick, general understanding of the nature of wealth differences in a project area, and for determining the approximate wealth status of participants.

Three types of livelihoods were identified by communities together with LFSP staff:

1 Rich households3

  • Can maintain household livelihood security on a continuous basis
  • Able to withstand shocks (a one-off event, such as the death of a breadwinner or a season of drought) and prolonged stress (a long-term event, such as continued drought, illness or unemployment)

2 Moderate households

  • Suffer shocks and stress, but have the resources to be able to recover relatively quickly

3 Poor households

  • Become increasingly vulnerable as a result of shocks and prolonged stress

CARE project staff found that identifying people's criteria of well-being and livelihood categories were critical elements in establishing baseline information for subsequent monitoring activities. The wealth-ranking exercises enabled staff to:

  • develop a deeper understanding of vulnerable households within their project area
  • identify criteria for monitoring the improvement or decline of household livelihood security over time, particularly in poorer households
  • review and target project activities more specifically at households' livelihood and food security needs.

Institutionalizing learning at the community level: experiences of the LFSP

A major thrust of the LFSP has been to strengthen the capacities of local institutions to better plan, manage, and maintain livelihood activities. Monitoring the project's progress towards this objective is crucial to ensuring the continuity of project benefits beyond the lifetime of the project itself. In developing a CSM system, project staff have moved through four levels of learning – although learning, as noted earlier, was a much more dynamic, backwards and forwards process.

Learning facts and knowledge

Initially LFSP staff had limited or no experience in applying the HLS framework and using participatory techniques. As part of their training in participatory rural appraisal (PRA), project staff conducted a series of participatory appraisal activities (including wealth ranking) in three different farming system zones within the project area. CARE staff gained hands-on experience, working jointly with communities to establish baseline information, analyse farmer livelihoods, and learn about key issues affecting livelihoods and people's priorities (Mitti, Drinkwater and Kalonge, 1997).

At this stage, staff were simply acquiring new skills and knowledge and still absorbing new ideas and applying their training. Many had little experience of using participatory methods in the field and had yet to grasp the full implications of applying participatory approaches throughout the project-cycle. Their previous work experience with conventional projects (i.e. hand-out type activities) still strongly influenced their behaviours and attitudes in the field and in programming livelihood activities. Hence, external consultants and senior project staff needed to provide continuous support and advice. Regular reinforcement of team learning and self-assessments was critical in preventing project staff from returning to conventional, top-down planning.

Learning new job skills transferable to other situations

In March 1996 a project baseline exercise was conducted by LFSP field staff in 20 villages within the project area. Various PRA tools, including wealth ranking, were used to collect this data over a two-day period spent in each village. LFSP staff learned how to apply their new skills in participatory appraisal methodologies towards developing a baseline. The baseline was designed to provide a general impression of the current local context as well as a foundation for future monitoring exercises.

However, senior CARE staff were still needed to provide guidance in determining how collected information should be used. There still appeared to be a tendency amongst project staff to gather information without having a clear strategy for its use or for conducting follow-up activities. This then tended to make the process of obtaining baseline information extractive. Village participants functioned more as providers of information, rather than as active partners in analysing and using the information collected. In the future, follow-up activities and action planning will be needed if the project intends to use collected information towards refining its existing strategies and involving communities in project design and implementation.

Learning to adapt

Initially CARE staff used the HLS framework mainly to conduct a needs assessment for determining interventions and establishing baseline information. As project staff grew more confident working with villagers and gained a better understanding of local situations, project staff then carefully reoriented their strategy to emphasize the strengthening of community institutions. Project staff sought to build local capacities by establishing a CSM system through VMCs. Project staff worked together with VMCs to analyse and compare information across the project area, and to discuss how this information could be used to improve project interventions.

Being innovative and creative

The project is currently reviewing the initial CSM system and looking at different ways to improve and make the process more participatory and locally inclusive. Our experiences so far show that the level of community involvement in recording information and participating in the monitoring process has not been consistent. Some VMCs have been more conscientious in their data gathering than others, while several villages have not completed data collection. This is partly attributed to limited community ownership of the monitoring and evaluation process itself. There is limited, or lack of, local involvement in data analysis and in directly using information. There remains a tendency for staff to control the process and to conduct data analysis themselves, at the expense of community and local institutional learning. Emphasis is still on individual learning amongst project staff: generally, one or two staff members conduct data analysis in isolation. As a result, key lessons and insights are not widely shared, limiting new ideas and innovations from taking shape.

In the future, staff will need to provide greater support to VMCs by encouraging them to identify more locally-meaningful indicators and to use collected information directly for their own purposes. By encouraging villagers to find innovative ways of using information, it is hoped that they will begin to monitor their progress and teach or assist other villages in the project area. This would be an essential step towards creating strong and capable community institutions that will continue work on securing household livelihoods even after the project itself has been completed.

Lessons learned

Several of CARE Zambia's programmes have already implemented an HLS community-monitoring system to keep track of household livelihood trends and to monitor the impact of project activities. Our experience in establishing such a system has been mixed.

Lessons from this experience can be summarized as follows:

  • Participation is a process, not just an activity. CARE staff have been trained and have had to apply participatory methodologies in the field within a relatively short period of time. Adequate time must be given to the process of internalizing and applying these new attitudes and behaviours if they are to be sustained. Adequate training, follow-up and mentoring (especially from top-level management) are all critical components of this process.

  • Learning from the project context does not happen automatically, it needs to be an integral part of project activities. One problem that staff encountered is developing the ability to analyse and document the lessons that they are learning in the field. Staff have stronger interactive than recording skills; consequently, key insights are communicated mainly through anecdotes and remain undocumented. This then limits the potential for institutional learning. Hence, more effort and resources need to be invested in monitoring and evaluation, which should be considered an integral part of project activities.

  • Really 'handing over the right end of the stick' involves more than villagers collecting information for the project. Although VMCs have participated in the CSM process enthusiastically, they did not play a significant role in designing the self-monitoring surveys or analysing and using the information afterwards. Little thought was given to how communities might use the data themselves. As a result, villagers see monitored information as belonging primarily to project staff and being for CARE's use.

  • Ownership of a process is a vital component of capacity-building. Field staff need to follow up with VMCs more regularly to sustain progress, and provide support in dealing with problems that arise. Progress has been slow in some villages due to sporadic follow-up and lack of VMCs' sense of ownership of gathered information. In order to strengthen local capacities and to sustain community monitoring, villagers will need to play a greater role throughout the entire monitoring process, and not simply act as data gatherers.

The most important challenge is trying to overcome the tendency to extract information, and to empower participants through proactive self-monitoring and learning. Many of the tools used have great potential for encouraging participation at the grassroots and for genuinely involving project beneficiaries in the analysis and use of information. However, there is a danger of the process becoming extractive if insufficient dialogue is sustained with communities, and if they are only used as informationgatherers. When this occurs, information – and hence learning – will flow one way: out of communities and simply into project management. Joint analysis and discussion help communities and staff learn from each other and identify ways to improve programme planning and to build local and organizational capacities. A learning-oriented, monitoring process helps in motivating and empowering communities to take action themselves, which, in turn, strengthens local capacities and promotes self-reliance.







Prev Document(s) 10 of 24 Next



   guest (Read)(Ottawa)   Login Home|Careers|Copyright and Terms of Use|General Infomation|Contact Us|Low bandwidth