Document(s) 13 of 15

Terms Used in the Evaluation of Organizational Capacity Development

Action learning A process in which a group of people come together, more or less regularly, to help each other learn from experience. Participants typically come from different organizations or situations, and each of them is involved in different activities.

Action planning A process usually associated with training, linking improvements or actions with what has been learned. Action planning establishes a course of actions chosen to realize the application of what was learned or decided.

Action research A participatory, democratic process concerned with developing practical knowledge in the pursuit of worthwhile human purposes, grounded in a participatory worldview, which is currently still emerging. It seeks to bring together action and reflection, theory and practice, in participation with others, in the pursuit of practical solutions to issues of pressing concern to people, and more generally the flourishing of individual persons and their communities.

Adaptive capacities The capacities needed for an organization to learn and change in response to changing circumstances.

Capacity The ability of individuals and organizations to perform effectively, efficiently, and in a sustainable manner. (See ‘Organizational capacities’.)

Capacity development An ongoing process by which individuals, groups, organizations, and societies increase their abilities to perform core functions, solve problems, define and achieve objectives, and understand and deal with their development needs in a broad context and sustainable manner.

Case study The detailed investigation of one or more organizations, or groups within organizations, with a view to providing an analysis of the context and processes involved in the phenomenon under study. The phenomenon is not isolated from its context (as in laboratory research for example) but is examined in relation to its context.

Collective knowledge Collective knowledge is an outcome of organizational or institutional learning. (See ‘Organizational learning’ and ‘Institutional learning’.)

Commitment A pledge or obligation to carry out some action or policy or to give support to a policy or person.

Direct use of evaluation results The instrumental use of evaluation results by decision-makers as the basis for a decision. Direct use occurs when information or findings are applied directly to change an action or alter a decision. (See also ‘Indirect use’.)

Direct observation Gathering information about things that can be observed. For example, by visiting an organization, one can directly collect information on the physical surroundings. By observing meetings, one can observe who shows up, how people interact, and what decisions are taken.

Effectiveness The extent to which desired objectives are achieved. The extent to which an organization achieves its mission and objectives.

Efficiency The extent to which results are achieved with minimum use of resources. The degree to which an organization generates its products and services using a minimum of inputs.

Evaluation Systematic investigation of the worth, value, merit, or quality of an object. Assessment of the operation or the outcomes of a program or policy, compared to a set of explicit or implicit standards, as a means of contributing to its improvement. The criteria for evaluation may include relevance, effectiveness, efficiency, impact, and sustainability. (See also ‘Monitoring’.)

Evaluation culture An organizational culture that values evaluation and seeks solutions to problems, trying out tentative solutions, and weighing the results and consequences of actions within an endless cycle of supposition–action–evidence–revision that characterizes good science and good management.

Evaluation questions Questions formulated to help focus an evaluation on key topics or issues.

External operating environment The environment in which an organization operates. Includes such things as the administrative and legal systems that govern the organization, as well as the political, economic, technological, social, and cultural context in which the organization operates.

Financial resources The funding available to the organization to carry out its activities.

Goals The highest-level objective of an organization, project, or program.

Group interview A technique that uses a (small) number of informants to collect perceptions and opinions.

‘Hard’ capacities The tangible assets and resources of an organization, such as its land, buildings, facilities, personnel, and equipment.

Human capacities The knowledge, skills, and attitudes of the members of an organization.

Impact Any effect, whether anticipated or unanticipated, positive or negative, brought about by a development intervention. In some cases, ‘impact’ refers to

the long-term effects of an intervention on broad development goals. (See also ‘Output’ and ‘Outcome’.)

Indicator Quantitative or qualitative factor or variable that provides a simple and reliable means to measure achievement, to reflect the changes connected to an intervention, or to help assess the performance of a development actor.

Indirect use of evaluation results Conceptual use of evaluation results in decision making. Refers to an intellectual and gradual process in which the decision-maker is led to a more adequate appreciation of the problems addressed by the policy or program. (See also ‘Direct use’.)

Institution A socially sanctioned and maintained set of established practices, norms, behaviors, or relationships (i.e. trade regulations, land tenure, banking systems, and an organization’s staff rules) that persist over time in support of collectively valued purposes. Institutions have both formal and informal rules and enforcement mechanisms that shape the behavior of individuals and organizations in society. (See also ‘Organization’.)

Internal environment Factors inside an organization that make up the organization’s ‘personality’, and which influence the organization’s cohesiveness and the energy it displays pursuing its goals. Factors that make up the internal environment include: the organization’s culture, performance-related incentive, and rewards systems, the institutional ‘climate’ in general, the history and traditions of the organization, leadership and management style, the existence of a generally recognized and accepted mission statement, and shared norms and values that promote teamwork in the pursuit of the organization’s goals.

Institutional learning The learning that takes place among individuals in different organizations and groups, who are working together to achieve a common end and, in particular, to induce institutional change. (See also ‘Organizational learning’.)

Joint evaluation An evaluation undertaken by two or more parties to achieve a mutual objective. There are various degrees of ‘jointness’ depending on the extent to which individual partners cooperate in the evaluation process, merge their evaluation resources, and combine their evaluation reporting. Joint evaluation can help overcome attribution problems in assessing the effectiveness of programs and strategies, the complementarities of efforts supported by different partners, the quality of aid coordination, etc.

Key informant interview Key informants are those ‘who know’, and are not necessarily representative of a population. They are chosen for their knowledge or distinctive viewpoint. The key informant interview method forms part of focused interview techniques (as distinct from sample survey interviewing) and is governed by the need to identify a wide range of different viewpoints.

Leadership The capacity to assess and interpret needs and opportunities, to establish direction, to influence and align others towards a common aim, motivating and committing them to action, and making them responsible for their performance.

Logic model A simplified chain of relationships that portrays the logic and assumptions underlying a program or intervention and how it intends to achieve its expected results. It states the logic of the program, identifies the assumptions on which it is based, and outlines the logical connections between (a) the activities undertaken, (b) the outputs to be produced, (c) the immediate or short-term outcomes that are expected, and 9d) the ultimate or long-term impacts the program is designed to achieve.

Management The classical view emphasizes the management functions of planning, organizing, commanding, coordinating, and controlling—‘getting the work done by the best means available’. More recently, the enabling role of managers has been emphasized, ‘to create the conditions under which the work will be done, and done well’. In the context of agricultural research, management involves defining research goals, strategies, and priorities; formulating research programs; determining responsibilities; allocating resources; leading, motivating, and supervising staff members; and maintaining relations with stakeholders.

Monitoring Monitoring involves continuous, systematic observation and checking on activities and their results. The purpose is to ensure that activities are proceeding according to plan, to provide a record of how inputs are used, and to warn of deviations from initial goals and expected outcomes. (See also ‘Evaluation’.)

Needs assessment A decision-aiding tool for planning and resource allocation. Involves the gathering and analyzing of information on the organization, its environment, its capacity needs and problems, and possible solutions.

Objective An expression of an effect that a program is expected to achieve if completed successfully and according to plan. Objectives are often viewed as a hierarchy, beginning with strategic goals, purposes, outputs, and activities.

Operational capacities The capacities that an organization needs to carry out its day-to-day activities.

Operating environment The context or environment in which an organization operates, including the economic, technical, socio-cultural, institutional, legal, and political factors that influence behavior and performance.

Operational management Management concerned with mobilizing, coordinating, and guiding an organization’s staff and using its physical and financial resources to achieve defined objectives. Establishing a coherent set of rules (institutions) that guide behavior in the pursuit of organizational goals. (See also ‘Institution’.)

Organization Formal structures with designated roles and purposes. Entities composed of people who act collectively in pursuit of shared objectives. These organizations and individuals pursue their interests within an institutional structure defined by formal rules (constitutions, laws, regulations, contracts) and informal rules (ethics, trust, religious precepts, and other implicit codes of conduct). Organizations, in turn, have internal rules (i.e. institutions) to deal with personnel, budgets, procurement, and reporting procedures, which constrain the behavior of their members.

Organizational assessment framework A theoretical construct that aids in the diagnosis of an organization’s current state, to measure changes over time or to find ways to solve specific problems. This study employs a framework developed by the IDRC and Universalia that includes four analytical dimensions: the external operating environment, the internal environment, organizational capacity, and performance.

Organizational capacities The organization’s potential to perform. Its ability to define and realize goals effectively, efficiently, and in a relevant and sustainable manner.

Organizational capacity development An ongoing process by which an organization increases its ability to formulate and achieve relevant objectives. It involves strengthening both its operational and adaptive capacities. (See ‘Operational capacities’ and ‘Adaptive capacities’.)

Organizational change Alteration or variation in the character or performance of an organization. Such changes lie along a continuum from incremental change to fundamental, large-scale change or transformational change. While incremental change is less complex than fundamental change, both types involve three basic stages referred to as ‘unfreezing’, ‘moving’, and ‘freezing’. Fundamental or large-scale change refers to lasting change in the character of an organization that significantly alters its performance.

Organizational culture A pattern of shared basic assumptions that an organization develops as it solves its problems of external adaptation and internal integration, has worked well enough to be considered valid and, therefore, to be taught to new members as the correct way to perceive, think, and feel in relation to these problems.

Organizational goals Statements describing the external direction of success, ultimate achievement, or desired improvement in organizational performance. (See also ‘Organizational performance’.)

Organizational learning An organization’s capacity for accumulating knowledge from its own experiences, disseminating that knowledge to members throughout

the organization (and not to a single individual or group within it), reflecting on it and using it as a basis on which to build planning and programming activities, to adapt and to cope with change. A learning organization is one that facilitates the learning of all its members and continuously transforms itself.

Organizational performance The ability of an organization to meet its goals and achieve its overall mission. Typical indicators for evaluating organizational performance are effectiveness, efficiency, relevance, and sustainability.

Organizational self-assessment The assessment of an organization by those who are working in the organization. As with any organizational assessment, a self-assessment focuses on overall impact and performance, or specific aspects thereof.

Organizational values Statements describing the principles the organization wants to express as it moves in the direction described in its goals. Values that an organization regards highly and holds as its ideal. Ethical standards that guide how work is done. Values can include such things as fairness, respect, commitment, and embracing diversity. Managers are expected to serve as role models for values.

Outcome An immediate effect or short-term consequence of an action. (See also ‘Output’ and ‘Impact’.)

Output The direct results of an intervention, a ‘deliverable’ for which management is responsible.

Ownership Right over, and responsibility for, a process or activity. When local players own a project, and they adopt it as their own even if outside organizations are involved.

Participatory evaluation A process of self-assessment, collective knowledge production, and cooperative action in which stakeholders in a development process participate substantially in the identification of evaluation issues, the formulation of evaluation questions, the design of the evaluation, the collection and analysis of data, and the actions taken as a result of the findings.

Partner The individual and/or organization with which one collaborates to achieve mutually agreed upon objectives.

Partnership Negotiated relationships that exist between two or more entities that have voluntarily entered into a legal or moral contract.

Physical resources The land, facilities, vehicles, and equipment used by organizations to carry out their activities.

Planning The process through which goals and objectives are set, partners identified, inputs determined, activities specified and scheduled, and monitoring and evaluation mechanisms defined, so that expected outputs and outcomes might be achieved in a timely manner.

Process management Management of resources and internal processes that support research and development programs. These include staffing, human resource development, fund raising, financial management, and management of facilities.

Process use of evaluation Individual changes in thinking and behavior, and program or organizational changes in procedures and cultures that occur among those involved in evaluation as a result of the learning that occurs during the evaluation process.

Program management Management concerned directly with the production and delivery of services for clients or target groups. Program management skills and procedures include project cycle management, program formulation, and technical reviews, for example.

Relevance Refers to importance and practical utility. In organizational assessment, it refers to the degree of congruence between (a) the objectives and activities of an organization, and (b) the needs and expectations of key stakeholders.

Result The output, outcome, or impact (intended or unintended, positive and/or negative) of a development intervention.

Review of documents Systematic review of an organization’s documents to obtain information for an evaluation.

Self-assessment (workshop) A workshop process organized to assess an organization’s needs, capacities, a capacity development initiative, or the organization’s performance, and involving the organization’s management and staff and perhaps external stakeholders.

Social capital The institutions, norms, relationships, and networks that enable collective action and shape the quantity and quality of a society’s social interactions.

‘Soft’ capacities The human and organizational capacities, or social capital of the organization, including such things as management knowledge and skills, and organizational systems and procedures (such as management information systems, and procedures for planning and evaluation.).

Stakeholders Any group within or outside an organization that has a direct or indirect stake in the organization’s performance or its evaluation. Stakeholders can be people who conduct, participate in, fund, or manage a program, or who may otherwise affect or be affected by decisions about the program or the evaluation.

Strategic management Development and implementation of effective strategies to set and achieve an organization’s objectives. Strategic management involves five sets of tasks, (a) developing a strategic vision and mission, (b) setting objectives, (c) crafting a strategy, (d) implementing the strategy, and (e) evaluating performance and initiating corrective adjustment.

Strategic planning A process by which a future vision is developed for an organization, taking into account its political and legal circumstances, its strengths and weaknesses, and the threats and opportunities facing it. It articulates the organization’s sense of mission and maps out future directions to be taken, given the organization’s current state and resources.

Strategy A planned course of action undertaken with the aim of achieving the goals and objectives of an organization. The overall strategy of an organization is often known as organizational strategy, but strategy may also be developed for any aspect of an organization’s activities, as, for example, environmental strategy or marketing strategy.

Sustainability The ability of an organization to secure and manage sufficient resources to enable it to fulfill its mission effectively and consistently over time without excessive dependence on a single funding source. Ideally, sustainable organizations have (a) the ability to scan the environment, adapt to it, and seize opportunities it offers, (b) strong leadership and management, (c) the ability to attract and retain qualified staff, (d) the ability to provide relevant benefits and services for maximum impact in communities, (e) the skills to demonstrate and communicate this impact to leverage further resources, (f) community support and involvement, and (g) commitment to building sustainable (not dependent) communities.

Survey The collection of data from a population for the purpose of analysis of a particular issue. In a ‘sample survey’, data is collected from a sample of the population.

Symbolic use of evaluation results Refers to situations where evaluation results are accepted on paper or in public pronouncements, but go no further. Many evaluations are symbolic in that they are carried out simply to comply with administrative directives or to present an image of ‘modernity’. (See also ‘Direct use of evaluation results’ and ‘Indirect use of evaluation results’.)

Terms of reference Written document presenting the purpose and scope of the evaluation, the methods to be used, the standard against which performance is to be assessed or analyses are to be conducted, the resources and time allocated, and reporting requirements. Two other expressions sometimes used with the same meaning are ‘scope of work’ and ‘evaluation mandate’.

Triangulation A process of using multiple data sources, data collection methods, evaluators, or theories to study an issue from different perspectives, validate research findings, help eliminate bias, and detect errors or anomalies in results.

Unit of analysis The class of elemental units that constitute the population and the units selected for measurement, or the class of elemental units to which measurements are generalized. In an evaluation of an organizational capacity

development effort, the unit of analysis might correspond to the individual, group, project team, department, network, partnership, or other organizational unit.

Utility The extent to which an evaluation informs relevant audiences and is beneficial for their work.

Utilization-focused evaluation Evaluation done for and with specific, intended primary users for specific, intended uses.

Validation The process by which the soundness of causal relationships or the generalization of findings are established.

Document(s) 13 of 15