This clearly written and presented guide originated out of a workshop held in 1992 and attended by 55 NGOs which receive ODA funding. Illustrative examples are drawn from many of their presentations and there is a strong emphasis on participatory examples.
The use of RAP in evaluation in UNICEF is briefly outlined in this paper. In UNICEF, a Rapid Assessment Procedure [RAP] exercise "includes as a central base, the use of techniques derived from the field of anthropology such as focus group discussions, observation, and unstructured one-on-one interviews, that form a body of mainly qualitative data". Normally the qualitative information derived from RAP will be used in conjunction with quantitative data, which might already be available, or which might be collected as part of a RAP investigation itself. Although the report includes an admission that "RAP is not a particularly useful type of method to use for evaluations of a participatory nature..", the author does point out that RAP evaluations do have an important role in encouraging participatory development processes in projects, and that RAP style evaluation "can be used, if so desired, as a way of instilling a sense that more peoples participation may be more desirable in a development project".
Download available
This short report provides some useful comments on a two week training course on the use of PRA for PM&E conducted by the Development Support Programme Khartoum, part of the Community Development Services in Cairo. It took place over two weeks in Kordofan and involved fieldwork in villages, with varying involvement in SOS Sahel's Natural Forest Management Project, near to El Obeid. Among the problems that emerged from this course, were; 1. the difficulties in encouraging a "suitable PRA attitude" amongst course participants in their approach to the villagers; 2. A lack of confidence amongst the participants; 3. Problems with the use of symbols during the fieldwork. Although participants were slow to grasp the usefulness of PRA for PM&E, three "tools" were found to be most useful for evaluation, namely; Impact diagrams, Impact matrices and Evaluation Matrices. The later was developed from the Innovation Matrix in the SCF/IIED manual; PRA for community development (1991), and its purpose was to generate discussion about the advantages and disadvantages of forestry activities.
Two methods are used to elicit the participation of rural people ("stakeholders") in the monitoring and evaluation of development programs: the Stakeholder based evaluation [SBE], and the participatory assessment monitoring and evaluation [PAME]. The difference is in the degree and stage of the involvement of stakeholders. The SBE pertains to a larger perspective where all possible stakeholders are identified and involved; the PAME involves mostly the direct and indirect beneficiaries of a program. The advantages and limitations of the stakeholder approach are discussed as well as the concept, the benefits and the focus of PAME [see also Davis-Case (1990; 1989 in this bibliography]. Although the use of PRA methodologies is not discussed, the article systematically explores the same set of questions, concerning the relative merits of certain approaches (or tools) to Participatory M&E, that are likely to be raised in the PRA literature.
The editorial introduces the articles in this very useful Rural Extension Bulletin, and puts forward the view that there is "increasing evidence that systematic monitoring and evaluation may offer very real benefits to project management". The key to the emergence of an effective and practical methodology for 'extension M & E' lies in the creation of an coherent "management information system" which is not only focused on issues of long term planning, but also on the "decision making process of project evaluation. [Other articles from this journal reviewed in this bibliography include; Roche (1993). Mali: Auto evaluation - an NGO experience with community based evaluation; Shah et. al. (1993). Gujerat, India: Participatory Monitoring and Evaluation]
In this article the author describes the experience of an international NGO [ACCORD] in its attempt to develop, with community groups, a process known as "auto-evaluation" - which aims to establish project objectives that are consistent with both community and agency criteria, and to evaluate their achievement against "community-determined indicators". While PRA is not explicitly mentioned, the discussion concerning the practical lessons that have been learnt to date with the implementation of this type of participatory or community-based evaluation mechanism, is highly relevant.
Beneficiary Assessment [BA] is described as "tool for mangers who wish to improve the quality of development organisations". In this World Bank manual the author gives the rationale for BA, describes its design and comments on the techniques (direct observation, conversational interviews, and participant observation) which are employed. Although PRA is not mentioned, the discussion may be of interest, due to these latter comments - which are further elaborated in the annexes. Examples of the impact of BAs are provided and the question concerning the limited prevalence of participatory evaluation techniques, such as BAs, in Bank projects is briefly discussed.
The study examines the Farmer Participatory Research approach to technology generation for resource poor farmers, and how it compares to other approaches to agricultural technology generation. The Farmers Innovation and Technology Testing [FITT] is a farmer participatory evaluation method. Through FITT the Gambian Department of Agricultural Research has attempted to establish a 'technology link' with farmers groups that have been 'set-up' by NGOs. The core of this study is an evaluation of the FITT programme using PRAs in two villages. Although the report is perhaps too large for mailing as a whole, it does contain some extensive discussion of these 'PRA based evaluations': Chapter 4 covers the PRA methods employed, which included wealth ranking exercises, preference ranking of crops and the drawing of seasonal calendars. Chapter 7 provides some useful discussion concerning some of the limitations and problems encountered in this PRA exercise. Finally the appendices include some detailed description of the PRA exercises mentioned above.
The International Conference on Rapid Assessment Methodologies for Planning and Evaluating Health Related Programmes, was organised by the United Nations University, at the Pan American Health Organisation headquarters in 1990, to "explore anthropologically based methodologies for the design, evaluation, and improvement of programmes of nutrition and primary health care". The 42 chapters on Rapid Assessment Procedures [RAP], Rapid Rural Appraisal [RRA], and related approaches deal with a range of research tools. The book is divided in to seven sections: 1. The expanding role of Qualitative research in International Development; 2. Development and applications of RAP procedures in Africa, Asia and the Americas; 3. Community Participation and RRA; 4. Institutionalization of RAP; 5. Training for RAP and other qualitative methods; 6. Bringing RAP to the decision-making realm: effective communication and use; 7. Conference summary, comments, speakers and participants. The overall focus is on planning and evaluation of nutrition and health related intervention programs, but much that is discussed will have a direct bearing on other social development sectors. The stated aim of the volume, of contributing "toward increasing the understanding of RAP and RRA, both as tools of investigation and potentially as integrated components of the community development process itself", highlights this point.
The Aga Khan Rural Support Program (AKRSP) promotes community participation in natural resource management. Here the monitoring process that comprised part of the general 'Participatory Rural Appraisal, Planning and Evaluation' (PRAPE) used in watershed management programs by the AKRSP in rural Gujarat is described. The participatory monitoring sequence involved: 1. Discussion with each farmer in his field. 2. Decisions as to the variables to be monitored with the farmer groups. 3. Ground mapping of base line and impact maps. 4. Paper mapping, mainly through symbols. 5. Presentation of these findings to the watershed outlet groups. 6. Aggregation of the information collected, followed by preparation of aggregate maps. 7. Presentation of the findings to the village community. 8. And finally, the "generation of technology domains and adaptation to village circumstances" The major strengths and advantages of the "dynamic process" of participatory monitoring in projects concerned with natural resource management include; greater clarification of indicators which might otherwise be difficult to measure; the lowering of costs of development programmes in the villages; and eventually (it is hoped) the full internalisation of these methodologies within the village communities. A valuable, but brief, summary of an effective participatory monitoring process using PRA techniques in the field.
The metaphor of the "mirror", constitutes the central theme for this guide to self evaluation [SE] prepared by the evaluation service of the Swiss Agency for Development and Cooperation [SDC]. The "mirror of SE" refers to "a multitude of tools and methods which provide a critical and constructive analysis of our own activities and their consequences". Unlike external evaluation, a SE is always designed to "illuminate" one's own area of responsibility to help find possible improvements. 18 practical examples of SE are introduced in chapter 2, which are divided into three main groups; Externally initiated SE; SE involving partners; and Autonomous SE - which occurs entirely independently of outside influence (the key case study given in the latter group is the SE deployed in the Federation of Senegalese NGOs in Senegal, 1989). In chapters 3 and 4 respectively, an "analysis" and "valuation" of these case studies is carried out, with 8 "fundamental questions" providing the framework for discussion [questions include; "what is being evaluated?", "How is success being measured?", "who are the participants?", "how can SE be implemented?"] In chapter 5 the analytical concept of the "wheel with 8 spokes" is introduced as a specific approach through which SE can be conceptualised and put into practice. Although no mention of PRA is made, this guide does provide some relevant and stimulating discussion which is based around the large number of case studies. Certainly, the 56 arguments - or "excuses" - listed on p.32, which "are used to evade an SE" are equally likely to be employed with reference to the use of PRA methods for M&E. (61 pp)
Although PRA is not mentioned, this report - which is illustrated with reference to a wide range of reports of mainly FAO projects - provides a very useful summary of; the benefits of this type of PE; the obstacles and difficulties likely during its introduction; ways in which it can be implemented in different projects; and how donors can best support the effective use of PE. [See also Uphoff (1989)].
This 'module' was formulated by ITAD and furnished to all World Food programme [WFP] staff. It describes several applications of Beneficiary Contact Monitoring [BCM] which can be used by the WFP to "improve monitoring by contacting the beneficiaries in person and seeking their views about project activities". The stated aims of the module are; 1. to explain the applications of BCM, 2. To explain the techniques for individual and group interviews and, 3. To illustrate where these applications would be most suitable. PRA methods are not mentioned, although RRAs are discussed in some depth. It is also noticeable that no mention is made of women or those 'beneficiaries' that do not come to the "group interviewing sessions". Despite these omissions, there are some useful points in the module, especially with reference to how to formulate questions so as to ensure unbiased information is obtained.
In this paper, some history of and reasons for the PIDOW project in India are given. It provides an eco-history of the region in which the project started, describing how the natural resources disappeared over a period of decades. The early stages of the project and the approaches used to initiate activities are dealt with, as are ways in which strategies had to be changed. For those who now know the PIDOW project as a successful example of participatory watershed management, the paper provides interesting insights into the project's evolution.
Download available
The author had to field test alley cropping in Nigeria with limited-resource farmers. There was much resistance to a concept which did not translate well into any local language, thereby creating confusion. Alley cropping also promoted tree planting when local practice required clearing them; women were especially reluctant to plant trees on their land as it could lead to repatriation of their land by their men. Together with several villagers and teachers, the author came up with a play about the 'Fertiliser Bush' which could be performed by community members. The fertilizer idea was locally well- understood and the use of drama was traditionally respected and enjoyed.