Framing participatory evaluation

Citation: J. Bradley Cousins, Elizabeth Whittmore (1998) Framing participatory evaluation. New directions for evaluation (RSS)
DOI (original publisher): 10.1002/ev.1114
Semantic Scholar (metadata): 10.1002/ev.1114
Sci-Hub (fulltext): 10.1002/ev.1114
Internet Archive Scholar (search for fulltext): Framing participatory evaluation
Download: http://onlinelibrary.wiley.com/doi/10.1002/ev.1114/full
Tagged: Sociology (RSS) Participatory Research (RSS), Evaluation (RSS), Policy Evaluation (RSS), Popular Education (RSS)

Summary

Why do researchers involve research participants in collaborative inquiry? In this influential article, J. Bradley Cousins and Elizabeth Whitmore review the meanings of "participatory evaluation" (PE) as they were used from the 1940s through the mid 1990s. The authors identify two traditions: one that (a) includes people to improve the impact of research, and a second (b) that reallocates power for purposes of social change.

Summarizing Two Streams of Research

To start out, the authors offer a common summary of participatory evaluation: "when doing an evaluation, researchers, facilitators, or professional evaluators collaborate in some way with individuals, groups, or communities who have a decided stake in the program, development project, or other entity being evaluated" (5). These participants, or stakeholders are "those with a vested interest in the focus of the evaluation" (Mark & Shotland 1985).

In the Practical Participatory Evaluation (PPE) movement within the US and Canada starting in the 1970s, evaluators involve communities involved in decision-making, assuming that "stakeholder participation in evaluation will enhance evaluation relevance, ownership, and thus utilization" (6). Since the 1970s, researchers found that "utilization is often associated at least as much with the process of doing the evaluation as with the findings themselves." Many of these approaches define the research process in such a way that researchers do the technical work and stakeholders are involved in "definition of the evaluation problem, scope-setting activities, and, later, interpreting data emerging from the study" (7).

The Transformative Participatory Evaluation (TPE) movement "invokes participatory principles and actions in order to democratize social change," grounded in Latin American and post-colonial political contexts (Fals-Borda). These approaches offered "a reaction to positivist models of inquiry that were seen as exploitive and detached from urgent social and economic problems." (8) (Hall, 1992). The authors explain that "Although T-PE is now spreading to the university sector, it is deeply rooted in community and international development, adult education, and, more recently, the women's movement" (8). Influential theorists include Paolo Freire, Marx, Engels, Gramsci, Habermas, Adorno, and critical theory. Key concepts include:

• "the issue of who creates and controls the production of knowledge. One important aim of T-PE is to empower people through participation in the process of constructing and respecting their own knowledge and through their understanding of the connections among knowledge, power, and control" (8)
• "How is the evaluation conducted? The distance between researcher and researched is broken down; all participants are contributors working collectively" (8)
• "critical reflection requires participants to question, to doubt, and to consider a broad range of social factors, including their own biases and assumptions" (8)
• learning and education through the research process, not just investigation and action

T-PE processes are intended to "transform power relations and to promote social action and change" by thinking of research and evaluation "as a developmental process where, through the involvement of less powerful stakeholders in investigation, reflection, negotiation, decision making, and knowledge creations, individual participants and power dynamics in the sociocultural milieu are changed" (9).

The authors conclude that these approaches "differ in their primary functions–practical problem solving versus empowerment–and ideological and historical roots but overlap in their secondary functions and in other areas" (10).

Differentiating Process Dimensions of collaborative Inquiry

The authors advance three dimensions for making sense of a particular participatory evaluation project:

• "Control of the evaluation process, ranging from control of decisions being completely in the hands of the researcher to control being exerted entirely by practitioners. Control here relates particularly to technical decisions–those regarding evaluation processes and conduct–as opposed to decisions about whether and when to initiate evaluation" (10)
• "Stakeholder selection for participation, ranging from restriction to primary users to inclusion of all legitimate groups"
• "depth of participation, from consultation (with no decision-making control or responsibility) to deep participation (involvement in all aspects of an evaluation from design, data collection, analysis, and reporting to decisions about dissemination of results and use).

Next, the authors ask how the two approaches to participatory evaluation differ along these dimensions. They offer a table of 10 different kinds of participatory evaluation and compare them along these three categories. Overall, the "PPE" tradition tends to include people with existing power to implement policies, while the "TPE" tradition tends to include the people most affected. The paper also compares these approaches to other related models of evaluation, with summaries and references for each of the following:

• stakeholder-based evaluation
• school-based evaluation
• democratic evaluation
• developmental evaluation
• empowerment evaluation
• participatory action research
• emancipatory action research
• cooperative inquiry

Issues and Questions

Finally, the authors conclude with a set of questions of importance for all participatory research:

• "Power and its ramifications. Who really controls the evaluation? How does one account for and deal with variation in power and influence among participants and between participants and the evaluator?" (18)
• "Participant Selection. Who participates on the inquiry team, and how are participants identified and selected?"
• "Technical Quality. How is technical quality defined? By whom? Are there tensions related to data quality and the relevance of the evaluation to the local setting?"
• "Cross-Cultural Issues. how can cultural, language, or racial barriers be addressed?"
• "Training. how is the training of participants in evaluation and research methods to be accomplished?"
• "Conditions enabling PE. Finally, we ask, what conditions need to be in place for meaningful PE to flourish?"

The authors conclude by calling on participatory researchers to continue to share their findings in hopes of helping others develop more thoughtful answers to these questions in the future.

Notable References

(this article has four pages of great references, so you should look directly at the bibliography)

• Fals-Borda, O. (1987). The application of participatory action-research in Latin America. International sociology, 2(4), 329-347.
• Hall, B. L. (1981). Participatory research, popular knowledge and power: A personal reflection. Convergence, 14(3), 6.
• Hall, B. L. (1992). From margins to center? The development and purpose of participatory research. The American Sociologist, 23(4), 15-28.
• Huberman, M. (1995). The many modes of participatory evaluation. Participatory evaluation in education: Studies in evaluation use and organizational learning, 103-111.
• Mark, M. M., & Shotland, R. L. (1985). Stakeholder-based evaluation and value judgments. Evaluation Review, 9(5), 605-626.

Theoretical and Practical Relevance

This influential article summarizes multiple threads of participatory research and compares them across three criteria: who controls the evaluation process, who chooses the stakeholders, and how deep are communities' participation in the research process? The article focuses primarily on participatory evaluation aimed at influencing decisions and a parallel thread focused on emancipation and social change.