Reflections on participatory evaluation with young peopleare offered.

NEW DIRECTIONS FOR EVALUATION, no. 98, Summer 2003 © Wiley Periodicals, Inc. 87

7

Youth and Evaluation: EmpoweredSocial-Change Agents

David Fetterman

Kim Sabo opens this volume with a bold and accurate statement: “Partici-patory evaluation . . . is becoming ‘mainstream’” (Editor’s Notes). In fact,collaborative, participatory, and empowerment evaluation have all becomea part of the intellectual landscape in evaluation. Many markers support thisstatement in addition to the chapters in this volume, ranging from the NewDirections in Evaluation volume, Understanding and Practicing ParticipatoryEvaluation (Whitmore, 1998), to the Foundations of Empowerment Evaluation(Fetterman, 2001), and “Collaborative Evaluation in North America:Evaluators’ Self-Reported Opinions, Practices, and Consequences” (Cousins,Donohue, and Bloom, 1996). In addition, about one-fourth of the AmericanEvaluation Association is affiliated with the Collaborative, Participa-tory, and Empowerment Evaluation topical interest group (Susan Kistler,American Evaluation Association, personal communication, 2002). Thisvolume, however, takes a step beyond these landmarks by firmly plantingyouth participatory evaluation in the center of the evaluation discourse. Inaddition to contributing to the literature on capacity building and youthdevelopment, this volume contributes to the evaluation literature on uti-lization, particularly process use. The overarching theme that emerges fromthis collection is a shift from viewing youth as defective to that of assets incommunity development. An underlying theme is the shift from an exclu-sive focus on development to transformation as youth are stretched beyondtheir stage of development and in the process become the evaluation pro-fessionals they practice to be. I also agree with Sabo that this discussion hasthe “potential to invigorate evaluation practice.”

88 YOUTH PARTICIPATORY EVALUATION

Commentary on the Chapters

This collection begins with a description of youth participatory evaluationas a “field in the making.” This is an accurate description because this areais unfolding as you read this text. The editor presents the historical foun-dation of youth participatory evaluation by briefly discussing action researchand participatory evaluation, including practical and transformative partic-ipatory evaluation. In addition, Sabo discusses the parallel emergence andgrowth of positive youth development, shifting from a deficit model to anasset model “in which youth were understood as part of the solution.”Moreover, youth in this framework are viewed as community builders andempowered as social-change agents. This view works in tandem with a phi-losophy that posits that generalizable knowledge should be developed byand with community members. This natural confluence of historical devel-opments and shifts in consciousness and alliances helped generate youthparticipatory evaluation.

In Chapter One, Sabo discusses theories of development includingVygotsky’s “zone of proximal development.” Sabo’s focus throughout thischapter is on the concept of performance as developmental activity and akey factor in youth participatory evaluation. This is in alignment with theliterature on process use but extends the dimensions and boundaries to con-tributions to youth development itself. It also extends our evaluation under-standings about process use. It may not be enough to engage program staffmembers and participants in the evaluation. It may be time to allow themto “perform in roles that seemed to be far in advance of their development,”as Sabo explains in reference to youth. As described in this chapter, youthlearned many specific skills, including survey development and analysis,logic model development, program planning, focus group facilitation, andqualitative and quantitative data analysis. It may well be that this is wherethe real growth occurs and where the maximum potential for use lies, aspeople stretch and grow into their roles and “becom[e] who they were not.”

In Chapter Two, Les Voakes provides an example of how youth can bedirectly involved in planning and operational decision making concerningprograms that directly affect them. In this regard, Voakes describes ways inwhich youth were also involved in ongoing evaluations to improve programoperations. The summative component of the evaluation was also youthdriven. The success of the project, as evidenced by the youth-driven evalu-ations, led to program adoption in many communities.

One of the facilitating tools in this endeavor is partnering with adults whoshare the same ideals as youth. This is similar to conducting empowermentevaluation in settings that are philosophically aligned with empowerment; theyare more likely to succeed with less effort because everyone is on the samepage. Conferences, much like those found in cluster evaluations, were orga-nized to bring similar programs together so that participants could learn fromeach other. This was another powerful participatory evaluation tool in this

YOUTH AND EVALUATION: EMPOWERED SOCIAL-CHANGE AGENTS 89

project because it respected local knowledge. Once again, the evaluation prac-tice matched the philosophy of the program. Participatory research and eval-uation, as stated by the contributor, “emphasizes that the process is asimportant or more important than the result.” Process is often an attainableintermediate objective on one level of analysis. This evaluation combined pro-cess and outcomes. Bragging booths, flip charts, and graffiti walls representedadditional alternative evaluation tools.

In Chapter Three, Jonathan K. London, Kristen Zimmerman, and NancyErbstein discuss evaluation methods that link community and youth devel-opment practices. They focus on the efforts of Youth In Focus, “a nonprofitintermediary organization dedicated to youth empowerment through youth-led research, evaluation, and planning.” Youth In Focus youth are engagedin leadership opportunities in research and evaluation. These efforts aredesigned to foster community and youth development in a reciprocal ratherthan mutually exclusive relationship. Instead of developing youth with con-trolled activities, youth take an active role in real-world experiences. Thesereal-world experiences become the “learning laboratory” in a natural setting,with real circumstances and consequences. Youth participate in “ladders ofresponsibility.” They become agents of community change and, in the pro-cess, undergo training to be future leaders in the community. Concerningevaluation specifically, “[Youth-led Research, Evaluation, and Planning] goesbeyond this call by engaging youth in the documentation, research, andaction processes themselves and ultimately in the process of knowledge pro-duction that shapes these fields.” This is a form of process use. They aremore likely to take their experience seriously when it is real, as comparedwith an artificial experiment, in which their practice is manipulated for somepredetermined external outcome. The San Francisco Juvenile JusticeEvaluation Project provides a useful demonstration of this approach. It is acollaborative effort on many levels, including the participation of RisingYouth for Social Equity, Coleman Advocates for Children and Youth, theDelancey Street Foundation, the Center for the Study of Social Change, andYouth In Focus. Their work highlights the value of eliciting the “emic” orinsider’s view in an evaluation. For example, the youth familiar with thejuvenile justice system understood the issues, were well acquainted with the “local knowledge,” and were sensitive enough to pick up on the difficult-to-quantify but critical-to-include indicators such as “respect.” In addi-tion to illustrating the process by which youth are engaged in evaluation, theauthors also question basic assumptions. Youth assessing themselves maynot appear to be as “objective” as an outsider would be. However, that isbased on the assumption that an outside perspective is objective when it is not. Disenfranchised populations typically suffer from misinformed ste-reotypical views of the group or, in this case, youth as predators or as indi-viduals to be rescued or developed. That is the more accurate baseline to useto compare youth-led efforts. Youth are also closer to their own day-to-dayreality than outsiders in terms of real-world constraints and opportunities

90 YOUTH PARTICIPATORY EVALUATION

for change. Fundamentally, this chapter, drawing on a number of additionalexamples, including Youth IMPACT (a program of the San FranciscoDepartment of Children, Youth and their Families) and Serving Our Youthand Community (SOYAC), highlights how youth and evaluation transformprograms and city policies. It also provides an insight into how youth-ledevaluations transform youth.

In Chapter Four, Genevieve Lau, Nancy Netherland, and Mary L.Haywood provide insight into the concept of building an evaluative roadwhile running on it. The Youth Development Learning Network (YDLN)program “was intended to build evaluation processes into the practice ofsupporting youth development by modeling the process while conductingan examination of the goals of the YDLN.” Evaluators taught youth work-ers how to interview by having them interview each other. They used thosedata to “assess the effectiveness of workshop presentations and identifyeffects on participant learning.” This is a powerful illustration of experien-tial learning. It also highlights the reality of most self-evaluations, wheretraining and conducting the evaluation are coterminous—not preparatoryor prerequisite to the conduct of an evaluation. Similar to the practice inteaching hospitals, where students treat real patients (with guidance andsupervision) while they are learning how to treat them, youth workers learnto evaluate while evaluating (with guidance and supervision).

The contributors make an often-neglected but important observationabout this process: “Youth who participated in YDLN activities experienceda transforming change in their professional practice, in their attitude towardthe use of evaluation, and in an improved perception of themselves as pro-fessionals.” This is a common phenomenon in empowerment and partici-patory work. The act of doing becomes an act of becoming as people growinto their roles and responsibilities. This internalization, as manifested inthe concept or act of professionalization, is where real learning and trans-formation take place. The authors capture the essence of institutionaliza-tion in which evaluation becomes transparent and part of everyday life.They also explore the dimension of time in evaluation and development, asillustrated by the discussion about intermediate objectives as tools to accom-plish goals and the time required to evolve from passive consumers of eval-uation to active participants who use evaluation to improve their world. Thecontributors conclude with sound fiscal advice: the concept of youth eval-uating youth programs is economically sound.

In Chapter Five, “Using Participatory Methods to Further the DemocraticGoals of Children’s Organizations,” Roger A. Hart and Jasmine Rajbhandaryprovide useful tools and approaches to successfully involving young childrenin Nepal in participatory evaluations. Save the Children Norway and Save theChildren USA clubs represent natural experiments in which children areencouraged to develop their own programs and assess them largely by them-selves. In this chapter, the authors emphasize classic elements of participa-tory and empowerment evaluations: simple methods, data collection and

YOUTH AND EVALUATION: EMPOWERED SOCIAL-CHANGE AGENTS 91

analysis carried out by or with group members, and little time. Social map-ping (with yarn, cardboard, and stones or circles) or card sorting are excel-lent tools to demonstrate what can be done with young children with fewresources and little time. These tools helped identify patterns of communityinclusiveness or distance, as well as decision-making responsibilities. In addi-tion, matrices and skits were effective data-collection tools when workingwith young children. The use of dots to help children rank their preferencesresembled the dot prioritization exercises used in many empowerment eval-uations with adults (see Fetterman, 2001, pp. 24–27). The contributors pro-vide a critical analysis of these tools, informing readers of when the tools weretoo complex, distracting, or time-consuming, including problems with theuse of Venn diagrams by very young children. The contributors helped the children see their own patterns of decision making. However, they wereless confident about their ability to help children incorporate what they hadlearned in their daily life. This may be because they assumed more of the data-collection and analysis responsibilities than desired while working with theseparticular children.

In Chapter Six, Bonny Gildin describes how youth can and do conductevaluations with effective results. The All Stars Talent Show Network isunique in both the grassroots-funding model adopted and its “radical inclu-siveness”—if you audition, you make it into the show. It also recognizes thevalue of performance as a developmental activity, much the same as evalu-ation is a developmental activity (for youth and adults). This philosophicalprogram base of All Stars lends itself more readily to empowerment and par-ticipatory evaluation approaches. This talent show’s self-assessment is sim-ilar to many empowerment evaluations in which funders are invited toparticipate in the evaluation process and simultaneously the building of theprogram. The youth presented in this chapter were innovative and effective,using “Back to School report cards” or surveys. The findings resulted in“immediate changes to the program.” In this chapter, youth, funders, andadult volunteers are united in program development and evaluation. Onceagain, this parallels participatory and empowerment approaches by creat-ing communities of practice in which as many relevant stakeholders as pos-sible are brought together to solve real-world problems.

Conclusion

This collection covers a wide range of evaluation issues and concerns. Itplaces youth at the center of the discussion. It makes a compelling argu-ment for their role in conducting evaluations. The question remaining,however, is, “Will youth participatory evaluation shape the field of evalua-tion?” Building on Goodyear’s insights (cited in Youth Participation inCommunity Research and Evaluation, 2002), Will it demonstrate suffi-ciently how evaluation can contribute to democratic dialogue? Will it shiftour attention from the process-use question of how to engage youth in

92 YOUTH PARTICIPATORY EVALUATION

knowledge production to the development of democratic citizens’ questionof how to produce environments that support ongoing growth and change?Will it bring humanity back into evaluation practice? These are a few of thequestions this volume poses for us all. It forces us to stretch beyond wherewe are comfortable, and that is where real learning takes place.

References

Cousins, J. B., Donohue, J. J., and Bloom, G. A. “Collaborative Evaluation in NorthAmerica: Evaluators’ Self-Reported Opinions, Practices, and Consequences.”Evaluation Practice, 1996, 17(3), 211.

Fetterman, D. M. Foundations of Empowerment Evaluation. Thousand Oaks, Calif.: Sage,2001.

Whitmore, E. (ed.). Understanding and Practicing Participatory Evaluation. NewDirections for Evaluation, no. 80. San Francisco: Jossey-Bass, 1998.

Youth Participation in Community Research and Evaluation. Symposium Proceedings.Wingspread, Wisc., 2002.

DAVID FETTERMAN is consulting professor in the School of Education and direc-tor of the MA Policy Analysis and Evaluation, Stanford University, Stanford,California.