Listening to the Experts

A uniquely organized conference served as a tool forparticipatory evaluation. Positive youth developmentpractices were empowering, and youth and adults workedas democratic equals in achieving common goals.

NEW DIRECTIONS FOR EVALUATION, no. 98, Summer 2003 © Wiley Periodicals, Inc. 25

2

Listening to the Experts

Les Voakes

In March 2000, over 150 youth and adults gathered for a weekend in asmall town near Ottawa, Ontario, for a conference organized by the TownYouth Participation Strategies (TYPS) project, funded by Health Canadaand sponsored by TriCounty Addiction Services. The TYPS project(Voakes, 1998, 2001) recognized that youth possess perspectives, knowl-edge, and experience different from adults but equally valuable. Althoughseparated by socioeconomic and geographical differences, youth also sharea connectedness created by common adolescent experiences, sociallyimposed expectations and limitations, media-constructed identities, andan awareness of their future life challenges. The core method behind TYPSwas to have youth directly involved in the planning and operational deci-sion making within the institutions that affect their lives and to ensure thatyouth have important roles in their respective communities to improvetheir futures. TYPS found that youths’ involvement was most effectivewhen they partnered with adults who share the ideal of having youths’opinions and voices represented in all aspects of their community. Youthcenters, as places for youth to socially interact in loosely structured pro-grams or drop-in situations, had proved to be the most immediate and suc-cessful means for initiating new partnerships between youths and theircommunities.

The TYPS project began as a local initiative to assist five small townsin eastern Ontario develop youth programs and centers and have youth asactive partners in those operations. To ensure that all activities were par-ticipatory, from the youth and community perspective, ongoing evalua-tive measures were incorporated from the start. Several additionalmechanisms were later introduced by the participants, including sign-in

26 YOUTH PARTICIPATORY EVALUATION

logs recording all participants by name, sex, and age; board compositionof youth center management (youth-to-adult ratio), with attendancerecorded within the minutes; the use of interactive scrapbooks encourag-ing youth and adult contributions to both record and evaluate each activ-ity or event; youth inclusion on committees within the community; localmedia reports on the youth center; and whether or not a youth centerremained open and operating. Upon completion of the project, partici-pants evaluated the TYPS model and their youth centers’ successes,thereby contributing to the final report and allowing other communitiesto benefit from the lessons learned.

Following the success of the TYPS project, a series of weekend work-shops was funded by Health Canada as a response to requests from numer-ous communities that had heard about it. The project coordinator organizedand facilitated a dozen weekend workshops across the province of Ontario,with eight to ten towns taking part at each location. The workshops offeredto adult and youth participants the lessons learned through the TYPS proj-ect in an interactive process that included exercises and resources in lead-ership development, developing policy and procedures manuals to operatea youth center, a model for youth and adult board management, and sus-tainable approaches for financing and supporting youth center operations.In all, over one hundred communities participated, and youth centersquickly expanded through an evolving network of grassroots support fromcommunities across Ontario and to other areas of Canada.

An evaluation of the TYPS approach became increasingly importantas the model was adopted by more communities (all or in part) as a basisof operating their local youth centers. When TYPS originally started, theproject was designed to use a participatory approach with only the fivetowns involved. That project had been deemed successful because themain goals were achieved: youth centers were opened, and youth weredirectly involved in their operations. Although the project had concludedwith a participatory evaluation process and had delivered exceptionallypositive evaluations, it was based on a small number of youth centers andparticipants.

On a recommendation from past TYPS participants, a large confer-ence was organized to bring together groups involved in the provincialworkshops to share their experiences and evaluate the TYPS project. Theevaluation was to use a process as dynamic and evolving as that of the ori-ginal TYPS project, incorporating several ways for participants to expressand evaluate aspects of the TYPS model and their own youth center’sdevelopment.

Most delegates to the conference had been participants in one of theTYPS workshops or had otherwise received TYPS materials and resources.Although many of the groups knew of each other, only a few had previouslymet or worked together, and TYPS had been their only mutual link.

LISTENING TO THE EXPERTS 27

Preparation Before the Conference

Following a method and format developed during the five years of TYPSworkshops, this conference planned to address a variety of issues and top-ics identified by youth and staff from the youth centers. The learningmethod would include youth and adults acting as partners in the sharingand exchanging of information. A group of youth, some of whom had beeninvolved with their local youth centers, agreed to be part of the conferenceplanning and coordination. Months before the conference, TYPS staff con-tacted youth and adult participants from past workshops and asked twoquestions: what topics did they want discussed or presented in workshopsat the conference? and was there a topic that they wanted to present? Fromthe responses, presenters were selected, and organizations with specialinterest in youth issues, preferably having youth presenters, were contactedand invited.

Before the conference, delegates prepared themselves for a rapid infor-mation exchange by constructing displays, updating materials about theirprograms, and planning their presentations using methods (discussed indetail below) previously learned during the TYPS workshops. Conferenceorganizers (a team of youth volunteers and the project coordinator) endeav-ored to create a milieu that would encourage youth and adults to expresstheir ideas and be self-examining of their own and each other’s programsand operations. In keeping with the commitment to youth being givenvoice, groups were required to bring three youth for each adult delegate.

The Conference: Processes Within Processes

The conference officially began Friday evening with an open social eventthat included several icebreakers (dubbed the “TYPS Olympics”), livemusic, an open mike, dancing, and lots of pizza. The evening was relaxedand fun, with networking already starting between groups through discus-sions on how their respective youth centers operated, the activities that werethe most successful, and so forth. Conference organizers encouraged con-ference delegates to write on several flip charts and graffiti walls posted withquestions posed for the delegates. Some of the questions were as much foramusement as information: “How far did you travel to get here?” or “Thefunniest thing that happened to me on the way here was. . . .” Responseswere later recorded and put into the final report on the conference.

The next morning, all delegates started an active day of listening tospeakers, engaging in discussions, and attending workshops. Most speak-ers were youth, although adult presenters also shared their expertise on sub-jects from the perspective of researchers and community developers.Throughout the day, new flip charts and graffiti walls appeared, invitingresponses to new questions or opinions from both the youth and adults.


The Saturday evening itinerary included tours (by bus) to three localyouth centers while en route to Ottawa. The youth center tours took abouttwo hours, and eventually everyone arrived for a large Chinese restaurantbuffet, followed by an evening at a multiplex movie theater. Buses arrivedback at the conference hotel about midnight.

On Sunday, the last day of the conference, all delegates used the WorldHealth Organization’s Modified Social Stress Model (MSSM) as a framework,to be explained later, for a detailed participatory evaluation process. Thegoal of this last exercise was to determine the value and effectiveness of the TYPS model and youth centers as a whole.

Evaluation Processes and Results

The following section describes four specific methods used to evaluate theTYPS model and youth centers within the conference setting.

Flip Chart and Graffiti Walls. As an evaluative tool, comments onthe flip charts and graffiti walls provided insight and texture for the entireprocess of understanding the participants, their programs, and their moti-vations. This is a direct, nonmediated way to share experiences and to doc-ument perceived effects of the TYPS model. Everyone was encouraged tocomment, with youth asked to use one color of marker and adults anothercolor. In this way, the results could be seen as equally valid but with a clearrecognition of the youths’ input. The following are a few examples of theyouths’ (Y) and adults’ (A) comments.

I came to this conference because . . .Our youth, like all youth, need to have a place to call their own. (A)I want to know more about everything. (Y)Recommended that I attend by a teacher, due to my career goals. (Y)I am also interested in learning new things and take advantage of ALL my

opportunities. (Y)I want to help my friends who are having trouble, and I thought this would

help me learn how to help them. (Y)

My community needs a youth centre because . . .I need it!(Y)There are no shelters for youth in our city—but there is great need. (Y)Youth need to feel valued and not seen and judged by the behavior of

others. (Y)If you have nowhere to go, they will turn to drugs and alcohol and end up

getting in a lot of trouble. (Y)

Youths Need. . . . Acceptance. (Y)Someone to trust!(Y)


Nonjudgmental environment. (Y)To be heard. (Y)To be trusted. (A)To feel safe and like someone cares. (Y)To feel a part of their community. (A)

Our most successful activity has been. . . .Haunted house. (Y)Sky diving. (Y)Tournaments. (Y)Video dances. (Y)Our presentation to town council. (Y)Slam dunk (basketball). (Y)Tree planting. (Y, A)Day camp. (Y, A)Town parade. (Y, A)Opening our youth centre. (Y, A)

Bragging Booths. These booths provided a wealth of information.Each center selected materials that exemplified their best practices from theirperspectives. The bragging booths were put together primarily by the youthdelegates with input from their friends and youth center volunteers. Theprocess of creating the booths provided youth with time for self-reflectionand for appreciating what they had done and learned. The booths remainedup and available throughout the conference so that everyone could discusseach other’s best activities and take notes, collect information, and becomeinspired with new ideas.

The designers of each bragging booth used a variety of media to con-vey their information and knowledge. There were photographs of eventswith information sheets to explain the activities and results, booklets ontheir group’s programs, computer diskettes for distribution with programinformation and templates, homemade videos about their activities, T-shirtsand caps with their youth center’s name and slogan, successful fundraisingcampaigns and products, activity scrapbooks, and many newspaper articlesproudly displayed.

Workshops. Workshops were facilitated through a team of youth whoserved as workshop hosts, recorders of the presentation, and collectors ofparticipants’ comments. Many workshop presenters were youth from vari-ous organizations and youth centers. Workshop topics included best prac-tices in establishing youth centers, how to apply to write funding proposals,program development, youth and adult volunteer training, youth center pol-icies (for example, how to address tobacco, alcohol, and other drug use),and helping youth manage personal issues (for example, family breakup,depression, sexuality, and other life-affecting concerns). All presentersbrought information to share with the participants, and all workshops were


delivered in an interactive format. Evaluation comments on these work-shops indicated that participants found the format and information sharedto be of direct value to their youth center operations, and most participantsplanned to incorporate new ideas into their programs. The youth host orrecorder at each workshop provided reports, collected the facilitator’sshared information, and occasionally added comments from participantsabout the workshop. The youth host or recorders all met after the confer-ence to debrief, to share ideas for improving future workshops, and to orga-nize a “conference results report.”

Modified Social Stress Model. The MSSM was developed by theWorld Health Organization as an approach to train volunteers and streeteducators in understanding substance abuse by individual youth within alarger social context. It was also designed to be an evaluating tool to assessthe effectiveness of programs designed for youth involved in at-risk behav-iors. The MSSM had been a cornerstone in the TYPS workshop series as amodel that youth centers could refer to and use to assess their activities. TheMSSM considers how a given activity, program, or policy will affect youththrough three risk factors (stress, normalization, and experiences with a riskbehavior) and three protective factors (attachment, skills, and resources).Each factor is divided into two considerations (positive and negative) of theeffects the program or activity will create.

The conference process of using the MSSM was led and recorded byyouth. The final results were later reported to the entire conference by theyouth recorders. The adults at the conference then contributed their ownobservations and views. The first part of the review required two hours, withgroups of participants each tackling a specific factor of the model and con-sidering it as it applied to youth centers following the TYPS model. Youthrecorders kept all information on large flip-chart pages, and when every-one reconvened in the large conference room, the pages were taped to thewall to create a giant fraction equation (risk factors as the numerators, andprotective factors as the denominators) that assessed total level of risk toyouth. The points recorded demonstrated that the TYPS model of youthcenter development and management positively addressed all factors in theMSSM model and, therefore, that the level of risk to youth involved withthe youth centers was lower than to those uninvolved. The responses notedthat there were program components at the youth centers that addressedrisk factors of youth: coping with stresses, the de-normalizing of high-riskbehaviors (that is, drugs, unsafe sex, crime), and positively challenging neg-ative peer pressures. The responses demonstrated that the “protective fac-tors” were also being supported through improved attachments to the youthcenters, their staff, and volunteers; a long list of supportive resources; andan equally long list of improved skills derived through regular attendanceand involvement. Indicators and processes were also identified of where andhow the TYPS model could be improved on and which indicators could bestmonitor effectiveness of both youth centers and TYPS. In all responses,

there was an overlying theme that each positive aspect was enhanced by thepartnering of youth in developing and managing the programs and youthcenters.

After each group presented their findings to the entire conference, thediscussion was opened for delegates (youth and adult) to add their per-spectives and suggestions. In the end, the entire wall of a conference ball-room was covered with inputs, outcomes, assessment indicators, actualresults, and identified deficits. The final result was both an evaluation pro-cess and a report that was recognized by all delegates as representative ofeveryone’s perspective and that provided a blueprint on what had beenworking well and what needed to be done.

Other Indicators. One means of evaluating the effectiveness and sup-port of a model is to observe the enthusiasm and willingness of the partici-pants to stay involved. The interest and commitment to attend the TYPSconference itself was regarded as an evaluation of the model. The followingare samples of indicators worth noting:

There were 150 delegates, plus presenters, in attendance.Eighty-one of the delegates were youth.Delegates traveled from across the province, at their own expense, to attend

(over ten thousand kilometers were traveled by the delegates).Forty-six communities were represented at the conference.Delegates represented diverse cultural backgrounds and geographical areas

(native, francophone, far north, industrial or agricultural communities,and the like).

All workshop presenters attended either at no fee or requiring only a stipendto cover expenses.

Funding to support the overall conference expenses was contributed by allthree levels of government.

In-kind support of the conference was conservatively estimated to be val-ued at over $78,000.

Two universities and one government department specifically requestedpermission to attend.

The news media provided extensive coverage and interest.

Conclusion

Participatory research and evaluation emphasize that the process is asimportant as or more important than the results (Hart, 1997; Reason, 1993;Whitmore, 1991). The participatory evaluation process must involve all par-ticipants in a process that empowers them to identify past progress, currentcommon problems, and planned future actions while recognizing theknowledge and expertise they possess and can contribute for their own useand to aid others. Throughout the TYPS evaluation process, youth andadults, with the youth taking the lead, revealed what had worked best for


them within the context of the TYPS model. Through the collective reviewof all their comments, best practices emerged, an evaluation of the TYPSmodel was completed, new information was collected and distributed, therewas a sharing of capacity-building tools and resources, a new evaluativeformat was agreed on for future evaluations, each group experienced oppor-tunities to be reflective about their own progress and future needs (con-sciousness raising), and an overall plan was developed for the future of theTYPS model and a youth center network.

The TYPS conference evaluation process was respectful and attentiveto all points of view from both youth and adults; youth initiated and youthled wherever possible; informative and offering learning from individual andgroup perspectives; an evaluative and reflective “time-out” for each youthcenter and for the entire TYPS model; a consensus-building process ofanalysis and understanding; nonlinear in its willingness to accept ideas andviews throughout the weekend in a variety of modes of expression; an affir-mation to the youth involved that funders and local communities were valu-ing youth opinions and involvement; a time for all, especially the youth, tounderstand that they were part of a larger network and grassroots move-ment; and an approach that was exciting and fun for everyone involved,regardless of age.

At the end of the conference, the weekend was assessed by everyoneinvolved (from participants to funders) to have been a success. The unani-mous consensus was that TYPS should have these conferences as annualevents, which was recognized in itself as an evaluative statement.

References

Hart, R. Children’s Participation: The Theory and Practice of Involving Young Citizens inCommunity Development and Environmental Care. London: Earthscan and UNICEF,1997.

Reason, P. “Sitting Between Appreciation and Disappointment: A Critique of the SpecialEdition of Human Relations on Action Research.” Human Relations, 1993, 46(10),1253–1270.

Voakes, L. Town Youth Participation Project Final Report. Smiths Falls, Ontario:TriCounty Addiction Services, 1998.

Voakes, L. “Town Youth Participation Strategies Project: Applying Participatory ActionResearch in Small Town Canada.” Unpublished master of arts thesis, Department ofSociology and Anthropology, Carleton University, Ottawa, Ontario, 2001.

Whitmore, E. “Evaluation and Empowerment: It’s the Process That Counts.”Empowerment and Family Support, 1991, 2(2), 1–7.

LES VOAKES is a consultant and serves as project coordinator for Town YouthParticipation Strategies, part of TriCounty Addiction Services, Smiths Falls,Ontario, Canada.


Documents

Listening to the Experts