Youth Participatory Evaluations: A Case for Qualitative Techniques!

Bakhtawar Khan EDRD*6000: Qualitative Methods/Analysis

March 2013

Outline 1.  What is Participatory Evaluation? 2.  Where Did it Come From? 3.  Principles 4.  Roles of Young People in Evaluation 5.  Steps in the Evaluation Process 6.  Methods 7.  Advantages 8.  Limitations 9.  2 Examples 10. Ethical Considerations 11. Conclusion 12. Food for Thought 13. References

What is Youth Participatory Evaluation? (Sabo Flores, K., 2008; Act for Youth Centre of Excellence, 2013)

Youth Participatory Evaluation (YPE) is an approach seeking to engage young people in evaluations of projects, programs, organizations, and systems serving them.

In this method, young people conduct research on issues, articulate experiences, and help in the development of knowledge about their respective communities.

The data collected is analyzed and findings are then shared and put to use.

Different models of YPE exist: some are completely youth driven; while, others include adults to varying degrees.

Where did it come from? (Act for Youth Centre of Excellence, 2013)

Having emerged over the past several years, YPE is a recent and increasingly visible concept.

As a field of inquiry, it has roots in Positive Youth Development and Participatory Evaluation methods and approaches.

Its theoretical foundation has been laid in the form of core principles (presented in next slide) and scholarly work exploring its impact.

Yet, it remains an uncommon practice.

Core Principles of Engaging Youth

Adults and youth collaboratively articulated this declaration at the Wingspread Symposium on Youth Participation in Community Research, 2002.

1.  Youth participation in community research and evaluation transforms its participants. It transforms their ways of knowing, their activities, and their program of work.

2.  Youth participation promotes youth empowerment. It recognizes their experience and expertise, and develops their organizational and community capacities.

3.  Youth participation builds reciprocal partnerships. It values the resources and assets of all age groups, and strengthens supportive relationships among youth and between youth and adults.

4.  Youth participation equalizes power relationships between youth and adults. It establishes common ground for them to overcome past inequities and collaborate as equals in institutions and decisions.

5.  Youth participation is an inclusive approach to diverse democratic leadership. It increases the involvement of diverse groups, especially those who are traditionally underserved and underrepresented.

6.  Youth participation actively engages young people in real and meaningful ways. It involves them in all stages, from defining the problem, to gathering and analyzing the information, to making decisions and taking action.

7.  Youth participation is an ongoing process, not a one-time event. Participants continuously clarify the purpose, reflect upon the process, and use the findings for action and change.

Roles of Youth in Evaluation (Checkoway, B. & Richards-Schuster, K. n.d.)

Research reveals that young people play the following roles in evaluations: youth as subjects, youth as consultants, youth as partners, and youth as directors.

As subjects, young people are observed, tested, and analyzed by adult evaluators.

As consultants, young people are asked for their feedback while adults lead the evaluation process.

As partners, young people assist in the information gathering process but do not possess any decision-making power.

As directors, young people take the lead and adults support the process.

Steps in the Evaluation Process (Adapted from Checkoway, B. & Richards-Schuster, K. n.d.)

1.    

2.    

3.    

4.    

5.    

6.    

Methods!Some of the commonly used sources of data in qualitative

evaluations are questionnaires, key informant interviews, focus groups, photo novella, participatory video, participatory mapping, media analysis, and oral narrative. As the example coming up will illustrate,

According to Bonati (2006), some creative techniques that can be used to collect information with children are: body map, evaluation wheel, impact diagram, flow charts, force field analysis, problem tree as well as drama and role-play.

Opportunities !(Bonati, G., 2006)

Listening to views of young people can help organizers incorporate their needs into design and implementation.

Participation can increase young people’s communication skills, self – esteem, self-confidence, and can encourage adults to share more power with children and youth

Young people can learn and increase knowledge, especially with respect to analytical and critical thinking, cooperation, negotiation and problem–solving.

Participation can improve young people’s status in their respective family and/or community.

Participation can serve as an opportunity for mutual learning between adults and youth.

Limitations (Bonati, G., 2006)

Establishing trust and open channels of communication can be difficult especially if the evaluations are short-term ventures.

Participatory evaluations are resource intensive. They require time, lots of people, and considerable financial resources.

There is a risk of youth participation becoming tokenism as involving youth has become somewhat of a “trend.”

Young people will need to be trained in order to participate in evaluation.

Things may not work out as planned and this can be disappointing/discouraging for the participants.

Example 1: Girl Programming in Guatemala (Soledad M., 2012)

In this video, Guatemalan girls, who received capacity building training from Insight Share, share their experience with using Participatory Video for an evaluation.

Most Significant Change is an evaluation technique that uses Oral Narrative. It can be coupled with other methods like participatory video as illustrated in this example.

http://www.videogirlsforchange.org/pvmsc-process-videos/

Example 2: Opening Doors (Mutchler et al., 2006)

Established in 2003, the Waterbury Youth Leadership Project is designed for inner city youth.

For the 3 initial years, the evaluation team tried to use quantitative techniques, i.e. standardized survey instruments, to measure impact and failed to capture the success of the program. Despite the supporting evidence, they knew that the program was successful because of the anecdotal responses of participants.

As such, a qualitative focus group was added to the evaluation methodology. Ten participants took part in the focus group. The content analysis of the transcript attested to the success of the program.

Without the use of qualitative techniques, the evaluators would not have been able to capture program success and give voice impassioned youth opinions like: “ Coming here is like coming home.”

Please read the full article by Mutchler et al. (2006) for more details.

Helpful Hints (Adapted from Mutchler et al., 2006)

Like most programming aimed for children and young people, evaluations involving young people should provide:

1)  A safe setting, 2)  Supportive relationships with adults and

peers, 3)  A variety of engaging and stimulating

activities, 4)  Skills training, and 5)  A sense of meaningful involvement.  

Ethical Considerations

Three ethical concerns: consent must be fully informed, research must do no harm, and confidentiality must be protected (Williams, 2006).

There are two dominate schools of thought with regards to obtaining consent when working with children and young people:

1) Proponents of conventional wisdom argue that if children under the age of 16 are interested in participating in research or evaluation, they need to obtain consent from a parent or guardian before involvement (Wiles et al., 2005).

2) Opponents argue that young people need to be able to speak for themselves (Williams, 2006).

Also, confidentiality  and  anonymity,  as they are understood in the social sciences, need to be clearly explained to young people before they become involved in evaluation. The explanation must include the circumstances in which the welfare of the young person overrides confidentiality obligations (Williams, 2006).

Conclusion The common thread between the two examples

and the other references cited in this presentation is the use of qualitative methods.

Since qualitative methods are able to capture individual experiences and they tend to be fun and interactive, it is recommended that they be used when youth are involved in evaluation.

Food for Thought (and Discussion)

1. How does the use of qualitative methods in evaluations with youth compare to the use of mixed methods? Which design do you prefer and why?

2. How would you present the findings of the evaluations to your stakeholders, including youth?

3. When publishing the findings of evaluations conducted with youth, will you include youth as co-authors?

4. Should young people be compensated for their participation in evaluations? If so, how?

References Arena, T. (2006). Splatter [Painting]. Retrieved from http://www.123rf.com/

photo_4728828_abstract-paint-splatter-elements-in-a-cmyk-color-scheme-this-vector-element-is-fully-editable.html

Bonati, G. (2006). Monitoring and Evaluation with Children. Plan Togo. Retrieved from http://plan-international.org/files/Africa/WARO/publications/monitoring.pdf

Checkoway, B. & Richards-Schuster, K. (n. d.). Participatory Evaluation with Young People. Retrieved from http://ssw.umich.edu/public/currentprojects/youthAndCommunity/pubs/youthbook.pdf

Mutchler, M.S., Anderson, S. A., Grillo, M., Mangle, H., & Grimshaw, M. (2006). Opening Doors: A Qualitative Evaluation of the Waterbury Youth Leadership Project. Journal of Extension, 44 (6).

Sabo Flores, K. (2008). Youth participatory evaluation: Strategies for engaging young people. San Francisco: John Wiley & Sons.

Wiles, R., Heath, S., Crow, G., & Charles, V. (2005) Informed consent in social research: a literature review. NRCM Methods Review Papers NRCM/001. Southampton: ESRC National Centre for Research Methods.

Williams, B. (2006). Meaningful Consent to Participate in Social Research on the Part of People under the Age of 18. Research Ethics, 2 (1).