Summer Youth Employment Program Evaluation … Youth Employment Program Evaluation Report Summer 2011 ... youth participants have a structured and well-supervised opportunity

Summer Youth Employment Program

Evaluation Report

Summer 2011

Department of Employment Services

Government of the District of Columbia

Prepared by:

Nisha Sachdev, MPH, DrPh(c)

George Washington University

DC Children and Youth Investment Trust Corporation

Supervised by:

Karen McDonnell, Ph.D.

Associate Professor, Prevention and Community Health

The George Washington University School of Public Health and Health Services

Summer Youth Employment Program Evaluation Report

Fiscal Year 2011, Summer 2011 1

TABLE OF CONTENTS

Executive Summary .......................................................................................................................4

Introduction ....................................................................................................................................6

Background of Youth Needs ........................................................................................................6

Significance of Youth Employment Programs .............................................................................7

Positive Youth Development Framework ..................................................................................12

Purpose of the Evaluation Report ...............................................................................................15

District of Columbia Summer Youth Employment Program ..................................................16

History of Youth Employment and Training Programs .............................................................15

History of DC Summer Youth Employment Program ...............................................................17

Key Stakeholders ........................................................................................................................18

2011 Summer Youth Employment Program Design ..................................................................18

Goals of 2011 Summer Youth Employment Program ...............................................................20

Youth Eligibility .........................................................................................................................21

Youth Application and Selection Process ..................................................................................21

Host Agency Application and Selection Process .......................................................................23

Additional Rules of DC Summer Youth Employment Program Participation ..........................25

Overview of Methods of Evaluation ...........................................................................................26

Quantitative Methods .................................................................................................................26

Qualitative Methods ...................................................................................................................28

Ethical Considerations ................................................................................................................29

Findings.. .......................................................................................................................................30

Characteristics of Participants ....................................................................................................30

Characteristics of Job Placement and Supervisors .....................................................................35

Youth and Employer Satisfaction ..............................................................................................38

Youth and Employer Job Responsibility and Support ...............................................................41

Job Readiness and Work Skills ..................................................................................................43

Limitations .................................................................................................................................46

Conclusion ....................................................................................................................................47

Next Steps ..................................................................................................................................47

Recommendations ......................................................................................................................47

Conclusion ..................................................................................................................................49

Definition of Terms ......................................................................................................................50

References .....................................................................................................................................51

Appendices ....................................................................................................................................55

Appendix A: Youth Recruitment and Application Process

Appendix B: Youth Orientation

Appendix C: Youth Handbook

Appendix D: Supervisor Handbook and Information Packet

Appendix E: Supervisor Orientation

Appendix F: Placement Change Process

Appendix G: Youth Surveys

Appendix H: Supervisor Surveys

Appendix I: Focus Group and Interview Guides



LIST OF FIGURES

Figure 1: Positive Youth Development Model ..............................................................................14

Figure 2: Supervisor Oversight and Support .................................................................................26

Figure 3: Characteristics of SYEP Youth Participants ..................................................................32

LIST OF TABLES

Table 1: AED Youth Development Framework ............................................................................14

Table 2: DC SYEP Youth Participants and Supervisors (2008-2011) ..........................................20

Table 3: Youth Participant Survey Development ..........................................................................27

Table 4: Demographic and Academic Characteristics of SYEP Participants ................................31

Table 5: Characteristics of SYEP Youth Survey Participants .......................................................33

Table 6: Family Characteristics of SYEP Youth Survey Participants ...........................................34

Table 7: Organization Types Available .........................................................................................35

Table 8: SYEP Placement (Youth Survey Results) .......................................................................35

Table 9: Youth and Employer Satisfaction ....................................................................................40

Table 10: Youth and Employer Sense of Job Responsibilities and Support..................................42

Table 11: Job Readiness and Work Skills in Youth (Youth Assessment) .....................................44

Table 12: Job Readiness and Work Skills in Youth (Supervisor Assessment)..............................45



ACRONYMS

AED Academy of Educational Development

AYD Advancing Youth Development

CBO Community Based Organization

CCDC Community College of the District of Columbia

CFSA Children and Families Services Administration

DC District of Columbia and Washington, DC

DCPS District of Columbia Public Schools

DCPCS District of Columbia Public Charter Schools

DME Office of the Deputy Mayor for Education

DOES District of Columbia Department of Employment Services

DOL District of Columbia Department of Labor

DCPL District of Columbia Public Libraries

DPR District of Columbia Department of Recreation and Parks

IRB Institutional Review Board

GWU George Washington University

OSSE Office of Stated Superintendent for Education

MOU Memorandum of Understanding

MPD Metropolitan Police Department

MYLI Mayor’s Youth Leadership Institute

N/R No Response

PCSB Public Charter School Board

PYD Positive Youth Development

SAS Statistical Analysis System

SYEP Summer Youth Employment Programs

WIC/YIC Workforce Investment/Youth Council



EXECUTIVE SUMMARY

The District of Columbia (DC) Summer Youth Employment Program (SYEP) is a six week

program through the Department of Employment Services (DOES) designed to provide eligible

youth with enriching summer work experiences through placements in community-based,

private, or government sectors. The program is open to youth who ages 14 to 21 years, who are

DC residents and permitted to work in the United States. SYEP meets the needs of these youth

range using a youth development framework promoting positive work experiences.

The goal of SYEP is to provide DC youth with meaningful professional experiences to increase

employment related experiences and alleviate the potential for negative developmental

outcomes. Through SYEP, youth participants have a structured and well-supervised opportunity

to explore vocational interests, develop useful work habits and marketable skills, learn the value

of earning money through gainful employment, and obtain educational enrichment.

This evaluation utilized quantitative and qualitative methods including surveys, focus groups,

interview, and SYEP records. Key findings were given in four categories: characteristics of

participants, characteristics of job placements and supervisors, youth and employer satisfaction,

youth and employer job responsibility and support, and job readiness and work skills.

SYEP served about an equal number of males and females, with a majority in high school or

below between the ages of 14 and 17 years. In addition, a majority of the participants came from

Wards 7 and 8. It should also be noted that SYEP has a high retention rate. In addition, a

majority of the organization sites that youth were employed and supervisors that oversaw the

students were at were local non-profits and the District Agencies.

Posttest results revealed youth and employers were very satisfied with their participation in

SYEP. Over two-thirds of the youth stated that they were very satisfied as did two-thirds of

supervisors. In addition, over 95% of supervisors as well as over 95% of youth stated they

wanted to participate in SYEP again next year. It should be noted over half the supervisors stated

that they would not hire the same youth next year as this year, even though 75% of the

supervisors reported having identified the youth they wanted to employ prior to the job starting.

With regards to the application process, 89% of youth stated that the application process was

easy and 57% of supervisors stated the Host Portal was easy to use. With regards to the payroll

system, about half the supervisors stated they did not have problems with the payroll system with

the remaining majority only stating that they sometimes had issues

In addition, 80% of the youth said they felt safe at their job. Furthermore, a majority of youth

stated they would not have gotten in trouble if they had not participated in SYEP. It should be



noted that when asked what they would have done if they had not participated, many youth said

they would have hung out with their friends, tried to find another job, or babysat.

A majority of youth felt prepared for their job with supervisors confirming this. However,

supervisors did express dress code as a major challenge expressed by the supervisors. Also, a

majority of the supervisors stated they had a clear understanding of their responsibilities before

the start of SYEP and felt that they were given the information necessary to properly plan a high

quality program. With regards to support, a majority of youth stated they were very satisfied with

the support they received from their supervisors. Also, supervisors stated that their questions

were answered in a timely manner by SYEP staff.

With regards to future orientation and job readiness and work skills, over half of youth felt that

SYEP would help them later in life and youth reported having gained more of an understanding

of career interests and qualifications for future careers. In addition, the work skills that youth

stated and supervisors reported the youth gained were responsibility, reporting to work on time,

and communication.

This evaluation was conducted through support from DOES, the DC Children and Youth

Investment Trust Corporation (CYITC) and George Washington University (GWU). Comments

and questions are welcome and can be directed to the evaluator, Nisha Sachdev at

[email protected] or the Associate Director of Youth Program at DOES, Gerren Price at

[email protected].



INTRODUCTION

Background of Youth Needs

The successful transition from youth to adulthood is not only critical to individual development

but the well-being of society. The societal consequences of a well-educated citizenry include

increased productivity, lower crime rates, and increased community service. Unfortunately,

many issues plaguing youth in the United States--including poverty, sexual health, substance

abuse, low academic achievement, and crime--hinder this successful transition. While

adolescents often maintain high educational and occupational aspirations, the transition is often

characterized by few institutional supports, lack of persistence in education, and a lack of

guidance with respect to the combination of post secondary education, work, and family

(Mortimer, Zimmer-Gembeck, Holmes & Shanahan, 2002).

In addition, in today’s economy, making a successful transition into adulthood often requires not

only finishing high school but also earning a post-secondary education or training credential and

maintaining a job. Unfortunately, this is not achieved by many youth from the District of

Columbia (DC) (Ross, 2011). In 2007, only 43% of DC youth graduated from high school within

five years and only 29% of those students enrolled in post-secondary education within 18 months

of graduation (Double the Numbers (DTN), 2006). Moreover, DC Public Schools’ (DCPS)

students have the fourth highest dropout rate in the nation. In fact, in 2003, 29% of DC’s youth

(ages 18 to 24 years) were not in school, not working, and had not attained a high school diploma

(UA, 2010). This contributes to the fact that 67% of DC’s youth cannot find viable employment

(UA, 2010; Annie Casey Foundation (ACF), 2011).

Furthermore, when compared to other urban cities, DC youth have a lack of opportunities and

resources and are threatened by higher rates of high school dropout, teenage pregnancy, violence,

and substance abuse (Chaplin, 1999; UA, 2010). Sixty percent of youth live in single-parent

households and over half of youth are in households earning below the living standard (200% of

the federal poverty line) (ACF, 2010). In 2008, the teenage pregnancy rate in DC was 51

pregnancies per 1,000 girls ages 15 to 19 as compared to 41 per 1,000 nationally (ACF, 2011).

Violence continues to be higher than the national average with DC having nine times more child

murders than the national average (Finkelhor & Ormrod, 2010).

These statistics further show that poverty, educational inequalities, environmental threats, and

lack of access to health care, which many low-income youth experience, might lead to more

negative health and social outcomes (Centers for Disease Control and Prevention (CDC), 2010).

In 2009, about one in four of youth who were African American was considered disconnected

from education and work (Ross, 2011). Also, low-income African American youth are faced

with limited resources and generally have the poorest record of student academic success



(Thomas, 2000). In urban areas, effective out of school time programs can provide a positive

environment to help decrease the negative outcomes by giving opportunities and resources. For

example, youth who participate in at least one hour of activities per week are 49% less likely to

use drugs and 37% less likely to become teen parents (National Recreation and Park Association,

2010). The need for these programs is also expressed by the DC community. At a Citizens

Summit in 2010, DC residents asked “the District to offer more support for teenagers as they

transition to adulthood” with the top three suggestions being to increase mentoring, vocational

training, and life skills programs (UA, 2010).

Specifically, youth employment programs play an encouraging role in youth’s lives by exposing

them to work environments, teaching leadership, interpersonal and occupational skills, provide

opportunities to explore careers and serving as a catalyst for a positive youth development (Ross,

2009). Current studies show that participation in these programs can have lasting academic,

vocational and life benefits including increased high school graduation rates, greater

employability skills, decreased drug use, and reduced teenage pregnancy rates (Flannery, Hussey

& Thomas, 2009). Lastly, summer youth employment has been found to provide the greatest

short term benefits to society (Mael, Morath & McLellan, 1997).

Significance of Youth Employment Programs

Without the necessary academic and employment skills, many urban youth such as those in DC

will continue to achieve negative outcomes such as involvement in gangs, criminal activity,

substance abuse, and early childbearing (Hastings, Tsoi & Harris, 2010). Employment programs

exhibit potential to expose youths to supportive relationships, increasing self-worth, reducing

criminal behavior, decreasing high school dropout rates, and decreasing teenage pregnancies.

Also of importance is the need to diversify opportunities to better meet the needs and interests of

an increasingly diverse population of young people (Harvard Graduate School of Education

(HGSE), 2011). Employment experiences such as internships, summer jobs, and part-time jobs

allow youth to explore their interests, while providing opportunities for learning skills and

exposing them to real world workplace practice (Ross, 2011). Although, these experiences are

noted, many youth face challenges in obtaining job experiences.

Post-World War II years included a booming economy where high school graduates had little

trouble securing a job and the transition from adolescence to adulthood was typically smooth.

Today however, radical changes in the job market, including the growing demand for post-

secondary graduates and technology-based global economy, have sharply decreased

opportunities making it more difficult for young adults to become economically self-sufficient

(HGSE, 2011). This trend is posing a serious challenge on the United States as there are a



significant numbers of workers retiring over the next 10 years and there is a lack of prepared

youth to meet the workplace demands (Casner-Lotto & Barrington, 2006).

During the late 1970’s American attitudes toward youth employment started to shift from a

positive to negative view when new research concluded that employment during high school

tended to weaken youth commitment to their academic and other extracurricular opportunities

(Whalen, DeCoursey & Skyles, 2003). However, more current research in the early 2000’s

revealed that youth jobs do in fact have more beneficial than negative outcomes such as the

development of employability skills, increased maturity and confidence in communication with

adults, and more motivated youth to aspire to seek jobs in their interests through education.

Workplaces, it now appears, may be a unique developmental asset for youth and providing youth

with job opportunities can have a very positive impact on future prospects for employment and

earnings (Whalen et al, 2003; HGSE, 2011).

It should be noted that these benefits are often tied to amount of time worked. Studies have

suggested that youth who work long hours might have conflicts with school and engage in more

negative behaviors (Staff & Mortimer, 2010). Contrasting, youth who work a moderate number

of hours are more academically engaged and perform better in school than if they were not

working (Mortimer, 2003). It is important to distinguish the number of hours working when

exploring current research.

Overall, employment, education, and training in job skills equip adolescents with the ability to

secure jobs and assist them in becoming self-sufficient adults (Jekielek, Cochran & Hair, 2002).

Research studies have also shown that young people who work are more likely to graduate, less

likely to be involved with crime, less likely to become teenage parents, and more likely to

achieve greater lifetime earnings (Pennsylvania Partnerships for Children, 2011). Alternatively,

low-income teenaged males who cannot find work are more likely to become connected with the

law and females are more likely to become single mothers (HGSE, 2011).

Likewise, work experience benefits individual youth by proving them with opportunities that

assist in the development of work readiness skills including social responsibility,

communication, professionalism, and teamwork. In addition, it provides career exploration,

financial benefits, education, work preparedness, and future employment. Lastly, exposure to

work during adolescence assists in the growth of adult identities through opportunities for

increased responsibility, financial independence, and exposure to adult roles and expectations.

Employers also gain from work experiences and receive benefits such as increased productivity

and opportunities to train future workers.



Career Exposure and Awareness

Youth may have limited aspirations for careers and fields of study because they may not have

exposure to diverse options. Low-income parents and schools often lack the knowledge and

resources to provide this exposure to their children and youth (Americas Promise, 2011).

Although career guidance and counseling is a component of the traditional school system, it is

often inadequate due to high ratios of students to counselors. Moreover, many counselors are

trained in the area of mental health and do not have the expertise or training to provide high

quality career guidance. The lack of adequate guidance often leads youth to pursue courses in

which they are not engaged which may serve as a precursor for dropping out of schooling.

Providing a visible connection between a program of study and tangible opportunities in the

work world reduces the likelihood of this (HGSE, 2011). In addition, providing opportunities and

career experiences to young people allows them to develop individualized career goals and

pursue high school courses and post-secondary options that align with these goals.

Youth in low-income urban areas especially lack the information or the connections to help them

determine and obtain the jobs they want (McClanahan, Sipe & Smith, 2004). It is important that

they learn about different careers as it has been found that jobs that they find unchallenging in

nature create negative attitudes toward work and acceptance of unethical practices (Mael et al,

1997). It is particularly important for youth to learn how to translate their personal interests and

strengths as a tool to help guide their career choices and educational options (Whalen et al,

2003). Employment programs can inform youth about career and educational options and

motivate them to see the connection between high school studies and work (Whalen et al, 2003).

Career Exploration

Likewise, youth are more likely to succeed in both school and the workplace when they are able

to explore topics and acquire skills that are relevant to their interests, when they have supportive

adults guiding them, and when they are given opportunities to be exposed to different careers

(Lippman & Keith, 2009). Specifically with regards to urban youth, consistent exploration in

professional work settings provides them not only with work experiences but also often with the

opportunity to be exposed to new neighborhoods, diverse populations, and life skills such as

managing a bank account (Whalen et al, 2003). Also, youth who work during their high school

years develop strategies of time management that stay with them through their educational career

(Staff et al, 2007). Employment programs can help better prepare youth for the world of work

and provide unique learning opportunities to acquire these applied skills.

Financial and Social Benefits

Low-income African American and Hispanic youth are generally more susceptible to the

consequences of economic fluctuations (Land, 2010). When the economy is doing well, usually



their well-being improves also. However, when the economy falls, they experience more

hardships than their white counterparts (Land, 2010). Youth with limited work experience also

face limited earnings later in life, perpetuating the cycle of poverty (Ross, 2011). For example,

between 1979 and 2005, real hourly wages for college graduates rose by 22%, remained constant

for high school graduates, and fell by 16% for high school dropouts (Mishel, Bernstein &

Allegretto, 2005). These statistics are of particular importance to low-income youth as they often

have lower rates of high school completion and college attendance.

Youth in urban neighborhoods often lack positive adult role models for employment as many of

the working adults they know often earn low wages, may not have positive experiences to share,

and possess only a few occupational skills (McClanahan et al, 2004; Allen, 2006). This may not

only inhibit their awareness of careers but also make them complacent with engaging in low

paying jobs. Also, parents of youth in poverty often lack the connections to help the youth obtain

jobs and do not encourage their children to obtain employment (Allen, 2006).

Youth employment programs have been found to provide long term benefits such as higher

annual earnings, greater likelihood of receiving fringe benefits, and higher status occupations.

(Jekielek et al, 2003). Also, programs that have supervisors that assume the role of a natural

mentor may help youth engage in positive health behaviors (Bauermeister, Zimmerman, Gee,

Caldwell & Zue, 2009). Furthermore, it has been found that youth who earn their own money

access social services such as medical care and money – all the more reason to adequately

prepare low-income youth for successful transition into the workplace (Bauermeister et al, 2009).

Educational Benefits

Youth participation in out of school time activities including employment programs is predictive

of academic success as measured through test scores, absenteeism, school dropout rates,

homework completion and school grades, and course enrollment (Simpkins, 2003). Youth who

have quality work experiences are also more likely to be inspired to stay in school, graduate, and

form concrete goals (HGSE, 2011).

Between 2000 and 2015, about 85% of jobs will require education beyond high school (Casner-

Lotto et al, 2006). Employment programs can promote positive academic attitudes and increase

the likelihood that youth will take academic courses of interest (Jekielek et al, 2003). A majority

of young people understand the necessity of a post-secondary degree and aspire to go to college,

yet lack the knowledge and motivation to not only apply and enroll in college, but in some cases

even to graduate high school. Since many low-income youth face the prospect of a difficult

transition into the work or college world, practical work experiences can provide not only

income benefits, but can also help them recognize why educational attainment is important,



increase their interactions with working adults, and expand their aspirations and achievable goals

(McClanahan et al, 2004).

Work Preparedness

Work experience helps youth develop employability skills. A study by Greenberger & Steinberg

(1986) found that working adolescents describe themselves as possessing qualities such as being

dependable, punctual, and responsible more than nonworking adolescents (Greenberger et al,

1986). This reinforces the fact that college readiness alone does not equip young people with all

of the skills and abilities they will need in the workplace or to successfully complete the

transition from adolescence to adulthood (HGSE, 2011).

In addition, employers believe that youth are not equipped with the adequate skills needed to

succeed in today’s workforce (HGSE, 2011). According to a survey of several hundred

employers, 80% rated professionalism and work ethic as the most important skills needed by

entrants to succeed in today’s workforce (Casner-Lotto et al, 2006). In addition, over 40% of the

same employers responded that new entrants with a high school diploma are poorly prepared in

these skills (Casner-Lotto et al, 2006). Furthermore, human resource executives interviewed

emphasized the need for proper dress, strong interviewing and communication skills and an

understanding of the job application process (Casner-Lotto et al, 2006). Youth employment

programs are an excellent venue to prepare youth to enter the workforce.

Future Employment

Research also shows that the more teenagers work in one year, the more likely they are to work

in the following year (Ross, 2011). Traditionally, adolescence is a period where youth are

structured to engage in long-term academic preparation instead of activities that expose them to

the adult world (Whalen et al, 2003). This causes youth to not be exposed to workplace norms

and have unrealistic expectations about the work world (Whalen et al, 2003). Reducing the share

of youth with low or no qualifications is a key to addressing the challenges facing youth in

America (OECD, 2009). By providing experiences early on, youth are exposed to interests and

areas that they may pursue in future employment.

Role of Employers

Work experience not only benefits young people but employers as well. Employers play an

important role in preparing youth for successful transition into adulthood. Not only do they

provide opportunities for work-linked learning but often also advising and training in relevant

skills (Casner-Lotto et al, 2006). Employers also can provide developmental assets to youth that

no other setting can fully duplicate including exposure to the mainstream economy, practices of



the working world, authentic information about career options and paths, and opportunities to

apply formal learning to solve real-world problems in a team setting (Whalen et al, 2003).

Employers can increase their financial and productivity goals by investing in a skilled workforce,

particularly in the current economic environment (Martinson, 2010). Employers spend over $400

billion a year in providing both formal and informal training to employees who have already

completed their schooling and are currently working full-time (Casner-Lotto et al, 2006). By

providing jobs through youth employment programs, not only are employers preparing youth at

an early age for employment, but can also rely on the program to support training efforts. This

allows for employers to spend less time and costs on training while preparing their future

workforce. Specifically, summer youth employment programs are an excellent avenue to include

employers as it allows for them to spend more time providing direct service to youth and less

time on program administration (Whalen et al, 2003).

Positive Youth Development Framework

This evaluation follows a positive youth development (PYD) approach, focusing on the strengths

of youth rather than their weaknesses (Breinbauer & Maddaleno, 2005). This approach

recognizes that all youth can be successful if provided support, guidance, and opportunities that

meet their needs (Clymer et al, 2002). PYD suggests that helping young people achieve their full

potential is the best way to prevent them from engaging in risky behaviors (US Department of

Health and Human Services, 2010). The fundamental principle underlying PYD is that youth can

successfully progress through adolescence by developing skills and abilities including social and

interpersonal skills, basic academic skills, capacity to understand and plan for the future, ability

to take responsibility, and obtain knowledge of vocational skills and career interests (Clymer,

Edwards, Ponce & Wyckoff, 2002). There are many variations of this approach but important

constructs included in all are promoting a sense of safety; providing appropriate structures;

creating supportive relationships; providing opportunities to belong; providing positive social

norms; giving youth responsibilities and meaningful challenges; and providing opportunities for

skill building (see Figure 1).

PYD occurs in a wide range of settings such as programs, organizations, socializing systems, and

communities (Public/Private Ventures (PPV), 2005). Many young people, particularly in low-

income communities, rely on PYD programming to help them make a safe and healthy transition

into adulthood. PYD programs help youth gain skills and provide them with the resources

necessary for them to learn to solve issues they are facing and make decisions that result in

healthy living (Clymer et al, 2002).



As mentioned, much empirical research has shown that urban youth lack direction with the

transition from school to the postsecondary and workforce worlds. This issue is further

intensified by the lack of academic preparation, economic issues, and need for youth

development skills. There is also growing empirical evidence that well-designed PYD

interventions can lead to both short and long term positive outcomes for youth. PYD approaches

have also been found to be effective with youth employment programs by increasing workforce

competencies, providing education and training opportunities, and increasing future orientation.

In addition, they usually involve caring adults who serve as role models. Lastly, they incorporate

activities that allow peers to interact (Zuckerman, n.d.). Overall, incorporating

Many PYD theoretical constructs and frameworks have been developed and used to guide

programming and studies (PPV, 2005). DC has developed a citywide strategy centered on youth

development utilizing the Academy of Educational Development (AED) Advancing Youth

Development (AYD) Curriculum Framework. As the local provider of this curriculum, the DC

Children and Youth Investment Trust Corporation (CYITC) provide training for District

agencies and providers, front line staff, supervisors and policy makers. In particular, these

trainings have been customized for DCPS teachers and staff and DC Metropolitan Police

Department (MPD) school resource officers. To date, over 3,000 youth workers have completed

the 30-hour training, representing more than 180 CBOs, Department of Employment Services

(DOES), the Department of Parks and Recreation (DPR), the Children and Families Services

Administration (CFSA), and the DC Public Libraries (DCPL).

AED’s Youth Development Model and the citywide use of this model make it the appropriate

framework for this evaluation. AED’s model identifies opportunities and supports for youth that

are necessary to achieve 12 outcomes that indicate healthy development in youth. The model

further categorizes these 12 outcomes in areas of identity (youth demonstrate a positive identity

when they have a sense of personal well-being and a sense of connection and commitment to

others) and areas of ability (youth demonstrate ability when they gain knowledge, skills and

attitudes that prepare them for adulthood) (see Table 1). Specifically, SYEP works towards

mastery and future, employability, and responsibility and autonomy, and the development of the

evaluation in this study is appropriately grounded in these core constructs.



Figure 1: Positive Youth Development Model.

Source: PPV, 2005.

Table 1: Academy for Educational Development Advancing Youth Development Framework

Youth Development

Outcome Definition

Identity

Safety and

Structure A perception that one is safe in the world and that daily events are somewhat predictable.

Self-Worth A perception that one is a “good person” who contributes to self and others.

Mastery and

Future A perception that one is “making it” and will succeed in the future.

Belonging and

Membership A perception that one values, and is valued by, others in the family and in the community.

Responsibility

and Autonomy

A perception that one has some control over daily events and is accountable for one's own actions and

for the consequences on others.

Ability

Self-Awareness

and Spirituality

A perception that one is unique and is intimately attached to extended families, cultural groups,

communities, higher deities, and/or principles.

Physical Health The ability and motivation to act in ways that best ensure current and future physical health for self and

for others.

Mental Health The ability and motivation to respond affirmatively to and cope with positive and adverse situations, to

reflect on one's emotions and surroundings, and to engage in leisure and fun.

Intellectual

Ability

The ability and motivation to learn in school and in other settings, to gain the basic knowledge needed

to graduate from high school, to use critical thinking, to be creative, to use problem-solving and

expressive skills, and to conduct independent study.

Employability The ability and motivation to gain the functional and organizational skills necessary for employment,

including an understanding of careers and options, and the steps necessary to reach goals.

Civic and Social

Ability

The ability and motivation to work collaboratively with others for the larger good and to sustain caring

friendships and relationships with others.

Cultural Ability The ability and motivation to respect and affirmatively respond to differences among groups.

Source: CYITC, 2011.



Purpose of the Evaluation Report

Ongoing, rigorous evaluation of programs helps determine what works, why it works, and what

are replicable youth development interventions (Bernat & Resnick, 2006). The purpose of this

evaluation was to design and implement a qualitative and quantitative analysis to document the

value, benefits, innovations, capacity building, and key challenges of SYEP for the summer of

2011, as well as to provide recommendations for implementation for the following years.

Specifically this report will examine the extent to which SYEP is reaching the youth and

providing diverse programming with which the participants and employers are satisfied. In

addition, it will examine the short term changes in the youth participants with regards to job

readiness and work skills. This evaluation is also very timely, as it will provide insight for

SYEP’s action plans being developed for the Summer 2012.

Specifically, this report addresses:

What changes have occurred to address shortcomings identified in previous program

evaluations?

What are the support mechanisms for participating youth and employers?

What kinds of youth and supervisors participate in SYEP?

What kinds of jobs are youth placed in through SYEP?

Are youth and supervisors satisfied with their participation in SYEP?

What are the short term impacts in regards to job readiness and specific work skills for

participating youth?

This evaluation report was compiled for the Council and DOES to provide an evaluation of

SYEP and the results are intended to provide guidance to future program implementation. It

should be noted that an extensive research study is currently being conducted by the evaluator

(Nisha Sachdev) through support from George Washington University (GWU), CYITC, and the

Department of Labor (DOL) Employment and Training Administration. The purpose of this

extensive study is to expand the understanding of the impacts that summer youth employment

programs have on youth as they transition to adulthood in not only DC but other cities

implementing youth employment programs. Specifically, the study will document the

implementation of the program and the short term behavioral impacts of SYEP on the youth

participants while providing a framework to assess longer term impacts. This study will not only

assist DOES, but also the stakeholders at large such as other agencies, youth, parents, and

taxpayers, with results on the effectiveness and quality of programming and resulting behavior

change. The results of this comprehensive study will be made available in May 2012.



DC SUMMER YOUTH EMPLOYMENT PROGRAM

History of Youth Employment and Training Programs

In the early 1960’s, to reduce poverty and inequality in the labor market, the Manpower

Development and Training Act was established which provided federal funding for employee

training and development. This allowed for the development of youth employment and training

programs geared towards low-income youth to provide means for them to not only earn money,

but also learn new skills and explore careers (McCalahan et al, 2004). Specifically, publically

subsidized summer youth employment programs were implemented to provide urban youth who

lacked opportunities with career experience as well as activities to help them stay out of trouble

(McCalahan et al, 2004).

However, in the early 1980’s, research revealed that the jobs were often poorly planned and

supervised and did not portray real-world work experiences. In addition, critics of the programs

stated that an educational focus needed to be included as it is linked to career success

(McCalahan et al, 2004). This coupled with the economic growth in the 1990’s, which provided

youth alternative opportunities for employment, decreased the visibility of youth employment

programs (McCalahan et al, 2004). Summer youth employment programs experienced similar

trends with rates also reaching new historical lows, especially within minorities, low-income

youth (Sum et al, 2008). Currently there is only some federal money reserved to youth

employment programs, and no fully federally funded summer job programs. However, many

local jurisdictions such as Washington, DC have developed their own summer programs using

supports from the private, local government, and non-profit sectors (McCalahan et al, 2004).

In 2010, the unemployment rate for all 16 to 24 year olds fell to its lowest since the end of World

War II (18.1%), making 3.8 million unemployed (Weeter et al, 2011). Teens from low-income

families who were African American or Hispanic were more severely affected than the average.

Only 19% of low-income African American teens worked during 2007 compared to almost 50%

of their more affluent White counterparts (Sum, Khatiwada, McLaughlin, Palma, 2008). In

addition, according to Kuhen & McDaniel (2009), by the age 24, around half (57%) of low-

income African American youth were employed, compared to nearly three quarters (74%) of

white youth (Kuhen et al, 2009). Over the past decade the summer youth employment of teens

fell from 45% to 25.6% (McLauglin & Sum, 2011).

The lack of jobs is further exacerbated for inner city and minority youth, due both to spatial

isolation and discrimination, denying youth these developmental benefits (Whalen et al, 2003).

Although local jurisdictions are in fact implementing summer youth employment programs,

many youth are often turned away due to lack of space with the limited resources. The high rates

of applications submitted to participate suggests that many youth do want to work, but are not



provided the resources or opportunities due to limited space or financial resources (Hastings,

Tsoi & Harris 2010).

History of DC Summer Youth Employment Program

SYEP was initiated by then Mayor Marion Barry in 1979 through both Federal and local District

funds. The program was open to all DC youth ages 14 to 21 years. In 1988, the Federal

Workforce Investment Act was established which prohibited federal summer job program that

were not connected with year round programs. Recognizing the importance of the program,

Mayor Barry continued to support it through local funds and employed almost 16,000 youth

(Curnan, Kingley, LaCava & Frees 2010). Although there was strong support from both the

Mayor and the public, low oversight and accountability led to a lack of quality, scale, and scope

over the years.

SYEP aims to prepare participants for work; help them explore career and vocational

opportunities by placing them in supervised career-related jobs or opportunities; provide them

adult support and guidance; and provide career-related experiences such as resume writing and

interviewing skills. SYEP has grown substantially since its inception in 1979. Key programmatic

milestones were implemented beginning in 2006 including expanded youth employment

opportunities to include the private and government sector, providing electronic payments on an

individualized debit card, a more comprehensive online application system, providing youth with

neighborhood based placements, taking in account youth interests and strengths, and site visits to

assess quality work assignments. This has led to youth experiences that are diverse and meet

their needs.

In 2008, with Mayor Adrian Fenty in office, there were hopes of doubling the youth enrollment

from 10,000 to 20,000 youth and increasing the number of worksites. While this goal was

reached, little preparation was put into the necessary upgrades in management infrastructure,

capacity-building, and payroll systems leading to much criticism from the media and press

(Curnan et al, 2010). In an attempt to diminish these criticisms, considerable efforts were made

to improve SYEP in the summer of 2009 including new organizational, management, tracking,

and payroll systems (Curnan et al, 2010). With these systemic successes, in 2010 efforts were

then reinforced on ensuring youth had positive experiences. Pre-screening site visits of potential

worksites were implemented, youth development training was provided to supervisors, a Work

Readiness Assessment was performed by the supervisor to evaluate youth performance, and

financial training workshops were provided for the youth. This was done while maintaining the

success of providing opportunities to 20,000 youth at over 1,300 worksites.



Also in the summer of 2010, Brandeis University conducted a qualitative study of SYEP to

identify successes, challenges, and lessons learned. Through interviews with SYEP staff at a

select number of worksites as well as worksite observations, they found that although there have

been many short term and technical advances in the program, longer term strategies are needed

such as ensuring age appropriate worksites, decreasing the number of participants to provide

quality experience, better communication and more planning time (Curnan et al, 2010). This

study draws on these findings and furthers explores the implementation of SYEP as well as

outcomes on for the youth.

Key Stakeholders

DOES is the primary agency responsible for workforce development in DC. Youth programs and

services within DOES include a Year-Round In-School Program, Out-of-School Youth Program,

the Mayor’s Youth Leadership Institute (MYLI), and SYEP. CYITC is a public-private

partnership with the DC Government and is the primary resource for expanding and improving

services and opportunities youth in DC, especially during out of school time. Specifically,

CYITC provides grants, technical assistance, capacity building, learning opportunities, policy

support, and youth development training to youth workers using AED’s AYD curriculum in the

District. Since the summer of 2009, CYITC has partnered with DOES to provide funding

through a competitive proposal process to qualified applicants that provide high quality summer

programs for SYEP registered youth ages 14 to 15 years. Specifically, CYITC seeks to support

the delivery of a variety of workforce exploration and experience based programs that will

provide purposeful and developmentally appropriate employment and career exploration

opportunities. Applicants must employ the youth development philosophy in their approach and

program design. There are about 136 CBOs in DC that provide education, training, or

development services to youth (Ross, 2011). These services include GED preparation, academic

assistance, work readiness training, occupational skills training, job and internship placement,

wrap-around services, and case management (Ross, 2011). Many of these programs act as Host

Worksites for SYEP and provide counselor positions for older youth to work with younger youth

as well as employability or work readiness training.

2011 Summer Youth Employment Program Design

In 2011, SYEP was a six week program, locally funded initiative administered by DOES that

provides DC youth ages 14 to 21 years with enriching and constructive summer work

experiences through subsidized placements in the private and government sectors. The program

is structured to provide youth with experience and training to develop their employment skills

and career awareness. It strives to provide young people with the opportunity to earn money and

gain meaningful work experience; learn and develop the skills, attitudes, and commitment



necessary to succeed in today’s world of work; gain exposure to various exciting career

industries; and interact with dynamic working professionals in a positive work environment. The

2011 program began on Monday, June 27, 2011 and ended on Friday, August 5, 2011 (with July

4, 2011 an observed holiday).

Though SYEP is a short term employment and training program, the goal is to introduce DC

youth to employers and experiences that will positively impact their futures. Employers in the

DC metropolitan area make this program possible by volunteering to serve as Worksite

Supervisors (hereby referred to as supervisors) and provide structured job opportunities for youth

during the summer. They provide guidance and training which enable young people the

opportunity to develop positive work habits, attitudes, and the valuable job skills necessary to

enter the workforce prepared and qualified to be productive employees.

This summer, youth were paid the federal minimum wage ($7.25 per hour in 2011). All of these

wages were fully paid by DOES. Participants were only compensated for time worked up to the

maximum number of hours permitted by the program. This includes 20 hours per week for youth

ages 14 to 15 years and 25 hours per week for youth ages 16 to 21 years. Youth signed in and out

daily via an online system (SYEP Youth Portal), signed up to receive their bi-monthly pay on an

ADP Visa Debit Card. Youth ages 18 years and older had the option to sign up for direct deposit

with partnering banking institutions.

Youth who participated were required to participate in an in-person orientation hosted at DOES

that went over program details including logistics, rules, and expectations. In addition, there was

a supplemental an online orientation through the SYEP Youth Portal available prior to the start

of the program. The orientation consists of a series of short videos that addressed specific SYEP

content questions. In addition, during the first week of work, youth were provided orientation at

their worksite provided by their supervisors. This orientation included information pertaining to

the hours the youth would be working, the regulations of the worksite, time and length of lunch

breaks, emergency contacts to notify when the youth may be late or absent, safety procedures

and steps to take in case of an accident, appropriate attire for the workplace, and a clear

explanation of the duties and responsibilities including the criteria by which they would be

evaluated.

Changes to Address Shortcomings in Previous Program Evaluations

Based on the findings of the study conducted by Brandeis University as well as budget cuts and

pressure for sound oversight and accountability, SYEP 2011 incorporated significant changes to

improve quality. First, the program decreased the number of youth to serve about 12,000 youth

and placed an emphasis on older youth with 3,000 slots available to youth ages 14 to 15 years



and 7,000 slots available to youth ages 16 to 21 years. An additional 2,000 slots were allocated to

youth who are participating in CYITC funded programs. It should be noted that an extra 990

slots were added during the first week of the program due to extra external resources and were

made available due to the large waitlist (see Table 2).

Furthermore, the online application through the Youth Portal was accompanied with in-person

events to provide ample opportunity for assistance with the application process and eligibility

requirements. The new system also allowed youth to apply for jobs in which they are interested

and in which they feel meet their needs. Job placements were also categorized in two different

strands: (1) work experience programs designed to provide youth with hands-on work experience

and (2) work readiness programs designed to provide youth with an opportunity to receive basic

skills training and enrichment in a non-work setting such as a CBO. The system and an SYEP

Job Expo not only allowed for multiple opportunities for supervisors to interview and screen the

youth participants, but also allowed for the youth to learn about the different opportunities

available. This helped ensure placements were made that met the needs of both the youth and

the employer. More emphasis was also placed on youth orientation and transparency about all

program dates and deadlines by providing this information at the start of the program. Lastly, all

supervisors were required to participate in youth development training provided by the CYITC to

help with program quality.

Table 2. SYEP Youth Participants and Supervisors (2007 – 2011)

Year Total Youth Placements

(% change from previous year) Total Supervisors

2011 13,641 (-36%) 2,243

2010 21,297 (+6%) 1,350

2009 About 20,000 (+5%) n/a

2008 About 19,000 (+37%) n/a

2007 About 12,000 (n/a) n/a

Source: Ross, 2011; Curnan et al, 2010; DOES, 2011.

Goals of 2011 Summer Youth Employment Program

The core activities of the SYEP program aim to expose youth to meaningful summer

employment experiences. These work experiences will provide a motivating context integral to

the PYD model and will ultimately lead to positive transition to adulthood while decreasing

negative behaviors. Although SYEP’s primary goal is to provide DC youth with meaningful

professional experiences and basic work skills, the program was designed with the hope that

these short term impacts will lead to long term impacts in the development of the youth



participants. These impacts include school engagement, workplace readiness, decreased

engagement in risk behaviors, retention in the program, and enrollment in college which

ultimately leads to youth being able to enter the workforce and becoming self-sufficient adults.

Youth Eligibility

Youth ages 14 to 21 years make up about 11.5% (about 69,352 youth) of the total population in

DC (U.S. Census Bureau, 2011). It should be noted that Wards 2 and 8 have the greater share of

the older teenage population (18 years and older) in the city compared with other wards,

according to the 2000 Census, at 10.4 and 10.7 percent, respectively (U.S. Census Bureau, 2011).

The demographics of these two wards are extremely different. The high share of teenagers in

Ward 2 is presumably driven by enrollments at Georgetown University and George Washington

University. The share of teenagers in Ward 8 is primarily low-income youth who presumably

grew up in the District (or nearby).

Program recruitment targeted youth whose ability to access employment opportunities may be

limited. Taking this and the above into consideration, a more accurate estimate of the target

youth for SYEP is 33,577 youth ages 14 to 21 years (U.S. Census Bureau, 2011). This estimate

was calculated by looking at the youth population of those ages 14 to 21 in Wards 1, 4, 5, 6, 7,

and 8. This is also where a majority of the youth were recruited and participated from. During

the summer of 2011, SYEP served 13,641 (41%) of these youth.

To be eligible for the SYEP program, youth had to be a resident of DC, be between the ages of

14 and 21 years (prior to the start of the program), provide a social security card and birth

certificate to verify eligibility to work in the United States, and have parental or guardian

permission to participate (if under 18 years of age). Each SYEP participant was placed in a job at

a CBO, private, or public agency under the guidance of a supervisor. This provided participants

with the opportunity to develop work skills in a real work environment.

Youth Application and Selection Process

The youth application process consisted of an online application through the SYEP Youth Portal

and a document certification. The application process for 2011 remained open from February 25,

2011 until March 19, 2011. A total of 20,463 youth applied to SYEP during the enrollment

period. Each youth was required to provide their full social security number and a valid email

address to access the system. Recruitment efforts included providing youth with information on

public computers available in their neighborhoods (see Appendix A).

After the youth successfully completed the online application, they were required to bring their

eligibility documents to DOES. These documents included a parental consent form for youth



under the age of 18 and verification of residence in DC, age, and permission to work in the

United States. If the youth was a prior participant, documents were migrated over from the

previous year with the exception of the parental consent and DC residency verification. DOES

understood that transportation might be an issue for the youth and therefore held SYEP

Eligibility Certification Events in local neighborhoods on evenings and weekends. Youth were

also provided email reminders about these events as well as the documents they needed to submit

(see Appendix A).

SYEP accepted 14,183 eligible youth who completed these steps (69% of those who applied).

Youth who completed these steps and did not fall in the spots were placed on the waitlist. It

should be noted that although 14,183 youth were accepted into SYEP, only 13,641 youth

participated in the program as by the start of the program some youth had found other

employment or had outstanding circumstances.

Once the complete application was submitted and accepted, youth were then required to submit a

resume, complete an online and in-person orientation course, and apply for specific job

opportunities that were of interest to them, all through the SYEP Youth Portal (see Appendix B).

In addition, youth could view important messages sent by SYEP, learn about financial

management, and view total hours worked per pay period on the SYEP Youth Portal. In addition,

the youth were provided a handbook with expectations, logistical information, and further

information to help guide their time in the program (see Appendix C).

Once youth applied for specific job opportunities through the SYEP Youth Portal, supervisors

had the ability to screen, interview, and select, through the Host Employer Portal, the specific

youth from the eligible applicant pool who they would like to hire. SYEP also hosted a SYEP

Job Expo on April 20, 2011 and April 21, 2011 at the Building Museum that was open to all

eligible youth applicants and provided them with an opportunity to meet employers and gain

additional information about available summer job opportunities. In addition, employers were

able to interview candidates on the spot and make selections of youth who they wanted to hire.

For the employers who did not wish to screen or interview youth, DOES placed youth on their

behalf. Youth were matched based on selections made by the employers and/or the interests that

the youth listed their online application.

SYEP placements followed DOL regulations on child labor laws, although most of the jobs

restrictions listed are out of the scope of SYEP and Host Agencies. This includes that youth ages

14 to 15 years cannot engage in public utilities or construction jobs, driving a motor vehicle or

helping a driver, manufacturing and mining occupations, power-driven machinery, public

messenger jobs, and warehousing and storage. In addition, youth ages 16 to 17 years may not



work in jobs involving manufacturing and storing of explosives, driving a motor vehicle and

being an outside helper on a motor vehicle, mining, logging and sawmilling, most power-driven

machines, roofing operations, and excavation operations. Youth ages 18 years or older were no

longer affected by the child labor laws.

Host Employer Application and Selection Process

Employers who were interested in participating as a host employer were required to submit a

Host Employer Application though the Host Employer Portal. The application consisted of

contact information, names of employers who would be supervising the youth (required a

minimum ratio of one adult to 12 youth workers), contact for who will be coordinating

timesheets, and job descriptions detailing the specific opportunities to be offered along with the

age and skill criteria required for the positions. Once the application was submitted, a DOES

Representative completed a site visit to ensure the site was safe, structured, and properly

supervised. Applications were reviewed on a rolling basis and final decisions were made on

April 1, 2011. A total of 2,243 supervisors (465 Host Agencies) applied and were accepted. This

allowed for a total of 16,629 positions for the youth.

A Supervisor’s Handbook and Information Packed helped guide the process of employing youth

(see Appendix D). The handbook also provided information related to payroll, the role of staff,

working with youth, and the necessary paperwork required by DOES.

Each Host Agency had to have an indentified team to ensure a positive experience for both the

employers and youth. The team consisted of a Host Coordinator, Payroll Coordinator, and

supervisor. The Host Coordinator was responsible for serving as the primary point of contact

between the Host Agency and DOES, communicating problems or questions regarding the

program to DOES, and ensuring supervisors collect all required documentation. The Payroll

Coordinator was responsible for submitting time electronically on behalf of the youth weekly

and maintaining copies of all timesheets and providing them to DOES at the end of the program.

The supervisors were responsible for ensuring that youth were properly supervised at all times,

ensuring youth time and attendance procedures were followed, keeping the Host Coordinator

informed of all issues, and administering performance evaluation of each youth under their direct

supervision. The supervisors become one of the primary adults with whom the young people

formed a relationship during the program. They not only provided participants with training and

career exposure, but also serve as an adult role model.

Each approved application was provided one of seven SYEP Liaisons based on the sector from

which they are applying: CBO, DC Government, Federal Government, Private Sector, or Schools

(see Figure 2). This SYEP Program Liaison was an SYEP employee who served each sector (e.g.



private sector, CBO, etc.) as the primary contact for the Host Agency. The SYEP Program

Liaison was responsible for providing support and guidance to employers, communicating

pertinent information about the program, facilitating the process for transfers and terminations,

resolving pay disputes, and addressing other program issues. In addition, an SYEP Program

Monitor was assigned to worksites to monitor the environment, advocate for youth and support

supervisors with any issues they may have experienced. Specifically, they monitored the site to

ensure youth were receiving quality work experience, supervisors were receiving adequate

assistance and resources from DOES, tracked the submission of necessary paperwork, and again

assisted youth and supported supervisors with any issues they may be experiencing.

Once the application was approved, all employers at the respective organization or agency who

were identified as supervisors were required to attend a mandatory SYEP Supervisor Training

and Orientation that provided information about program logistics, payroll instructions, and

program planning (see Appendix E). Here they were also provided extensive training on youth

development following AED’s AYD curriculum. These trainings were conducted by CYITC as

they are the local provider of this curriculum. Supervisors working directly with youth

participants under the age of 18 years were also required to obtain a criminal background

clearance within the last two years through the DC MPD prior to the start of the program. DOES

provided these clearances free of charge. In addition, during the first week of work, supervisors

were required to provide the assigned youth with an onsite orientation. Information to help with

this process was in the Supervisor’s Handbook.

A select number of CBO sector worksites received funding through a partnership with CYITC

and DOES. These organizations went through a competitive proposal process and were chosen

for their capacity and ability to provide high quality summer programs for SYEP registered

youth ages 14 to 15 years. A total of a total of 41 worksites were selected that served 2,048.

These sites were referred to as CYITC-Funded programs and received extra guidance and

support to provide intentional work readiness programs to the youth they served. In addition,

they were required to also follow all guidelines of SYEP.



Figure 2: Supervisor Oversight and Support

Additional Rules of Summer Youth Employment Program Participation

The youth were required to give advance notice of any absences from work, regardless of the

reason. If this could not be done in person, the participant was told to telephone the supervisor as

soon as they know they will be unable to report to work that day. If the absence continued

beyond one day, the participant was to notify the supervisor each day that they would not be

present. In addition, the supervisor had to notify their SYEP Program Liaison about the youth

worker’s absenteeism when the youth was absent more than three consecutive days. Youth were

not paid for any absent days; however they could schedule makeup hours if approved by the

supervisor or Host Coordinator.

Youth and/or supervisors could request a transfer if there was a safety issue, health concern, site

closure, or another approved extenuating circumstances. The Host Coordinator would process

transfer requests and make the final determination of the transfer. Youth could face termination

from the worksite if they engaged in misconduct including possessing, selling, or using illegal

drugs or alcohol while on the job, failing to report to work on three consecutive work days

without prior approval; disruptive behavior such as fighting; physical or verbal assaults; stealing

property from the worksite, employees, or other youth workers; falsifying time records; refusing

to adhere to the worksite’s rules and regulations; and verbal, sexual, or physical harassment. All

incidents leading to termination had to be documented and submitted to the Host Coordinator. If

approved, an official termination letter was sent to the youth worker. If a SYEP participant

believed they had been wrongfully terminated from a worksite, they had the opportunity to

schedule an appointment with the Host Coordinator (see Appendix F).



OVERVIEW OF METHODS OF EVALUATION

The evaluation encompassed qualitative and quantitative methods of data collection including

interviews and focus groups, surveys, and SYEP records to obtain information geared toward

understanding youth and supervisor characteristics, experiences, successes, and challenges in the

program and their overall development. The study sample includes the SYEP youth and

employer participants for the summer of 2011 as well as a small sample of non-participants. For

the purpose of this report, the data from the posttests will be used as all it incorporates all of the

areas being addressed (i.e. satisfaction and support, job responsibilities, and work related skills).

Quantitative Methods

Youth Survey

Youth participants completed a baseline survey at the start of the program and again at the end of

the six week program. The survey included demographics, satisfaction with program, career

interests, academic characteristics, work orientation and attitudes towards risk behaviors. Youth

received the link via Zoomerang, an internet-based survey tool to complete the questionnaire via

multiple recruitment efforts including SYEP reminders, site reminders, social media sites, as

well as emails sent to the youth (See Appendix G). The survey took between 10 to 15 minutes to

complete. All data was analyzed in Statistical Analysis Software (SAS) version 9.2.

The multiple attempts made to have all the participants complete the survey resulted in a 7%

response rate for both the pretest (930 youth completed) and for the posttest (888 youth

completed). Although this rate may seem low, for a population size of over 10,000, an adequate

sample size is 350 (Fitz-Gibbon et al, 1987). This allows for ample consideration for non-

responses. Analysis of this subgroup of respondents shows similar representation of the total

youth participant population with respect to age, gender, ward, and grade level.

The survey instrument was developed specifically for this evaluation using items and scales from

existing validated surveys. When developing the instrument, items were constructed and

modified from other tools such as Detroit’s Summer Youth Employment Program Evaluation

2010, MyCom Summer Work Readiness Assessment, The Colorado Trust Youth Participant

Survey, and the Ascend Summer Youth Employment Program 2005 (Shanks & McGee, 2010;

Flannery et al, 2009, The Colorado Trust, 2004; Nielsen & McGhee, 2005). These surveys

address youth employment experiences as well as risk behaviors, however very few provided

information over their reliability and validity (see Table 3). The AED AYD framework guided

development of the survey.



Since a new survey was being developed from existing instruments, steps were taken to ensure

the new scale was valid and appropriate for intervention specific outcomes. Validity is the

scale’s ability to measure what it intended to measure. It is especially important to conduct

validity checks when the instrument is new or has been changed from its original form as many

factors affect the validity of a measure. First, face validity, which is the degree to which an

instrument appears to measure what it is designed to measure, was assessed. This was done by

having non-experts review the survey as well as the youth pilot-testing the survey. The surveys

were pilot-tested by a convenience sample of ten individuals ranging from 18 to 22 years of age

who previously participated in the program but did not be this year due to age restriction or

missing application deadlines. They specifically assessed the length and readability of the

questions. Content validity is based on the extent to which a tool reflects the specific intended

domain of content. Content validity was assessed using a panel of experts including faculty at

GWU and researchers in the youth development field, who will judge the relevance of the items

asked on the survey in order to ensure that the questions asked is actually. In addition, 96% of

the 888 posttest youth respondents reported that they were honest in taking the survey (the last

question asked “How honest were you in filling out this survey?”) further validating the results.

It should be noted that there was an attempt to implement a quasi-experimental design with a

comparative group of those youth who have applied to the program and not been accepted due to

lack of space in the program or not turning in the necessary documents to enroll in the program.

Due to logistical and consent issues, the completion rate of these surveys was low therefore were

not included in this evaluation. It is planned to implement this design for the summer of 2012.

Table 3: Youth Participant Survey Development

Source: Shanks et al, 2010; Flannery et al, 2009, The Colorado Trust, 2004; Nielsen et al, 2005

Description Sample Items

Response

Formats

Detroit’s 2010

Summer Youth

Employment Exit

Evaluation

Measures attitudes about how employment

opportunity might influence career/academic

aspirations, program impact on attitudes and

knowledge, and overall satisfaction of the program.

28 items: How did the program

help you academically? What

job skills did you learn from

this program?

Multiple

choice,

Yes/No, and

open-ended.

MyCom Summer

Work Readiness

Assessment

Measures interpersonal relationships, psychosocial

functioning, self-efficacy, out of school time

activities, and independent living.

20 items: Overall how satisfied

are you with your life right

now? How optimistic are you

with your future?

Likert-scale

The Colorado Trust

Youth Participant

Survey

Measures knowledge, attitudes, and behaviors with

regards to youth outcomes including academic

success, community involvement, arts and

recreations, life skills, sense of self and positive

choices.

20 items: I feel good about

myself. I am good at learning

new things. I do well at school.

Likert-scale

2005 Ascend

Summer Youth

Employment

Program

Developed from previous studies, measured

pre/post attitudes towards specific program goals

(gaining computer skills) as well as self-efficacy

and post-secondary career/educational skills.

I am a good student. I know

what it takes to succeed on a

job. I know about jobs or

careers I might be good at.

Yes/No



Supervisor Survey

A supervisor survey was administered within the three weeks following completion of the

program to gather information on their satisfaction with the program as well as their assessment

of how the youth performed and increased work related skills. The survey was sent to all

supervisors via Zoomerang (see Appendix H). The survey included items adapted from Detroit’s

Summer Youth Program 2010 Evaluation (Shanks et al, 2010). An email was sent from DOES to

all participating supervisors at the end of August. In order to increase the response rate, two

additional reminder emails were sent in early September as well as one week before the survey is

scheduled to close. A response rate of 9% (213 supervisors) was achieved. Analysis of the

subgroup shows that the sample that completed the survey has similar representation of the total

supervisor population with respect to organization type. All data was analyzed in Statistical

Analysis Software (SAS) version 9.2.

Validity was also assessed with the supervisor survey as it was an adapted version of an existing

survey. This was done pilot-testing of the survey occurred with a convenience sample of five

supervisors who have previously participated in SYEP, however did not this year due to

relocation of program or position. In addition, experts in the field such as SYEP staff and GWU

faculty reviewed the survey to ensure all relevant content was being captured. The individuals

were selected by the researcher through previous knowledge of them in a different capacity.

They specifically assessed the length, readability, and overall content being captured.

SYEP Records

SYEP records provided information regarding characteristics of the agencies and organizations

that host the youth, the characteristics and total enrollment of participants, program offerings,

and youth attendance. DOES provided this information to the evaluator at the start of the

program with exception to the youth attendance which was obtained at the end of the program.

Specific de-indentified information provided was the number of youth served, average number of

hours worked per week worked, school youth attends (if applicable), ward youth resides in, age

of youth, and hours worked per youth.

Qualitative Methods

To gain more in-depth information that was not captured in the survey, in-depth interviews and

focus groups with SYEP and employers as well as a small sample of non-participant youth were

conducted. All interviews and focus groups were audio recorded and were transcribed verbatim

for analysis. Once the initial data was transcribed, NVIVO version 8, a qualitative software, was

used to code the transcribed data for emerging themes and concepts. The coded data was then

reviewed to determine emerging concepts of youth and employer experiences and future

recommendations for SYEP.



Supervisor and Youth Focus Groups

Individuals engaging in these focus groups and interviews were chosen at random as well as an

email inviting all SYEP Host Agencies and youth to participate was sent. A focus group protocol

was developed with input from SYEP staff (see Appendix I). All supervisor focus groups took

place at DOES and included a brief presentation about SYEP 2011 and then break out groups to

discuss specific areas such as application process, payroll, and overall strengths and challenges.

The youth focus groups took place at DOES and Sasha Bruce Youthwork and included an

icebreaker, short presentation, and smaller breakout groups to discuss overall experiences in the

summer. Transportation money and refreshments were provided to the youth with support from

GWU. Over 100 supervisors and 50 youth engaged in the focus groups.

Supervisor Interviews

Structured interviews were conducted by phone with 35 supervisors at the completion of the

program. Interviews consisted of open ended questions addressing staff satisfaction of the

program, SYEP’s progress towards objectives from their prospective, and strengths and

challenges of program implementation (see Appendix I). DOES provided the evaluator a list

(compiled at random) of potential supervisor interview participants as well as their contact

information. The evaluator then reached out via email to these supervisors and scheduled

interviews at a convenient time for the supervisor with those that were willing to participate. The

interviews ranged from 20 to 45 minutes.

Ethical Considerations

Ethical approval to conduct this evaluation was granted by the Institutional Review Board (IRB)

of GWU School of Public Health and Health Services in Washington, DC. In addition, approval

to recruit from the SYEP program was obtained and a memorandum of understanding (MOU)

signed with CYITC and DOES. Eligible participants were identified by SYEP and all

communication took place by the evaluator via SYEP staff except for recruitment of the

supervisor interviews in which SYEP staff provided the contact information to the evaluator.

Data was analyzed and stored in a locked file on the computer and the evaluator ensured that

information related to an individual subject’s participation was protected and maintained in a

confidential manner. Informed consent was obtained from all individuals enrolled in the study as

well as their parents if they were under the age of 18 years. No investigator or supervisor

enrolled a participant without having obtained the informed consent from of the participant or

his/her legally authorized representative using an IRB and DOES approved consent document. It

was clear at all times that participation in the evaluation was voluntary and participants may opt-

out at any time. In addition, with regards to the survey, youth and supervisors had the option to

skip questions they did feel comfortable answering.



FINDINGS

A majority of the findings provided include data from the focus groups, interviews, and posttest.

The posttest was primarily used as the purpose of the report is to describe the outcomes of the

youth and supervisors post program that were not applicable prior to the start of the program (i.e.

satisfaction, supports, and placements).

Characteristics of Youth Participants

This summer, SYEP employed 13,641 youth. Table 4 describes the demographic and academic

characteristics of the participants. SYEP served about an equal number of males and females. In

addition, a majority of the youth were in high school or below (76%) and between the ages of 14

and 17 years (62%). Although youth participated from all wards, most of the participants came

from Wards 7 and 8 combined (53%). Lastly, most of the youth identified themselves as Black

(Non-Hispanic) (84%). Lastly, SYEP has a high retention rate with about 88% of the participants

returning to participate from a previous year. This high rate of retention shows that youth value

the program and want to return. Figure 3 shows visual representations of the demographic

breakdowns.

Posttest Survey Subsample

In addition, a subsample of the youth (888 youth or 7%) completed posttest. Analysis of this

subgroup of respondents shows similar representation of the total youth population with respect

to age, ward, and grade level of the youth (see Table 5). The posttest results provided further

demographic information including parental and household characteristics (see Table 6). These

characteristics can be used as an estimation of what these demographic characteristics would like

in the entire SYEP youth participant pool. Therefore, the findings provided of the survey can

also be representative of the entire SYEP youth participant population.

This subsample shows that over half of the participants (58%) live with their mother. In addition,

most of the participants lived in a household with three or more people and English was the

primary language (92%). Furthermore, the highest education level obtained by their mother or

father was high school or a GED, however more mothers had some college (23%) compared to

fathers (10%). In addition, it was found that about 70% of youth have at least one parent that is

currently employed. Lastly, about 8% of youth reported having a child, with a majority of these

teenage parents (77%) having a child between the ages of 0 to 2 years.



Table 4. Demographic and Academic Characteristics of SYEP Youth Participants*

*Data is reported for 12,651 youth. The remaining youth began the program after the evaluation had

begun (the first week of the program).

**n/r means No Response.

Characteristic Youth

(n=12,651) Percent

Gender

Male

Female

5,664

6,987

55%

45%

Age

14

15

16

17

18

19

20

21

1,736

2,049

2,060

2,051

1,869

1,339

982

565

14%

16%

16%

16%

15%

11%

8%

4%

Ward

1

2

3

4

5

6

7

8

n/r

829

258

60

1,563

2,096

1,018

3,444

3,245

138

7%

2%

1%

13%

17%

8%

27%

26%

1%

Race/Ethnicity

American Indian

Asian (Non-Hispanic)

Black (Non-Hispanic)

Hispanic/Latino

Pacific Islander

White (Non-Hispanic)

Other

n/r

61

72

10,576

385

8

36

230

1,283

1%

1%

84%

2%

<1%

<1%

2%

10%

Education Level

High School or Below

High School Graduate

GED Recipient

Left High School Before Graduating

College Student

n/r

9,631

1,066

232

186

1,530

6

76%

9%

2%

1%

12%

<1%

Prior SYEP Participant

Yes

No

11,131

1,520

88%

12%



Figure 3. Characteristics of SYEP Youth Participants



Table 5. Characteristics of SYEP Youth Survey Participants


(n=888) Percent

Gender

Male

Female

n/r

293

586

9

33%

66%

1%

Age

14

15

16

17

18

19

20

21

n/r

94

160

126

117

138

107

76

51

19

11%

18%

14%

13%

16%

12%

9%

6%

1%

Ward

1

2

3

4

5

6

7

8

n/r

40

16

7

116

96

74

183

201

155

5%

3%

1%

13%

11%

8%

21%

23%

17%

Race/Ethnicity

American Indian or Alaska Native

Asian or Pacific Islander

Black or African American

Hispanic or Latino

White

n/r

5

7

783

22

7

64

1%

1%

88%

2%

1%

7%

Highest Level of Education Completed

7th Grade

8th Grade

9th Grade

10th Grade

11th Grade

12th Grade

Graduated from High School/GED and Not in College

Freshman in College

Sophomore in College

Junior in College

Senior in College

Left College Before Completing

Graduated from College

n/r

5

59

153

162

141

154

36

81

45

20

7

4

1

20

1%

6%

17%

18%

16%

17%

4%

9%

5%

2%

1%

<1%

<1%

2% Prior SYEP Participant

New Participant

Second Summer

Third Summer

Four of More Summers

n/r

217

180

196

238

57

24%

20%

22%

27%

7%



Table 6. Family Characteristics of SYEP Youth Survey Participants


(n=888) Percent

Family Structure

Lives with Both Mother and Father

Lives with Father

Lives with Mother

Other

n/r

201

36

520

104

27

23%

4%

58%

12%

3%

Household Size (Number)

1

2

3

4

5

6

7

8 or More

n/r

13

120

205

209

146

92

36

45

22

2%

14%

23%

24%

16%

10%

4%

5%

2%

Average Age of Parent

Mother

Father

40 – 45 years

45 – 50 years

Highest Education Level of Parent

Mother

Middle School or Junior High School

High School or GED

Some College

College or Above

n/r

Father

Middle School or Junior High School

High School or GED

Some College

College or Above

n/r

35

288

223

205

137

40

333

134

94

287

5%

32%

25%

23%

15%

5%

38%

15%

10%

32%

Parent Employment (At Least One Parent Employed)

Yes

No

n/r

624

149

115

70%

17%

13%

Primary Language Spoken at Home

English

Spanish

Other

n/r

816

13

14

44

92%

1%

2%

5%

Teenage Parent

No

Yes

n/r

Age of Child

0 to 2 years

3 to 6 years

Over 10 years

Don’t know

770

79

39

61

9

8

1

87%

8%

5%

77%

12%

10%

1%



Characteristics of Job Placements and Supervisors

Job Placements

A majority of the organization sites that youth were employed at were local non-profits (CBOs)

(34%) and the District Agencies (40%). In addition, these sectors provided a majority of the

supervisors (see Table 7). This is consistent with the youth self-reporting on the survey where

most youth reported working in an office setting (15%) or government entity (15%). Also, 13%

of youth reported having engaged in educational or academic classes (see Table 8). With regards

to job placements, 88% of youth were satisfied with their job placement (See Table 9 in Youth

and Supervisor Satisfaction section below).

Table 7. Organization Types Available

*This represents the total number of spaces available at each site.

Table 8. SYEP Placements (Youth Survey Results)

Organization Type Total Hosts (%)

(n=465)

Total Supervisors (%)

(n=2,243)

Total Open Positions (%)*

(n=16,629)

DC Charter School 22 (5%) 52 (1%) 1,303 (8%)

DC Public School 13 (3%) 74 (2%) 655 (4%)

District Agency 76 (16%) 1,090 (48%) 6,692 (40%)

Federal Agency 46 (10%) 235 (10%) 791 (5%)

Non-Profit (CBO) 179 (38%) 612 (27%) 5,802 (34%)

Private Sector 129 (28%) 280 (12%) 1,476 (9%)

Organization Type Total Job Types Identified

(n=1,150) Percent

Business 100 9%

Government 176 15%

Office Work 176 15%

Mentoring or Tutoring 73 6%

Community Cleanup/Improvement 52 4%

Research or Data Collection 44 4%

Camp Counselor 102 9%

Gardening/Outdoor Maintenance 29 2%

Building Maintenance 23 2%

Academic or Educational Classes 149 13%

Other 235 21%



Job Selection Process

Most of the supervisors responding to the survey (73%) reported having identified the youth they

wanted to employ prior to the job starting. Of these supervisors 90% reported that they received

some, if not all of the youth they requested. Also, multiple supervisors expressed an

improvement this year in the process of identifying and requesting. For example, a supervisor

from a CBO stated and a supervisor from a District agency stated:

“It was a great improvement and it really helps us to be able to identify certain kids that

are interested in working for our organization and to be actually be able to interview the

kids and receive the kids we choose.”

“The biggest improvement was that the options for the screening process was a lot better”

It should be noted that an area of challenge expressed around this process was the concern that

youth need to also engage in the placement process. For example, one supervisor stated:

“The quality of the youth – the screening process was a lot better – but we need to help

youth to realize that they have to be serious and if they don’t follow suit they could lose

their employment. They need to engage in the process with us.”

In addition, youth expressed that they do in fact like to be able to choose their job placements

depending on what their interests are. For example, some youth expressed:

“They should ask us, which they do, our interests and maybe what we want to focus

on. Like say somebody wants to be a lawyer, they can work at a law firm part time,

or, me I want to do news broadcasting so I would want to be in that office, you know,

something like that. It’s a big difference.”

Another youth expressed how choosing job placements are also related to actual job skills that

they want to learn. It was stated:

“Some people in SYEP, they work at a recreation center, or a school, some were at

an actual office. And you learn different things depending where you are. Depending

on what type of person you are, what type of skills, or what you want to get from this,

where your job is at is important.”

Supervisors

A subsample of the supervisors (213 youth or 9%) completed a supervisor survey. Analysis of

this subgroup of respondents shows similar representation of the total supervisor population with

respect to type of organization (see Table 9) therefore results of the survey can be used as a

representation of the supervisors participating in SYEP. The characteristics of supervisors reveal

that most supervisors are executive directors or program managers (56%). In addition, many of

the supervisors are returning, with over half participating for three or more summers. This high



retention rate shows commitment to the program. In addition, more than half have been working

with their organization for over five years. Most of the organizations hosted one to ten youth

(47%). Lastly, a majority of the supervisors reported the main purpose of their organization was

to serve youth (25%) or educational (21%).

Table 9. Characteristics of SYEP Supervisor Survey Participants

Characteristic Supervisors

(n=213) Percent

Type of Organization

Government Agency

For-Profit Organization

Non-Profit/Community Based Organization

School/University

Other

70

22

98

21

2

34%

10%

46%

10%

<1%

Purpose of Organization

Youth

Faith

Law enforcement

Education

Community improvement or development

Research or data collection

Outdoor beautification

Arts or culture

Sales or retail

Health

Sports or recreation

Childcare

Other

84

5

9

67

44

4

2

22

9

16

3

5

66

25%

1%

2%

21%

13%

1%

1%

7%

2%

5%

1%

1%

20%

Years Worked at Organization

Less than 1 year

1 to 2 years

3 to 5 years

5 to 8 years

More than 8 years

27

40

43

44

59

12%

19%

20%

21%

28%

Role in Organization

Administration

Assistant

Executive Director/Manager

Program Manager

Youth worker

Other

43

15

59

60

9

27

20%

7%

28%

28%

4%

13%

Prior SYEP Participant

New Participant

Second Summer

Third Summer

Four of More Summers

n/r

55

50

32

75

1

26%

23%

15%

35%

1%

Age of Youth Employed

14 to 16 years

17 to 21 years

n/r

79

131

3%

37%

62%

1%

Number of Youth Employed

1 to 10 youth

11 to 20 youth

More than 20 youth

n/r

100

30

79

4

47%

14%

37%

2%



Youth and Employer Satisfaction

Overall Satisfaction

Overall, the youth and employers were very satisfied with their participation in SYEP. Over two-

thirds of the youth (69%) stated that they were very satisfied, with 29% saying they were

somewhat satisfied. Only 2% of youth stated they were not satisfied at all. A majority of the

youth (96%) stated they would participate next year. Of these youth, over half (54%) said they

would want to stay in the same job. Of the 4% of youth that said they would not participate, main

reasons included that they would be in school, have a better job, or not live in DC. Only eight

youth stated they would not participate because of a bad experience.

Likewise, 68% of supervisors stated being extremely satisfied, with the remaining 28% stating

they were somewhat satisfied, and only 4% stating they were not satisfied at all. Almost all of

the supervisors said they would participate next year (96%). In addition, 93% of supervisors

reported they would encourage others to apply to be a Host Agency.

Satisfaction with Placements

However, it should be noted that over half (59%) of supervisors stated they would not hire the

same youth they had this year. In addition, as mentioned, over half of the youth (54%) that said

they would participate next year would like the same job. Similar findings were seen in the

interviews with statements such as the following:

“I think you all are doing a great job, I am impressed. Are you are certainly doing a great

service to the city, and you are making a huge deposit in the lives in youth. I think the payoff

will be great.”

“I think that SYEP did a great job this summer and I hope that we can keep the same

employees next year – I think having the relationship with my SYEP liaison was good – and I

hope we can have the same one next year.”

However, it should be noted that most supervisors did state there is still room for improvement in

certain areas, however they were very pleased with the progress that SYEP has made in the past

two years. For example, one supervisor stated:

“I think that they were definitely trying to improve things – there were definitely things that

really make a good effective program.”

Application Process

With regards to the application, 89% of youth stated that the application process was easy and

57% of supervisors stated the Host Portal was easy to use (with 43% stating it was somewhat

easy and only 1% stating it was never easy to use). In addition, when supervisors were asked if



they had issues with the payroll system, about half (46%) stated never and (49%) stated

sometimes. Only 4% stated they consistently had problems. These findings are complemented

with the challenges that were expressed in the interviews and focus groups around the

application process, specifically with supervisors and youth workers and youth enrollment. For

example, one supervisor stated:

“The multiple steps for application made it hard for them [the youth] to apply – lines for

certification was discouraging – it is hard for kids. I know it is something they have to do

but it is hard for them.”

“The only thing that was hard was tracking down kids to make sure they had everything

in. It is hard because a lot of the kids could not find their stuff like birth certificates, etc.”

“What was hard was supporting the kids to have all of their stuff for their application – it

was hard to have the time sources to help the kids. I took a few kids about enrollment

events but even that took a lot of time – kids did not have a lot of stuff – kids do not know

how to do this stuff and it comes down on the teachers to help.”

Youth expressed similar issues, especially around the deadline and communication of the

deadline:

“I wanted to be in the program, I heard about the application process late and the

deadline was too short.”

“When I went down to the counselor’s office to talk to her about SYEP, she gives me my

application, mind you, its April, she knows that the deadline is up. I did my application,

and when I finally give it back to her, she’s all like “oh wait, the deadline is over, you

can’t get this job, because no one really informed us.”

The focus groups also complemented the supervisor survey results with regards to the Host

Portal and further satisfaction was discussed with the transparency of information on the

website. Comments included:

“It was my first year doing this and it was user friendly.”

“The portal overall worked well, everything online. They also did a much better job of

informing us with program information. This was done by email and DOES website.

Email was good and having everything posted clearly on DOES website was good.”

Safety

In addition, 80% of the youth said they felt safe at their job. However, 18% said they only

usually felt safe and 2% said they never felt safe. When asked if they felt that having this job

kept them out of trouble, a majority of youth (82%) stated it would not have.

In addition, when asked what they would have done if they had not participated, many youth said

they would have hung out with their friends, tried to find another job, or babysat.



Table 9. Youth and Employer Satisfaction

Youth Satisfaction Youth

(n=888) Percent

Easy to Apply

Yes

No

n/r

734

94

60

89%

11%

-

Satisfied with Placement

Yes

No

n/r

707

123

58

88%

14%

-

Safe at Job

Always

Usually

Never

n/r

660

145

18

65

80%

18%

2%

-

Felt Job Kept Them Out of Trouble

Yes

No

I Don’t Know

n/r

58

537

58

235

9%

82%

9%

-

Participate Next Year

Yes, at the Same Job

Yes, but at a Different Job

No

n/r

359

279

30

220

54%

42%

4%


Very Satisfied

Somewhat Satisfied

Not Satisfied at All

n/r

446

184

19

239

69%

29%

2%

-

Supervisor Satisfaction Supervisor

(n=213) Percent

Easy to Use Host Portal

Always

Sometimes

Never

I did not use the portal

n/r

108

83

1

20

1

56%

43%

1%

-

-

Issues with Payroll System

Never

Sometimes

Always

Did not use payroll system

n/r

90

95

9

17

2

46%

49%

5%

-

-

Participate Next Year

Yes

No

n/r

188

7

18

96%

4%

-

Hire Same Youth Next Year

Yes

No

I Don’t Know

n/r

37

114

43

19

19%

59%

22%

-


Very Satisfied

Somewhat Satisfied


n/r

133

56

7

17

68%

28%

4%

-



Youth and Employer Job Responsibility and Support

Orientation, Professionalism, and Preparedness

A majority of the youth (91%) stated they received an orientation (see Table 10). This is

consistent with the responses of the supervisors where 92% of supervisors stated giving a job

orientation. Of those that received an orientation, most of them consisted of a tour, introduction

to other staff members, an overview of job responsibilities, an overview of the schedule, and

explanation of the dress code. In addition, 93% of youth felt prepared for their job. When

supervisors were asked if they felt that they youth were prepared for the job, 68% stated that a

majority of their youth were and 29% stated only some of their youth were. Overall dress code

was a major challenge expressed by the supervisors. The following comments sums up thoughts

from the focus groups and interviews:

“Work place attire was a challenge. Some of the youth had to be pulled aside.”

“I continued to tell them and even provided them with a dress code and emphasized

importance. Males continued to dress inappropriately. Dress code should be standard.

Everyone should have some sort of uniform; it prepares them for the real world.”

Job Responsibilities

Furthermore, 98% of supervisors stated they had a clear understanding of their responsibilities

before the start of SYEP and furthermore, 88% felt that they were given the information

necessary to properly plan a high quality program. About 63% of supervisors stated participating

in the AYD training provided by CYITC. Of those supervisors, 83% stated that this training was

very helpful. Supervisors felt that the training helped reinforce concepts and ideas with working

with youth and it was especially beneficial to those who do not usually work with youth.

Support

With regards to support, 75% of youth stated they were very satisfied with the support they

received from their supervisors with 21% somewhat satisfied. Likewise, 88% of supervisors

stated that their questions were answered in a timely manner by SYEP staff and all of the

supervisors interviewed expressed how helpful and supportive the SYEP staff is. For example:

“Interaction between my liaison and I was great – they were very proactive, sent emails,

constant contact, that was the biggest strengths of the program.”

“Communication with the program staff was excellent. They were right on everything.”

Furthermore, over half (62%) stated that they felt that SYEP would help them later in life with

the remaining stating it would help them a little bit (31%) and only 7% stated it would not help

them at all. In addition, 58% of the youth stated that they talked to their supervisor about future

careers they were interested in.



Table 10. Youth and Supervisors Sense of Job Responsibilities and Support

Youth Job Responsibility Youth

(n=888) Percent

Orientation

Yes

No

n/r

752

74

62

91%

9%

-

Prepared for Job

Yes

No

n/r

757

59

72

93%

7%

-

SYEP Will Help Later in Life

Help Me Very Much

Help Me a Little Bit

Not Help Me at All

n/r

471

231

55

131

62%

31%

7%

-

Support By Supervisor


Somewhat Satisfied

Very Satisfied

37

169

613

4%

21%

75%

Discuss Future Careers with Supervisors

Yes

No

n/r

403

293

192

58%

48%

-

Supervisor Job Responsibility Supervisor

(n=213) Percent

Orientation

Yes

No

n/r

193

16

4

92%

8%

-

AYD Participation

Yes

No

n/r

119

69

94

63%

37%

-

Questions Answered

Yes

No

Did Not Have Questions

n/r

175

25

7

6

88%

12%

-

-

Understood Job Responsibility

Yes

No

n/r

204

5

4

98%

2%

-

Adequate Information

Yes

No

n/r

183

25

5

88%

12%

-

Youth Prepared

Majority of the Youth

Some Youth

None of the Youth

n/r

136

57

6

14

68%

29%

3%

-



Job Readiness and Work Skills

Tables 11 and 12 shows indicators related to job responsibility and work skills. The work skills

that youth reported gaining were responsibility, reporting to work on time, and communication.

This is similar to the supervisor’s assessment of the youth where they reported the top work

skills being responsibility, reporting to work on time, communication, and accepting supervision.

Supervisors also reported that the youth gained critical thinking skills and more of an

understanding of the importance of a career.

With regards to work readiness and future employment skills, youth reported having gained more

of an understanding of career interests and qualifications for future careers. Supervisors reported

that they learned good work ethic and an introduction to a professional atmosphere. In addition,

the youth learned leadership skills and working in teams.

When the supervisors were asked to do an overall assessment of all of their youth around

performance indicators at work such as arriving to work on time, following instructions,

accepting constructive criticism, working well with others, and behaving in a professional

manner, the supervisors reported that at least 75% of the youth accomplished the skills. This was

also expressed in interviews. The following quote from one supervisor summarizes themes

around work readiness that were discussed in the interviews.

“I think they achieved the main goals of having a job – getting the soft skills, calling into

work, contacting their supervisor, showing up on time, communicating with their

supervisor.”

A majority of the youth reported spending their money on something they really needed, saving

their money, or using it for food or transportation. During interviews with supervisors, money

management was a common theme that many of the expressed concerns around.

“The kids did not understand the concept of savings. They had no reason or incentive to

save. I think they need more help with money management. I know that some of them

wanted to open a bank account, but the process was actually not that easy.”

Overall, the youth and the supervisors reporting gaining work readiness and job skills this

summer.



Table 11. Job Readiness and Work Skills (Youth Assessment)

Youth Assessment Youth

(n=888) Percent

Work Skills Gained

Computer Skills

Problem-solving

Public Speaking

Accepting Supervision

Financial Management Skills

Importance of Career

Communication Skills

Organization

Reporting to Work on Time

Dressing Appropriately for Work

Completing Assignments on Time

Asking for Help

Being Responsible

Using Numbers

Other

None

195

259

289

256

152

307

391

279

398

353

278

265

431

71

101

44

5%

6%

7%

6%

4%

8%

10%

7%

10%

9%

7%

7%

11%

2%

2%

1%

Job Readiness Skills Gained

Helped Decide Job Interests

Showed me how to search for jobs

Showed me how to fill out a job application

Helped me create a resume

Prepared me for a job interview

Referred me to potential jobs

Qualifications for Future Career

Helped me arrange child care

Other

None

390

187

188

224

229

137

266

28

88

102

21%

10%

10%

12%

12%

7%

14%

2%

5%

6%

Name Three Careers

Yes

No

n/r

628

71

189

90%

10%

-

Arrive to Work on Time

Always

Sometimes

Never

n/r

578

100

1

209

85%

15%

<1%

-

Clean Appearance at Work

Yes

No

n/r

675

4

209

99%

1%

-

Manage Money

Always

Sometimes

Never

n/r

428

316

27

117

55%

41%

4%

-



Table 12. Job Readiness and Work Skills (Supervisor Assessment)

Supervisor Assessment Supervisor

(n=213) Percent

Work Skills Gained

Computer Skills

Problem-solving

Public Speaking

Accepting Supervision

Financial Management Skills

Importance of Career

Communication Skills

Organization

Reporting to Work on Time

Dressing Appropriately for Work

Completing Assignments on Time

Asking for Help

Being Responsible

Other

47

59

25

63

12

33

72

15

59

44

26

19

99

9

8%

10%

4%

11%

2%

6%

12%

3%

10%

8%

4%

3%

17%

2%

Academic Skills Gained

Reading and Writing

Math and Science

Problem-solving and Critical Thinking

Study Habits

Importance of Education

Connections to Educational Resources

Computer and Technology

53

19

85

9

57

38

47

17%

6%

28%

3%

19%

12%

15%

Future Employment Preparation

Good Work Ethic

Introduction to Professional Atmosphere

Importance of Team Work

Leadership Skills

Exposure to Possible Future Careers

98

97

55

64

80

25%

25%

14%

16%

20%

Life Skills

Sharing Ideas and Feelings

Listening to Others

Working on a Team

Making Good Decisions

Setting Goals

Being a Good Leader

Solving Problems Without Violence

Adult Support

None

59

97

128

107

112

76

56

87

23

8%

13%

17%

14%

15%

10%

8%

12%

3%

Youth Performance Assessment*

On Time to Work

Called When Late or Absent

Regular Attendance

Dressed Appropriately

Positive Attitude

Accepted Criticism

Completed Tasks Appropriately

Followed Instructions

Worked Well With Others

Asked Appropriate Questions

Behaved in a Professional Manner

Showed Initiative

177, 21, 1

151, 38, 7

171, 24, 0

177, 16, 3

175, 17, 3

163, 18, 15

145, 15, 3

169, 13, 3

172, 9, 6

158, 15, 13

160, 30, 5

141, 45, 8

89%, 11%, <1%

77%, 19%, 4%

88%, 12%, 0%

90%, 8%, 2%

90%, 9%, 1%

83%, 9%, 8%

90%, 9%, 1%

91%, 7%, 2%

92%, 5%, 3%

85%, 8%, 7%

82%, 15%, 3%

73%, 23%, 4%

*At least 75% of youth accomplished, at least 75% of youth did not accomplish, unsure



Limitations

A major issue with self-reporting is that adolescents may not answer sensitive questions honestly

and instead may exaggerate or minimize involvement in certain activities. In addition, they may

respond to what they feel like is socially desirable. However, due to the large sample size, this

method was the best fit. To minimize response bias, participants were be notified that their

responses are anonymous and confidential and that their responses would not affect their

employment. Because a pre-post test was employed, tracking is necessary. To reduce anxiety

about confidentiality, only the first letter of their first and last name was asked as well as date of

birth. In addition, survey results were supplemented with qualitative results. This helps validate

the findings.



CONCLUSION

Next Steps

This evaluation report provides successes and challenges for SYEP 2011. As seen there has been

much progress made in the past year to strengthen program offerings, recruit youth and host

agencies, provide support to youth and staff, and to increase skills learned by the youth. This

evaluation report will be used by DOES in their strategic plan for summer 2012. In addition,

elements of this report will be used as baseline data for a longitudinal study on the program.

Recommendations

Although beyond the scope of this report, some key recommendations are offered to begin the

strategic planning stages of summer 2012 including youth placements quality around host sites

and supplemental activities, diversity in participants, and collaboration. These were guided by

the findings and additional discussions in the focus groups and interviews

Consistent Quality and Supplemental Activities

SYEP has begun to incorporate activities that are age and developmentally appropriate in an

environment that engages the youth. In order to continue to build on this it is important that the

youth are in quality sites and being engaged positively. For example, worksites should undergo

site visits with regards to programming to ensure they are providing positive programming. In

addition, worksites that are returning should demonstrate how learning opportunities exists there.

Work placements should continue to be based on each youth’s interests, education, and career

goals. In addition, a recommendation suggested by a supervisor to alleviate issues that may arise

with multiple youth placements is that once a selection of a youth is made, the youth should be

removed from the roster until the Host Agency states they are not able to take the youth or the

youth declines.

In addition, more intentional job placements should occur around youth interests and

expectations of the youth. This could be done through mandatory workshops before being placed

at a worksite where participants can learn about the goals of SYEP, different jobs what is

expected of them, and worksite rules and responsibilities. These workshops could continue on

quarterly with the help of the Host Agencies and include topics like college planning, life skills,

and leadership skills.

As SYEP is working with youth who are developing and learning skills to help navigate their

transition into adulthood, Program elements should be supplemented by other program activities,

including community service, career portfolio development, money management, career and



educational exposure events, and youth leadership development. Community service should also

be promoted as this can further expose youth to career interests and long-term opportunities.

Diversity in Participants

Although SYEP has made progress in diversifying their participants and target youth from at-risk

areas these efforts should continue to recruit youth who are at higher levels of negative risk

factors such as dropping out of school and poverty. In addition, SYEP should continue to make a

concerned effort to reach those youth who are disengaged such as youth who have dropped out

of high school. SYEP could utilize their offerings to reengage the youth. For example, selected

spots could be reserved for these youth and coupled with a GED or credit recovery programs to

help reengage them. In addition, SYEP should continue to partner with schools to assist with the

application process.

Collaboration

There are multiple agencies in DC that provide, fund, and oversee youth education, training, and

employment services (Ross, 2011). However, the different funding streams and performance

measures often cause a lack of collaboration and ability for data sharing (Ross, 2011). The main

agencies that play a role in youth development employment services in the district can be

categorized by Policy/Oversight, Education, and Funding/Service Providers.

Policy/Oversight Agencies

The main policy and oversight agencies in DC include the Workforce Investment/Youth Council

(WIC/YIC), Office of the State Superintendent for Education (OSSE), Office of the Deputy

Mayor for Education (DME), and the Public Charter School Board (PCSB). WIC/YIC is

mandated by the Federal Workforce Investment Act to oversee workforce development and

policy, however have been inactive or unsuccessful in the past decade (Ross, 2011). OSSE and

PCSB set policies, apply oversight, and direct resources to ensure quality education and

resources to DC youth. Lastly, DME oversees the development and implementation of an

Education and Youth Development Plan, a comprehensive overview and plan of the current

education and youth development policy and practice within DC as well as recommendations for

future youth policies and regulations.

Education Agencies

The key education players include DCPS, DC Public Charter Schools (DCPCS), and the

Community College of the District of Columbia (CCDC). DCPS and DCPCS offer a variety of

special initiatives and programs to engage those enrolled as well alternative programs for those

off-track (Ross, 2011). CCDC offers academic and career-focused associate degrees and



workforce development programs for high school graduates who need further study to become

college-ready (Ross, 2011).

Funding/Service Providers

The key employment funding and service providers include CBO’s, CYITC, and DOES. Their

main functions with regards to youth employment are discussed above.

By pulling in collaboration with all of these agencies, a strategic plan with regards to youth

workforce development can be achieved and involve not only summer programming but ongoing

year round opportunities. SYEP has already begun to make efforts to collaborate with some of

these agencies, and by continuing these efforts; the program can leverage the resources and

opportunities available. For example, working with other agencies such as OSSE can streamline

recruitment and documentation that they require for programs such as DC Tuition Assistance

Grants. SYEP should also reach out to local universities to provide assistance with not only the

implementation of the program but also to provide resources and opportunities such as college

tours and career fairs. Lastly, reaching out to local banks such as PNC and Bank of America to

help youth open accounts and teach money management skills.

Conclusion

Although there are limitations, this report does add to the current research of the effects of

summer youth employment programs on youth development specifically in DC. In addition, this

report focused on a large sample of urban youth. Few studies regarding youth employment have

utilized a similar sample without constraining their analysis across races or ethnicities (Johnson,

2004). A 2009 report by the Wallace Foundation stated that there was a lack of evidence found

for youth development outcomes because those outcomes were rarely, if ever, evaluated

(Terezen, Anderson & Hamilton, 2009). Moreover, a majority of the studies that have evaluated

specifically summer youth employment programs focus on process evaluation and not behavior

change in participants. Programs are not trained to conduct comprehensive evaluation and use

simple research methods to develop quick information about the program and its function. This

evaluation combines academic research with a practical model for evaluations of summer youth

employment programs. By combining academia with the youth programming field, it allows for

researcher to learn the needs, language, and culture while sharing evidence-based practices. As

mentioned a comprehensive report will be available in May 2012 which explores youth behavior

change and process for replication of SYEP for other cities.



DEFINITION OF TERMS

For the purpose of this report, the following definitions will be used:

Employability Skills: Set of skills, knowledge and personal attributes that make an individual

more likely to secure a job and be successful in the workforce.

Future Orientation: One’s expectations and the degree to which one is thoughtful about their

future.

Higher Education: Two or four year college/university

Job Readiness Services: Services that include career awareness and exploration activities,

general job training activities, resume, cover letter, and interview preparation, and general

workforce readiness and skills.

Low-Income: An individual whose family’s taxable income for the preceding year did not

exceed 150 percent of the poverty level amount.

Work Ethic: Demonstrating personal accountability and effective work habits such as

punctuality, working productively with others, time and workload management, and

appropriate dress.

Youth: The entire adolescent period, from ages 10 until 24 years.

Youth Development: The physical, social, and emotional processes that occur during the

adolescent period, from ages 14 until 24 years.

Youth Development Programs: Programs focusing on fostering the skills necessary for

personal, social, and career-related success.

Urban: Belonging to a densely populated city.



REFERENCES

Allen, M. (2006). A formative program evaluation of Bridgepoint’s great expectations.

Louisville, KY: Spalding University.

America’s Promise. (2011). America’s Promise: Parent engagement. Retrieved from

http://www.americaspromise.org/Resources/ParentEngagement.aspx.

Annie Casey Foundation. (2011). Kids count online database. Retrieved from

http://www.aecf.org/MajorInitiatives/KIDSCOUNT.aspx.

Bauermeister, J., Zimmerman, M., Gee, G., Caldwell, C. & Zue, Y. (2009). Work and sexual

trajectories among African American -youth. Journal of Sex Research, 46(4), 290-300.

Bernat, D. & Resnick, M. (2006). Healthy youth development: Science and strategies. Journal of

Public Health Management and Practice, 12(6), 10-16.

Breinbauer, C. & Maddaleno, M. (2005). Youth choices and change. Washington, DC: PAHO.

Cardoza, K. (2011). Summer program cuts will affect thousands of students. Accessed from

http://wamu.org/news/11/05/26/summer_program_cuts_will_affect_thousands_of_dcps_students

.php.

Casner-Lotto, J. & Barrington, L. (2006). Are they really ready to work? Employers’

perspectives on the basic knowledge and applied skills of new entrants to the 21st century US

workforce. New York, NY: Conference Board.

© 2006 by

Centers for Disease Control and Prevention. (2010). Factors that contribute to health disparities.

Accessed from http://www.cdc.gov/HealthyYouth/disparities/black/factors.htm.

Chaplin, D. (1999). Summary of capacity and needs assessments: Youth activities in the District

of Columbia. Washington, DC: Urban Institute.

Clymer, C., Edwards, K., Ponce, J. & Wyckoff, L. (2002). Supporting youth employment: A

guide for community groups. Philadelphia, PA: Public/Private Ventures.

Curnan, S., Kingley, C., LaCava, L. & Frees, J. (2010). Unfinished work: Building excellence in

Washington, DC’s summer youth employment program. Waltham, MA: Brandeis University.

Double the Numbers. (2006). Double the numbers for college success: A call to action for the

District of Columbia. Accessed from http://www.doublethenumbersdc.org.

Finkelhor, D. & Ormrod, R. (2010). The homicides of children and youth. Washington, DC: U.S.

Department of Justice, Office of Juvenile Justice and Delinquency Prevention.



First Focus. (2010). Children’s budget 2010. Washington, DC: First Focus.

Fitz-Gibbon, C. & Morris, L. (1987). How to design a program evaluation. Newbury Park, CA:

Sage Publications.

Flannery, D., Hussey, D. & Thomas, R. (2009). 2009 Summer youth employment report.

Accessed from

http://www.youthopportunities.org/images/MyCom%20Summer%202009%20Report.pdf.

Greenberger, E. & Steinberg, L. (1986). When teenagers work: The psychological and social

costs of adolescent employment. New York, NY: Basic Books.

Harvard Graduate School of Education. (2011). Pathways to prosperity: Meeting the challenge of

preparing young Americans for the 21st century. Boston, MA: Pearson’s Foundation.

Hastings, S., Tsoi, R. & Harris, L. (2010). Building a comprehensive youth employment delivery

system: Examples of effective practice. Washington, DC: CLASP.

Jekielek, S., Cochran, S. & Hair E. (2002). Employment programs and youth development: A

synthesis. Washington, DC: Child Trends.

Johnson, M. (2004). Further evidence on adolescent employment and substance use: Differences

by race and ethnicity. Journal of Health and Social Behavior, 45, 187-197.

Kuehn D. & McDaniel, M. (2009). Vulnerable youth and the transition to adulthood fact sheet:

low-income African American youth. Washington, DC: Urban Institute.

Land, K. (2010). The 2009 foundation for child development child and youth well-being index

report. Durham, NC: Duke University.

Lippman, L. & Keith, J. (2009). A developmental perspective on workplace readiness: Preparing

high school students for success. Washington, DC: Child Trends.

Mael, F., Morath, R. & McLellan, J. (1997). Dimensions of adolescent employment. The Career

Development Quarterly, 45, 351-367.

Martinson, K. (2010). Partnering with employers to promote job advancement for low-skill

individuals. Washington, DC: The National Institute for Literacy.

Matsuba, K., Elder, G., Petrucci, F. & Marleau, T. (2008). Employment training for at-risk

youth: A program evaluation focusing on changes in psychological well-Being. Child Youth

Care Forum, 37(15), 15-26.

McCalahan, W., Sipe, C. & Smith, T. (2004). An evaluation of the summer career exploration

program. Philadelphia, PA: Public/Private Ventures.



McLaughlin, J. & Sum, A. (2011). The steep decline in teen summer employment in the US,

2000-2010 and the bleak outlook for the 2011 summer teen job market. Boston, MA: Center for

Labor Market Studies.

Mishel, L., Bernstein, J. & Allegretto, S. (2005). The state of working America: 2004/2005.

Ithaca, NY: Cornell University Press.

Mortimer, J., Zimmer-Gembeck, M., Holmes, M. & Shanahan, M. (2002). The process of

occupational decision-making: Patterns during the transition to adulthood. Journal of Vocational

Behavior, 61, 439-465.

Mortimer, J. (2003). Working and growing up in America. Cambridge, MA: Harvard University

Press.

National Recreation and Park Association. (2010). The key benefits: Synopsis of 2010 research

paper. Ashburn, VA: National Recreation and Park Association.

Nielsen, N. & McGhee, R. (2005). Ascend summer youth employment program 2005.

Washington, DC: SRI International.

Organisation for Economic Co-operation and Development. (2009). Jobs for youth. Accessed

from http://www.oecd.org/dataoecd/22/3/44031073.pdf.

Pennsylvania Partnerships for Children. (2011). The state of youth employment. Accessed from

http://www.psaydn.org/Documents/Employment_fact_sheet.pdf.

Public/Private Ventures. (2005). Youth Development: Issues, Challenges, and Directions.

Philadelphia, PA: Public/Private Ventures

Ross, M. (2009). Blog: The district’s dime: Youth employment programs should focus on

quality, not just numbers served. Washington, DC: Brookings Institute.

Ross, M. (2011). Strengthening educational and career pathways for DC youth. Washington, DC:

Brookings Institute.

Shanks, T. & McGee, K. (2010). Detroit summer youth employment program: Results of

employer and youth employee exit surveys. Ann Arbor, MI: University of Michigan.

Simpkins, S. (2003). Does youth participation in out-of-school time activities make a difference?

The Evaluation Exchange, 9(1).

Staff, J. & Mortimer, J. (2007). Educational and work strategies from adolescence to early

adulthood: Consequences for educational attainment. Social Forces, 86(3), 1169-1194.

http://www.dcfpi.org/category/blog



Sum, A., Khatiwada, I., McLaughlin, J. & Palma, S. (2008). The collapse of the national teen job

market and the case for an immediate youth jobs creation program. Boston, MA: Center for

Labor Market Studies at Northeastern University.

Terezen, M., Anderson, M. & Hamilton, K. (2009). Effective and promising summer learning

programs and approaches for economically-disadvantaged children and youth. Washington, DC:

Wallace Foundation.

The Colorado Trust. (2004). After-school initiative’s toolkit for evaluating positive youth

development. Denver, CO: The Colorado Trust.

Thomas, S. (2000). The distribution of dropout and turnover rates among urban and suburban

high schools. Sociology of Education, 73, 39-67.

Urban Alliance. (2010). About us: Statistics. Accessed from

http://www.theurbanalliance.org/about/statistics.

US Census Bureau. (2011). FactFinder: District of Columbia. Accessed from

http://factfinder2.census.gov/.

US Department of Health and Human Services. (2010). Positive youth development fact sheet.

Accessed from http://www.acf.hhs.gov/programs/fysb/content/positiveyouth/factsheethtm.

Weeter, C. & Martin, N. (2011). Building roads to success: Key considerations for communities

and states reconnecting youth to education. Washington, DC: National Youth Employment

Coalition.

Whalen, S., DeCoursey, J. & Skyles, A. (2003). Preparing youth for the workforce: Exploring

employer engagement in the Chicago region. Chicago, IL: Chapin Hall Center for Children.

Zuckerman, A. (n.d.). The more things change, the more they stay the same: The evolution of

youth employment programs. Washington, DC: National Youth Employment Coalition.