EOC Delphi Survey and University Assisted Community Schools

EOC DELPHI SURVEY AND UNIVERSITY ASSISTED COMMUNITY SCHOOLS EVALUATION Amy Lanza Mentor: Gretchen Suess, PhD

AGENDA The Netter Center Evaluation of Social Programs Aims Methods Findings Role in Project Lessons Learned

THE NETTER CENTER FOR COMMUNITY PARTNERSHIPS Founded in 1992 Benefits West

Philadelphia, the University and society at large

Redefines university-community relations

Source: Netter Center 2012

PERSPECTIVES “The Penn students

become more concerned with the problems that afflict our world - and they get a better set of tools to help solve those problems. We don’t just do a project and move on. We remain involved with the community on an ongoing basis.” - Frank Johnston,

Anthropology Professor


UNIVERSITY ASSISTED COMMUNITY SCHOOLS (UACS)

7 schools in West Philadelphia

Sustainable, mutually beneficial partnerships

Community problem solving built into school curriculum and after school projects


UACS Penn students

involved through Academically Based Community Service (ABCS) courses, internships, work-study, and volunteer opportunities


EXAMPLES OF NETTER CENTER PROGRAMS

Agatston Urban

Nutrition Initiative Moelis Access Science College Access and

Career Readiness Community School

Student Partnerships


EVALUATION OF PENN PARTNERSHIPS Social program feedback

necessary Evaluation essential to

ensuring success and that programs can adapt to ever-changing needs in community

Multi-disciplinary approach taken by UACS - establishing core set of evaluation metrics can be challenging

Source: Cook, et al. 1990

EVALUATION OVERSIGHT COMMITTEE Established in 2009 Diverse group of scholars invested in Penn

community brought together to improve evaluation

Faculty subcommittees of EOC: Small but diverse group of faculty Help the Netter Center to develop broad foundational

research and evaluation questions From Penn Med, GSE, CHOP, Nursing, Design, SP2,

Urban Studies, Population Studies, Wharton…

Source: Harkavy 1996

INTENDED IMPACTS - BROADLY DEFINED 1) Education: “employability,” college-going/post-

secondary graduation rates, critical thinking/higher order learning among students, academic achievement, school climate/safety …

2) Human health: life expectancy, asthma, blood lead levels, obesity/underweight/healthy BMI, nutrition, physical activity, mental health, development …

3) Environmental health: environmental contaminants in the home, pollution/wastes, use of solid/biomass fuels, green space, community infrastructures, access to quality education and health care …

4) Civic engagement and partnerships: focus intellectual resources and problem-based learning on local need/social change, increase knowledge and access to such knowledge, expose more Penn faculty and students to the realities of urban poverty and social inequality …

AIMS The goal of the EOC

team: to establish a core set of measures to evaluate the impact of University Assisted Community schools on the overall health and well-being of youth and adults in the West Philadelphia community Source: http://en.wikipedia.org/

METHODS

THE DELPHI SURVEY TECHNIQUE

Source: http://trendsoutheast.org/


ROUND 1: DEVELOPING RESEARCH QUESTIONS AND INDICATORS FOR EVALUATION

Get perspectives from all faculty on important research/evaluation questions and indicators

Created and distributed surveys through Qualtrics


ANALYZING RESPONSES IN ATLAS


ROUND 2: ESTABLISHING PRIORITIES


ROUND 3: FINALIZING PRIORITIES A few indicators…

Human health Levels of substance abuse Rates of teen pregnancy Reported rates of bullying

Environmental Health # of safe parks and play spaces for children and community

gatherings Education

High school graduation rates Program design and outcomes are evaluated based on quality

impact over number served Civic Engagement

Youth have access to high quality education that provides them with networks, learning, and tools needed to provide them with opportunities as adults

Penn student, faculty, and staff satisfaction with Netter Center programming and ABCS courses

NEXT STEPS NCCP evaluation Faculty research Desire to expand evaluation beyond individual

schools and programs How can UACS programming address broader issues

in the community? EOC Subcommittee Final Culminating Meeting

ROLE IN PROJECT Analyzed Delphi Survey Technique results with

ATLAS Conducted literature/research reviews Generated preliminary findings Planned long-term research plan centered on

impact areas of: Education Human Health Environmental Health Civic Engagement

LESSONS LEARNED Possible limitations of project

Not enough representation in each impact area Persistence in reaching out to faculty Small sample (but still diverse results)

ATLAS, Qualtrics Analysis of data Literature reviews Importance of evaluation

THANK YOU! Dr. Gretchen Suess The EOC Evaluation Team at the Netter Center

Kimberly Kolor Sneha Sabnani Joseph Boselovic Hayley Germack

QUESTIONS/COMMENTS?

Documents

EOC Delphi Survey and University Assisted Community Schools