Upload
arnold
View
32
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Post School Outcomes: What Can We Learn from Trend Data?. Pattie Johnson, WOU Charlotte Y. Alverson, UO Building Capacity Institute, 2013. Session Description. Oregon has three years of PSO data with consistent definitions for educational and employment outcomes. - PowerPoint PPT Presentation
Citation preview
Post School Outcomes: What Can We Learn from Trend Data?
Pattie Johnson, WOUCharlotte Y. Alverson, UO Building Capacity Institute, 2013
Session Description•Oregon has three years of PSO data with
consistent definitions for educational and employment outcomes.
•What can we learn from examining the trends?
•Where are the key areas that districts can use to evaluate progress toward increased engagement for their students?
•Where will program changes have impact on improving performance?
Consistency•Since FFY 2008, states have had the same measure and definitions for Indicator 14, post-school outcomes.
•With FFY 2011 data collection, we have 3 to 4 years of PSO data.
•We can now start to examine trends in outcomes across years.
Data Collected Yearly by StatesStates are measured on their implementation of IDEA through 20 Part B Indicators.
#14: Percent of youth who are no longer in secondary school, had IEPs in effect at the time they left school, and were:1. Enrolled in “higher education”2. In “competitive employment”3. Enrolled in “other postsecondary education
or training”4. In “some other employment”
5Outcomes for Student with Disabilities
as Measured by Indicator 14
0.0
20.0
40.0
60.0
80.0
100.0
26.8
56.3
72.5
29.0
57.2
72.5
Median Percentage for Each Measure National FFY 2009 National FFY 2010
Perc
ent
of R
espo
nden
ts
United States Department of Education, Office of Special Education Programs (2011, 2012). Part B State Performance Plan/ Annual Performance Reports 2011 & 2012 Indicator Analyses .
Indicator 14 for Federal Reporting
1 HE 2 CE 3 OEd 4 OW 5 NE
We will look at the five outcome categories because these are more meaningful for
understanding our data and being able to use our data for program improvements
Questions Guiding the Analysis •How representative are these data? •What direction are our outcomes going?•Are there differences in outcomes by
subgroups? ▫Gender: Male, Female▫Disability: ID, ED, SLD, all other▫Method of Exit: Regular diploma, Completed, Dropout▫Ethnicity: Minority, Caucasian
•What is contributing to our outcomes?•How can we use the information?
Looking at Data • How representative are these data?
▫Aggregate of response representativeness• What direction are our outcomes going?• Graphs of:
▫Overall A, B, & C Measure x 3 years ▫Overall 1, 2, 3, 4, & 5 x 3 years
• Are there differences in outcomes by subgroups? • Gender Disability Method of Exit and Ethnicity categories x 3
years • What is contributing to our outcomes?
• What supplemental survey questions will help answer this question?
PSO in Oregon• 1-year prior to conducting the survey, districts
can collect accurate contact information on exiting students
• All Local Education Agencies (LEAs) collect follow up data, larger LEAs are provided with prioritized list of leavers selected to achieve a representative sample of leavers based on race, disability, gender, and method of exit
• LEA personnel conduct phone interviews
• Responses are recorded in online secure website
9
How representative are these data?
Representativeness: Basic Numbers from Three Years
School year 2008-09 2009-10 2010-11Interview year 2010 2011 2012Total Leavers 4295 4425 4244Selected for interview 2770 2779 2714Completed interviews 1911 1989 1748Response rate 68.9% 71.6% 64.4%
Oregon uses a stratified sample: • All districts conduct interviews with students each year• Small districts (15 leavers or less) interview all leavers• Larger districts are provided with a sample of required students to
interview.
NPSO Calculator Representativeness: Combining three years of data
Overall LD ED MR AO Female Minority Dropout
Target Leaver Totals 12974 6630 1246 1053 4045 4410 3198 3015
Response Totals 5648 2840 497 461 1850 1907 1304 961
Target Leaver Representation 51.1% 9.6% 8.1% 31.2% 34.0% 24.7% 23.2%Respondent Representation 50.3% 8.8% 8.2% 32.8% 33.8% 23.1% 17.0%
Difference -0.8% -0.8% 0.1% 1.6% -0.2% -1. 6% -6.2%
Dropouts are under underrepresented - a finding consistent with the each years’ separate response analysis. Importance: to ensure sampled group represents state population, the difference should be 3% or less. Caution should be used in interpreting any results using the dropout category.
What direction are our outcomes going?
▫Overall A, B, & C Measures▫Overall 1, 2, 3, 4, & 5 categories
National and OR State PSO Data
Measure A Measure B Measure C 0
20
40
60
80
100
27
56
72
24
51
66
25
54
68
25
55
72
National FFY 2009 State FFY 2009 State FFY 2010 State FFY 2011
Indicator 14 Measure
Perc
ent
of Y
outh
Data Source: National aggregate of FFY 2009 SPP Submitted February 1, 2011; State data reported in the SPP FFY 2009 & APR FFY 2010, 2011
14
What direction are our outcomes going: Outcomes by Three Years
Hi Ed
Comp E
mpl
Other
Scho
ol
Other
Emp
Not Eng
aged
0
20
40
60
80
100
2009 2010 2011
Outcome Category
Perc
ent
of R
espo
nden
ts
What do we see in the trends?• Higher Education initial increase, then static• Competitive Employment Increasing
▫More Oregon leavers employed than in education• Other School and Other Work relatively
unchanged▫Dip in middle year- 2010▫Slightly more leavers in Other Work than Other
Education• Not Engaged rate decreasing – right direction• Why look further?
Are there differences in outcomes by subgroups?
Gender x 3 years Disability categories x 3 years Ethnicity categories x 3 years Method of Exit x 3 years
Hi Ed CompEmp OthSch OthWork NE0
20
40
60
80
100
Engagement Males
2010 M 2011 M 2012 M
Differences in Outcomes by Gender
Hi Ed CompEmp OthSch OthWork NE0
20
40
60
80
100
Engagement - Females
2010 F 2011 F 2012 F
Perc
ent R
epor
ted
Observations for Outcomes by Gender
• More Females than Males in Hi Ed and both groups have fairly static trend
• More Males than Females in Competitive Employment with increase in trend for Males
• Other School engagement about the same rate for Females and Males
• Other Employment similar rates, but Females have increasing trend over time
• Not Engage decreasing trend for both groups
Differences in Outcomes by Disability Categories: SLD and ED
2009 2010 20110
20406080
100
Specific Learning Disability
1 HE 2 CE 3 Oed 4 OW 5 NE
2009 2010 20110
20
40
60
80
100Emotional Disturbance
Differences in Outcomes by Disability Categories: All Other (Low Incidence) and
ID
2009 2010 20110
20406080
100Intellectual Disability
2009 2010 20110
20406080
100
Low Incidence
1 HE 2 CE 3 Oed 4 OW 5 NE
Observations for Outcomes by Disability• SLD: slight positive trend for HE and CE and slight
negative trend for NE – trends going in desired direction• ED: negative trend in HE, but positive trend in CE;
negative trend in NE- need to explore HE • AO/Low Incidence: Slight increase in CE, other
engagement categories unchanged• ID: negative trend in HE, positive trend in all other
engagement categories; highest group NE, but decreasing
• Regardless of disability, about 1/3 of respondents are NE, HOWEVER, the trend is headed in the desired direction – seeing a negative trend in all disability categories - need to explore NE
Differences in Outcomes by Race/Ethnicity Categories
2009 2010 20110.0
20.0
40.0
60.0
80.0
100.0 Minority
2009 2010 20110.0
20.040.060.080.0
100.0
White
1 HE 2 CE 3 Oed 4 OW 5 NE
Combined Years for Sufficient Size of Race/Ethnicity Subgroups for Comparison
Asian n=128
Black n=185
Hispanic n=725
Nat Amer n=210
White n
=4348
0
20
40
60
80
100
Combined Three Years
1 HE 2 CE 3 Oed 4 OW 5 NE
Race/Ethnicity Categories
Perc
ent
of R
espo
nden
ts
Observations for Outcomes by Race/Ethnicity
• There are very minor outcome differences when all minorities are in one subgroup
• There are insufficient interviews to look at trends over the 3 years by each minority group
• Combining the data from the three years, as a representative sample was interviewed, allows further exploration
• Differences are seen in the students combined into the minority subgroup that need to be explored further by the state and districts
Outcomes by Method of Exit category Regular Diploma
2009 2010 20110
20
40
60
80
100
Regular Diploma
1 HE 2 CE 3 Oed 4 OW 5 NE
Outcomes by Method of Exit category Dropout
2009 2010 20110
20
40
60
80
100
Dropout- Not representative
1 HE 2 CE 3 Oed 4 OW 5 NE
Observations for Outcomes by Method of Exit Groups: Regular Diploma and Dropout
•Regular Diplomas: slight increases in HE and CE, and decrease in NE; trends are going in the desired directions
•Dropouts – not representative of the state▫Decrease in HE, slight increase in CE with a
dip in 2010▫Slight increase in Other Education; static in
Other Work▫Decrease in NE
Summary Observations•Not Engaged – negative trend
▫Rate is slow▫High number of youth in some
subgroups ID Dropout
•Higher Ed and Competitive Employment shifting to Other categories▫ED – negative trend on HE with
increase in Other Education
What is contributing to our outcomes?
Supplemental survey questions can help answer this question.
•Additional questions included on the follow- up interview in Oregon:▫Do you have a drivers license?▫What is your living situation?▫What one thing would you tell your school?▫Which independent activities can you do? ▫Do you receive benefits like co-workers?▫What do you do for recreation?▫If you haven’t worked, why not?▫Have you received support from adult
Agencies?
What Agency Services have you accessed since leaving school?The list of agencies on the follow-up
interview includes:▫Social Security Disability Insurance or
Supplemental Security Income ▫Developmental Disability services▫Office of Vocational Rehabilitation▫Temporary Assistance for Needy Families▫Supplemental Nutrition Assistance Program▫College Disability Services▫Loans, Financial Aid
Change in Outcome Classification
•For the next series of charts, the outcome groups were modified to allow a closer look at students who tried school or work, but were not successful.▫1 HE 2 CE 3/4 Other ed/work 5-Attempted 5-
None▫5-Attempted: students answered Yes to either
school/ training or employment, but did not continue long enough to qualify as ‘engaged’
▫5-None: students reported NO school/training or employment experience
Differences in Outcomes by Agency Services Received
2009 n=1695 2010 n=1792 2011 n=15560
20
40
60
80
100
21 27 2626 20 221911 912 17 17
No VR Services
2009 n = 216 2010 n = 198 2011 n = 1920
20406080
100
25 25 2526 30 3117 11 1117 21 17
Vocational Rehabilitation Services1 HE 2 CE 3/4 5 -Attempted 5 None
Differences in Outcomes by Agency Services Received
2009 n=205 2010 n=180 2011 n=1860
20
40
60
80
100 Developmental Disability Services1 HE 2 CE 3 or 4 5 Attempted
2009 2010 20110
20
40
60
80
100No DD Services
How Can We Use These Data: Next Steps
•Finalize the analysis ▫Other supplemental questions
•Share and discuss trend data▫ODE Transition Specialist▫Transition Advisory Council Stakeholders▫Agency Partners: ODDS and VR
•Determine what data to share and how to share ▫District and School Stakeholders
Looking at Data: Process summary • How representative are these data?
▫We explored the response size and how the subgroups matched the population
• What direction are our outcomes going?▫ Looked at graphs showing performance, trends, and
comparisons • Are there differences in outcomes by subgroups?
• Worked from general overview to more specific components• What is contributing to our outcomes?
• Looked at a combination of components, modified the question if necessary, and summarized what we learned at each step
•For more information:
▫Pattie Johnson Teaching Research Institute, Western Oregon University [email protected] 503-838-8779
▫Charlotte Y. Alverson National Post School Outcomes Center, University of Oregon [email protected] 541-346-1390