Performance Audit - Wa

Repor t Number: 1021127

At the request of the Legislature, the Office of the Washington State Auditor is conducting a long-term audit of educational outcomes for students enrolled in Alternative Learning Experience (ALE) programs. In the course of audit work, our Office concluded that student outcomes could not be measured due to poor data quality. Auditors reached out to individual ALE programs to examine whether they are meeting their intended purpose: to give schools flexibility they need to serve a diverse student population. This outreach included surveying all ALE programs, visiting ALE programs that reported innovative approaches to delivering educational experiences, and holding focus groups on how ALE instruction addresses the needs of certain student populations. Program staff and participants described the role ALE programs fill within their respective districts, current challenges, and the reasons students enroll in these programs. However, because of ongoing data quality issues, the effectiveness of the programs could not be independently verified. This report makes recommendations to the Office of Superintendent of Public Instruction (OSPI) intended to improve school districts’ compliance with ALE reporting requirements and to provide quality data on long-term outcomes for ALE students.

Performance AuditAlternative Learning Experience (ALE) in WashingtonStudent Characteristics and Innovative Approaches

April 11, 2018

O�ce of the Washington State AuditorPat McCarthy

Table of Contents

ALE: Student Characteristics and Innovative Approaches | 2

The mission of the Washington State Auditor’s Office The State Auditor’s Office holds state and local governments accountable for the use of public resources. The results of our work are widely distributed through a variety of reports, which are available on our website and through our free, electronic subscription service. We take our role as partners in accountability seriously. We provide training and technical assistance to governments and have an extensive quality assurance program.For more information about the State Auditor’s Office, visit www.sao.wa.gov.

Americans with DisabilitiesIn accordance with the Americans with Disabilities Act, this document will be made available in alternative formats. Please email [email protected] for more information.

To request public recordsPublic Records Officer 360-725-5617, [email protected]

State Auditor’s Office contactsState Auditor Pat McCarthy 360-902-0360, [email protected] Frank – Director of Performance Audit 360-902-0376, [email protected] Good – Principal Performance Auditor 360-725-5615, [email protected] Weber – Senior Performance Auditor 360-725-5419, [email protected] Patrick Anderson – Performance Auditor 360-725-5634, [email protected] Cimber – Performance Auditor 360-725-5430, [email protected]

Kathleen Cooper – Deputy Director for Communications 360-902-0470, [email protected]

Executive Summary ...................................................................................................3

Introduction .................................................................................................................6

Background ..................................................................................................................7

Scope & Methodology ............................................................................................11

Audit Results ..............................................................................................................13

Recommendations ...................................................................................................34

Agency Response .....................................................................................................35

Appendix A: Initiative 900 .....................................................................................37

Appendix B: Methodology ....................................................................................38

Appendix C: Statewide Results from ALE Program Survey .......................40

Appendix D: Summary of Telephone Focus Group Findings....................41

Appendix E: ALE Program Profiles ......................................................................44

https://portal.sao.wa.gov/saoportal/Login.aspx

http://www.sao.wa.gov/Pages/default.aspx

ALE: Student Characteristics and Innovative Approaches :: Executive Summary | 3

Executive Summary

All children are entitled to receive a basic education under the Washington State Constitution. Alternative Learning Experiences (ALE), like alternative high schools and charter schools, are one of a variety of options offered by public school districts outside of traditional school settings. ALE is a program or method of delivering instruction that any school, including charter and alternative schools, could choose to use; it may be an option within an existing public school or an entire school in itself. What distinguishes ALE from other public school options is its distinctive requirements for claiming state basic education funding. ALE programs allow school districts to claim funding for students who are not physically present in a classroom for a full day or class period.In 2013, the Legislature approved significant changes to the rules governing ALE programs. Engrossed Substitute Senate Bill (ESSB) 5946 was intended to encourage greater accountability and transparency in ALE programs statewide. It required the Office of the Washington State Auditor to conduct a biennial measure of student outcomes and financial audits of ALE courses, beginning with the 2013-14 school year and continuing through 2016-17. In addition to the financial audits, the Office decided to examine student outcomes through a long-term performance audit. A preliminary audit report, published in February 2015, and the first of three planned reports, published in February 2016, described ALE programs and students. These reports demonstrated that the ALE population differs from the traditional student population in many ways – the two groups are not directly comparable. The second audit phase built on previous work, exploring the role ALE plays for certain student populations.

First report revealed data quality issuesIn the first phase of this audit, auditors were unable to identify all ALE students in the Comprehensive Education Data and Research System (CEDARS), managed by the Office of Superintendent of Public Instruction (OSPI), because not all districts reported which students were ALE-funded. Determining who has received ALE instruction and who has not is essential to connecting students with their individual outcomes. The report made recommendations to OSPI to address these concerns. However, follow-up on these recommendations revealed compliance with ALE reporting requirements in CEDARS remains poor.

School district reporting of ALE enrollment is inconsistentThe current phase of this long-term audit planned to assess academic and post-high school progress for 2013-14 ALE students using CEDARS student data, along with post-high school outcomes from the Education Research and Data Center (ERDC)’s data warehouse. However, when CEDARS data was compared to the monthly reports that school districts submitted separately for funding, auditors found that 65 percent of districts reported fewer students enrolled in ALE courses in CEDARS than they claimed for funding in their fiscal report.

What is a “measure of student outcomes”?Examples include:

• Months of continued enrollment in school

• Graduation • College enrollment• Stable employment

School sta� enter data

Substantially incomplete

CEDARS student

information system

Enrollment report(P-223)

Audit compared

Substantially complete

ALE student data does not align in two key reporting systems

Student outcomes cannot be measured until individual student information can be reliably identified

ALE student data does not align in two key reporting systems


To identify some potential causes of reporting inconsistencies between the fiscal data and the data in CEDARS, auditors contacted eight school districts of various sizes and locations that either over-reported, under-reported, or did not report having any students enrolled in ALE courses in CEDARS compared to their fiscal reports. Some of the reasons for inconsistencies, as shared by staff at the ALE programs contacted, were misunderstanding the reporting requirements, believing they had received guidance that the information was no longer relevant, and using reporting systems that do not link with CEDARS or do not have a field that aligns with the ALE course type field in CEDARS.

OSPI needs accurate data to carry out its missionOSPI’s mission is “to provide funding, resources, tools, data [emphasis added], and technical assistance that enable educators to ensure students succeed in our public schools, are prepared to access post-secondary training and education, and are equipped to thrive in their careers and lives.” Measuring such outcomes requires reliable and accurate data and the ability to link it to data about students beyond high school. For OSPI to fully realize its mission, the agency must also ensure that CEDARS data is complete and accurately represented in a long-term data warehouse such as the one maintained by the Education Research and Data Center (ERDC) within the Office of Financial Management (OFM).

ALE program staff and participants report these programs provide flexibility, innovative approaches, critical services Lacking reliable data to measure ALE student outcomes, auditors gathered information directly from ALE programs on how they are meeting their intended purpose, which is to give schools flexibility to serve a diverse student population. Auditors conducted a survey of all ALE programs in Washington and visited eight programs that reported providing a unique or innovative educational experience. Auditors held focus groups with education professionals and surveyed students and parents to learn about their experiences with ALE. When students and parents/guardians were asked why they chose ALE, the most common response was that traditional schools were not working for them. Their responses are illustrated below.

Why is accurate data important?Researchers, educators, policy makers and other members of the educational system rely on good data to make informed decisions. No one outside the local program will be able to adequately monitor and evaluate ALE student outcomes until data systems are accurate and complete. Consistent and accurate data also demonstrate accountability in the use of public funds.

0%

5%

10%

15%

20%

25%

Traditionalschool not

working

Better/fasterlearning

Safe/less stress

School size

Schedule Learning style

Why did you/your student choose ALE?Responses from 246 students and parents

Source: Parent/student survey.


For students experiencing certain challenges, such as unavailability during typical school hours; needing to take breaks from their education; or overstimulation presented by the sights, sounds, and crowds of a typical school environment, traditional schools may not work. For these students, the flexibility of ALE instruction provides an alternative option they, their parents and program officials say better fits their needs. Students, families, staff, teachers, administrators and other educational professionals who are currently involved with ALE instruction across the state shared the following regarding the opportunities that ALE instruction provides:

• Some ALE programs have implemented learning experiences they describe as innovative, such as multi-disciplinary and trauma-informed approaches

• Some ALE programs collaborate with other schools, community businesses or organizations to provide additional resources and to enrich the educational experience

• A few ALE programs provide access to child care, health care or social services directly on campus

• The nature of the alternative learning style, individualized instruction and environment builds genuine relationships between students and teachers, improves their learning, and better prepares them for the future

RecommendationsIn order to be able to measure outcomes of ALE instruction, we recommend OSPI:

1. Identify and address factors that hinder school districts from accurately reporting ALE student coursework in CEDARS

2. Engage school district leadership to increase compliance with ALE reporting in CEDARS as required by RCW and WAC

In order to be able to measure long-term outcomes for all students, we recommend OSPI:

3. Collaborate with the Education Research & Data Center (ERDC) to ensure OSPI data in the ERDC data warehouse is complete and accurate

ALE: Student Characteristics and Innovative Approaches :: Introduction | 6

Introduction

All children are entitled to receive a basic education under the Washington State Constitution. While most students go to the public school that is closest to their home, Washington school districts offer students a variety of educational options outside the traditional public school setting, such as alternative high schools, charter schools and Alternative Learning Experience (ALE) programs. ALE is a program or method of delivering education that any school, including charter and alternative schools, could choose to use; it may be an option within an existing public school, or be an entire school in itself. Curriculum delivery varies widely in ALE programs. Some ALE programs may include online courses, or courses that emphasize independent student learning with limited direct instructional time. Some programs offer teacher-led courses in a traditional classroom style as well, but class sizes are typically smaller than in a traditional setting, they may meet less frequently, or involve a greater proportion of learning activities that occur away from a regular public school classroom setting.

“Alternative Learning Experience (ALE) is primarily distinguished by off-campus instruction. The intent of this type of program is to give schools flexibility to serve a diverse student population. The specific requirements and expectations of these away-from-school learning activities are detailed in a written student learning plan (WSLP) developed and supervised by a public school teacher.”

– Alternative Learning Department (ALD)Office of the Superintendent of Public Instruction (OSPI)

What distinguishes ALE from other public school options is its distinctive requirements for claiming state basic education funding. ALE programs allow school districts to claim funding for students who are not physically present in a classroom for a full day or class period.ALE programs must meet the same state standards as traditional schools; such as participating in state assessments and learning that is supervised, monitored, assessed, evaluated and documented by a certificated teacher. A student may enroll in ALE full-time or part-time. Ensuring that ALE programs deliver comparable education to traditional programs depends on the state’s ability to manage and maintain reliable data showing which child participated in what type of schooling. In 2013, the Legislature approved significant changes to the rules governing ALE programs. Engrossed Substitute Senate Bill (ESSB) 5946 was intended to encourage greater accountability and transparency in ALE programs statewide. It also required the Office of the Washington State Auditor to conduct a biennial measure of student outcomes and financial audits of ALE courses, beginning with the 2013-14 school year and continuing through 2016-17. Individual financial audits were conducted at school districts; performance audit reports addressed the measurement of student outcomes.

View all ALE audit reports published in April 2018, plus a video and a Tableau data presentation related to ALE performance audits, at www.sao.wa.gov/local/pages/ALEstudy_FinalResults

Earlier financial audits of ALE programs can be found at: www.sao.wa.gov/local/pages/SchoolsProgram

http://www.sao.wa.gov/local/Pages/SchoolsProgram.aspx

http://www.sao.wa.gov/local/Pages/ALEstudy_FinalResults.aspx

ALE: Student Characteristics and Innovative Approaches :: Background | 7

Background

Audit work to date The Office of the Washington State Auditor has been auditing ALE programs for the last 10 years. The timeline on page 10 shows a history of ALE audit results, legislative action regarding ALE, and actions taken by the Office of Superintendent of Public Instruction (OSPI) in regard to legislation and auditors’ recommendations. Work began on this four-year longitudinal performance audit in 2013, with a preliminary audit report published in February 2015. This preliminary report was followed by the first of three planned reports in February 2016, which described ALE programs and students. Auditors compared academic measures for a limited number of students identified as having received ALE instruction during the 2013-14 school year, as a group, to students not identified as having received ALE instruction. Auditors also visited 10 ALE programs that had a higher performing ALE student body. This work helped to establish a basic understanding of ALE programs in the state, but not enough time had passed to evaluate the long-term academic outcomes for these students. The preliminary audit report and first full report demonstrated that the ALE population differs from the traditional student population in many ways and that the two groups are not directly comparable. The second phase of this performance audit built on previous work, including a follow-up of the recommendations made in our first report, and an exploration of the role ALE plays for certain student populations. Auditors visited eight programs that reported they provide innovative approaches to delivering educational experiences to ALE students. Even if OSPI were to enact changes today to improve data quality, those changes would have no effect on this long-term audit. The last year of the audit, according to the legislative mandate, is the 2016-17 school year, which has already passed at the time of publication (April 2018). For these reasons, the State Auditor’s Office considers it has substantially fulfilled the mandate and will not pursue a third phase in this audit. Efforts to improve the data should nonetheless continue so that researchers, educators, policy makers and other members of the educational system will be able to adequately monitor and evaluate ALE student outcomes in the future.

OSPI is responsible for the accuracy of ALE program dataState law assigns OSPI the power and duty to supervise all matters pertaining to public schools in the state. This includes the accurate reporting of ALE student course information as required by law (see below). Although it has supervisory authority, OSPI must engage district leadership if it wishes to enforce compliance with ALE reporting requirements.

RCW 28A.300.040 Powers and duties. […] the powers and duties of the superintendent of public instruction shall be:(1) To have supervision over all matters pertaining to the public schools of the state;(2) To report to the governor and the legislature such information and data as may be required for the management and improvement of the schools...

Audit Number: 1013676

Performance AuditAlternative Learning Experience (ALE)in Washington A longitudinal study of ALE programs and student outcomes

Audit update issued: February 23, 2015

Th e State Auditor’s Offi ce is conducting a long-term study of educational outcomes for students enrolled in Alternative Learning Experience (ALE) programs. To prepare for this study, we performed some limited preliminary reviews of student data. We found that ALE students diff er from traditional students in a number of characteristics including race and ethnic makeup, disability status, participation in specifi c programs and services, instances of homelessness, and home schooling with part-time attendance. Any comparison of ALE and traditional student academic outcomes will need to control for these diff erences. Our future analyses will match ALE students with comparable students receiving traditional education.

Audit schedule• We will publish our fi rst full report in late 2015. It will evaluate ALE students statewide

and compare their outcomes from the 2013-14 school year to those of a matched set of students receiving only traditional instruction.

• Th e second report, planned for late 2016, will follow these matched student cohorts through the 2014-15 school year.

• Th e third and fi nal report, planned for late 2018, will follow these students through the 2015-16 and 2016-17 school years.

I ndependence • Respec t • I ntegrit y

AU

DITOR OF STATE

W

A S H I N G T O NNOV 11, 1889

Washington State Auditor’s OfficeTroy Kelley

Audit Number: 1016139

Th e State Auditor’s Offi ce is conducting a long-term study of educational outcomes for students enrolled in Alternative Learning Experience (ALE) programs. For this, the fi rst full-length report in the series, we visited 10 ALE programs associated with higher student outcomes and reviewed student data. Program staff and participants told us about the role their program fi lls within their respective districts, current challenges, and the reasons students enroll in ALE programs. Our analysis of student data has been aff ected by data quality problems. We were able to identify only about three-fourths of all ALE participants in the 2013-14 school year, which was meant to form our study’s baseline cohort (the group of ALE students whose progress we will follow for the next three years). Data quality on course types for the 2014-15 school year may also be unreliable. Some ALE program staff were uncertain how to interpret defi nitions of the new ALE course types set out in law. We recommend that the Offi ce of Superintendent of Public Instruction (OSPI) provide more clarifi cation to districts, as needed, so they can report ALE course types in the same way. We also recommend that OSPI match ALE student counts reported by districts for funding with ALE per-student data reported by districts, and follow up on substantial discrepancies.

Performance AuditAlternative Learning Experience (ALE) in WashingtonFebruary 1, 2016

Audit scheduleAs mentioned above, this is our fi rst formal report in a four-year study of ALE programs. Th e schedule of future reports is as follows:

• Audit Report #2 (anticipated publication date: December 2016) – Summary of ALE students during the 2014-15 school year, and the standing of 2013-14 ALE students one year later.

• Status Update #2 (anticipated publication date: December 2017) – Summary of ALE students during the 2015-16 school year, and the progress of 2013-14 ALE students two years later, compared to their non-ALE peers.

• Audit Report #3 (anticipated publication date: December 2018) – Summary of ALE students during the 2016-17 school year, and the progress of 2013-14 ALE students three years later, compared to their non-ALE peers. We also plan to visit ALE programs that show strong individual student growth.

Washington State Auditor’s Offi ce

I ndependence • Respec t • I ntegrit y

AU

DITOR OF STATE

W

A S H I N G T O NNOV 11, 1889

Earlier performance audit reports are available online at: www.sao.wa.gov/state/Pages/PA_ALEstudy

http://www.sao.wa.gov/state/Pages/PA_ALEstudy.aspx


School districts are required by law to designate ALE courses as such when reporting student coursework in CEDARS.

WAC 392-121-182 (9) Reporting requirements:(d) … School districts and charter schools must designate alternative learning experience courses as such when reporting course information to the Comprehensive Education Data and Research System.RCW 28A.232.010(7) Beginning with the 2013-14 school year, school districts must designate alternative learning experience courses as such when reporting course information to the office of the superintendent of public instruction under RCW 28A.300.500.

Presently, ALE students are reported to OSPI two ways:1. Student data is supplied by individual school districts to the

Comprehensive Education Data and Research System (CEDARS) managed by OSPI “to collect, store and report data related to students, courses, and teachers in order to meet state and federal reporting requirements, and to help educators and policy makers to make data driven decisions.”

2. ALE programs must also report accurate monthly headcount and monthly full-time equivalent (FTE) enrollments to OSPI (see sidebar for definitions used in this report). OSPI uses these figures to determine basic education funding for the programs.

OSPI and individual school districts collect a great deal of data about Washington’s schools and operations, including its ALE programs. For ALE student outcomes to be measured and reported to the Legislature, ALE students must be reliably identified in a centralized data system.

OSPI needs accurate data to carry out its missionOSPI’s mission is:

“… to provide funding, resources, tools, data [emphasis added], and technical assistance that enable educators to ensure students succeed in our public schools, are prepared to access post-secondary training and education, and are equipped to thrive in their careers and lives.”

Measuring such outcomes requires reliable and accurate data, and the ability to link it to data about students beyond high school. For OSPI to fully realize its mission, the agency must also ensure that CEDARS data is complete and accurately represented in a long-term data warehouse such as the one maintained by the Education Research and Data Center (ERDC) within the Office of Financial Management (OFM).Data issues discovered during the first phase of this long-term audit raised concerns about whether ALE student outcomes could be measured The original plan for our performance audit to measure ALE student outcomes called for auditors to use student data reported to OSPI in CEDARS and post-high school outcomes gathered through ERDC’s data warehouse. However, when auditors compared CEDARS data to headcount reports districts submitted separately for funding purposes, auditors discovered many CEDARS records with missing data in the ALE-funded field.

This report uses a number of terms to refer to students:Headcount measures each unique student served. A full-time equivalent (FTE) is a measurement of student enrollment for funding purposes. It provides an accurate estimate of the portion of time a student is served by a given program, with 1.0 referring to a full-time student.A course enrollment refers to a single student enrolled in a single course for a single term. For example, a single high school student taking a full load of courses would have 10 (if the district offers five periods a day) or 12 course enrollments (if six periods are offered) for the school year.Source: 2017 Online Learning Report to the Legislature (OSPI)

Why is accurate data important?Researchers, educators, policy makers and other members of the educational system rely on good data to make informed decisions. No one outside the local program will be able to adequately monitor and evaluate ALE student outcomes until data systems are accurate and complete. Consistent and accurate data also demonstrate accountability in the use of public funds.


Although OSPI had created a new data field in CEDARS for districts to identify ALE-funded coursework, the agency did not require school districts to enter this information until after the 2013-14 school year, yielding incomplete data. Inconsistent reporting of course types also hinders analysis of outcomes from different kinds of ALE courses. The same bill that mandated this audit also changed the way ALE courses are named and defined, eliminating “program” types and replacing them with three “course” types. OSPI directed districts to use these new definitions when reporting ALE coursework beginning in the 2014-15 school year. However, during the first phase of the audit, some ALE program staff said that they were uncertain how to interpret the legislative definitions. Without consistent application of the new definitions, programs may misclassify courses, which will compromise the reliability of any analyses based on ALE course type.The audit has identified several other data issues, including:

• ALE program names are not consistent between CEDARS and financial data systems.

• OSPI only collects grades for high-school-level courses in CEDARS, but ALE is a K-12 program.

• Changes to statewide testing during the timeframe of the audit make it difficult to track student academic growth over time using a standardized metric.

• Records of courses taken by students through homeschooling or private school are not part of OSPI reporting requirements, yet students who are enrolled part-time in public school and also homeschooled account for a sizeable portion of ALE program participants. Additionally, students who are part-time homeschooled often opt out of state testing.

• Only two years of discipline and attendance records are available through CEDARS. Additionally, the CEDARS database does not require entry of attendance records for all ALE students because “absence” is not defined for non-seat-time courses. Therefore, limited data are available to document any impact ALE may have on these two performance measures.

There are other methodology issues that complicate our measurement of student outcomes from ALE instruction:

• Many variables influence student academic outcomes and data is not available on key influences such as parent involvement, student motivation, and the proportion of a student’s overall educational experience that is from ALE.

• ALE is a diverse category of instruction, both within and across school districts. Student outcomes may vary widely across individual ALE programs.

In response to the data quality issues, the Office of the Washington State Auditor issued the following recommendations to OSPI in its report following the first phase of this performance audit:

1. Periodically evaluate whether the number of students reported by ALE programs for funding purposes is consistent with the number of students reported as ALE-funded in CEDARS, and follow up with programs where there are large discrepancies.

2. Evaluate whether ALE programs report course types in a way that is consistent and comparable, and provide further clarification as needed.


2008 | Issued accountability audit report • Three districts unable to provide documentation for

students reported in ALE internet-based programs

2011–2017 | Issued multiple accountability audits• Lack of documentation to show compliance with

state ALE laws and regulations • Inappropriate use of state funds

2013 | Began performance audit planning• Requested rosters of 2012-13 ALE students

2016 | Performance audit placed on hold to allow OSPI time to improve data quality. Auditors surveyed ALE programs to prepare for audit resumption in 2017.

2016 | Issued accountability audit report• Multiple issues resulted in questioned funding

2016 | Issued first performance audit full report • Data issues limited evaluation of outcomes from

ALE programs • Made recommendations to OSPI on data quality

2014 | Issued accountability audit report on ALE programs for 2011-2013 school fiscal years

• Recommended districts “ensure ALE program directors/staff are aware of the most current regulations”

2017 | Resumed performance audit (January) and identified continuing issues:

• Inadequate student outcomes data in CEDARS• ALE program staff continued to have challenges in

correctly identifying ALE course types

2015 | Issued preliminary performance audit report • Preliminary results of analysis for 2012-2013 rosters • Reported on data issues and limitations

2018 | Issued accountability audit report • Recommended that all ALE programs be approved

or reapproved by OSPI 2018 | Issued second performance audit full report

• Included recommendations to allow measurement of student outcomes of ALE instruction

Prior to 2013-14 school year • ALE defined by program type (online/digital,

contract-based, parent partnership)• ALE students not tracked in a centralized data

system

2013-14 • ALE student course enrollment flagged as “Yes/No”

in CEDARS• ALE course type not tracked in a centralized data

system

2014-15 • CEDARS system expanded to track ALE course type• ALE Program Manager position created (November

‘14)

Planned for 2017-18 • Ask the Legislature to update course type definitions

during the 2018 legislative session

2016-17 • White paper addressed possible solutions to

reporting issues (February ‘17)• Developed an ALE Reporting Discrepancies

Communication Plan (September ‘17)

Timeline of State Auditor’s Office audit activities and actions taken by the Office of Superintendent of Public InstructionAudit activities by calendar year, OSPI actions recorded by school fiscal years

2013 | ESSB 5946 signed into law• Mandated performance audit of ALE

student outcomes by State Auditor• Changed definitions of ALE course types

(Online, Site-based, Remote)• Provided rule changes to ALE reporting

requirements

2015-16 • OSPI begins research on data quality (summer ‘16)

ALE: Student Characteristics and Innovative Approaches :: Methodology | 11

Scope & Methodology

The audit was designed to evaluate ALE instruction by answering the following questions over a four-year period:

• What are the characteristics of students who enroll in ALE programs?• How do academic outcomes for ALE students compare to outcomes for

similar students receiving traditional instruction?• Are there types of students that respond better to ALE instruction

than others?• Does the type of ALE course taken affect academic outcomes?• What are ALE student outcomes after high school?• What are the characteristics of high-performing ALE programs?

Data reliability testingTo determine whether or not individual ALE students could be identified, auditors compared student headcounts as reported in CEDARS to the monthly report of school district enrollment eligible for basic education funding (P-223 report) through ALE. The annual unduplicated count of students who have taken one or more courses through ALE should be at least as high as, or higher than, the maximum number of ALE students the school district reported for funding during any particular month during the school year. However, in both the first phase and the current phase of this audit, auditors discovered that ALE student counts from CEDARS were substantially lower than they should be. Because CEDARS data is unreliable for identifying individual ALE students, auditors were not able to address all of these objectives.

Revised data collection strategiesFor the second phase of the performance audit, auditors gathered information directly from ALE programs to explore outcomes as reported by students, parents, teachers, counselors, administrators and other staff. In doing so, auditors also describe unique or innovative approaches intended to address the diverse needs of Washington’s student population. Auditors used several qualitative data collection strategies to evaluate ALE instruction, including electronic surveys, interviews and focus groups. The variety of approaches helped answer the following questions:

• What are the characteristics of, and challenges for, specific populations of students who enroll in ALE programs?

• What innovative approaches are ALE programs using to educate the state’s diverse student population?

• Are Washington’s ALE programs meeting their intended purpose: “... to give schools flexibility to serve a diverse student population”?

ALE program survey — We surveyed all 264 ALE programs and received responses from every program. Information gathered in the survey guided our selections for the site visits, telephone focus groups and additional data collection activities, which are described below.

ALE: Student Characteristics and Innovative Approaches :: Methodology | 12

Site visits — Auditors visited eight ALE programs in person that reported providing a unique or innovative educational experience. Visits included focus groups with district and program administrators, instructional staff, support staff, counselors, students and parents. Telephone focus groups — Auditors conducted telephone focus groups to ask education professionals to assess the benefits and challenges of ALE when working with a particular type of student, and the reasons why these students seek ALE instruction.Parent/student survey — Due to the risk of low participation from parents and students with certain characteristics during our site visits and concerns about confidentiality laws regarding educational information, we developed an anonymous survey to learn about the ALE experience for students with particular characteristics that present educational challenges.

Audit performed to standardsWe conducted this performance audit under the authority of state law (RCW 43.09.470), approved as Initiative 900 by Washington voters in 2005, and in accordance with generally accepted government auditing standards as published in Government Auditing Standards (December 2011 revision) issued by the U.S. Government Accountability Office. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. See Appendix A, which addresses the I-900 areas covered in the audit. Appendix B contains more information about our methodology.

Next stepsOur performance audits of state programs and services are reviewed by the Joint Legislative Audit and Review Committee (JLARC) and/or by other legislative committees whose members wish to consider findings and recommendations on specific topics. Representatives of the Office of the Washington State Auditor will review this audit with JLARC’s Initiative 900 Subcommittee in Olympia. The public will have the opportunity to comment at this hearing. Please check the JLARC website for the exact date, time, and location (www.leg.wa.gov/JLARC). The Office of the Washington State Auditor conducts periodic follow-up evaluations to assess the status of recommendations and may conduct follow-up audits at its discretion.

http://leg.wa.gov/JLARC/Pages/default.aspx

ALE: Student Characteristics and Innovative Approaches :: Audit Results | 13

Audit Results

To evaluate student outcomes, individual ALE students must be identifiable and distinguishable in the data from students who did not receive ALE instruction. Also, a student’s records must be matched to outcome data for the same student over time. Significant and ongoing data quality issues, first identified during the initial phase of this long-term performance audit in February 2016, continue to prevent the measurement of ALE student outcomes. The Office of Superintendent of Public Instruction (OSPI) first took action in response to recommendations from the first phase of this performance audit in the summer of 2016 by conducting analyses and research on data quality. This work led to a white paper in February 2017 that proposed potential activities to improve data quality. However, by this timeframe the 2015-16 school year had already ended and only a few months remained in the 2016-17 school year. Because of ongoing issues with data quality, the Office of the Washington State Auditor was unable to fully explore or verify outcomes down to the individual student level within the time frame of the mandate. Lacking reliable data to measure student performance quantitatively, auditors gathered information directly from ALE programs on how they are meeting their intended purpose: “… to give schools flexibility to serve a diverse student population.” The information gathered is of value to gain a better understanding of the role of ALE instruction in Washington and to share approaches that educators have found to be effective with certain student populations. However, we were unable to measure and corroborate these approaches as being effective and cannot verify that these represent “best practices.” Audit results are presented in three sections:

• Section One evaluates the current status of recommendations related to improving data quality that were made during the first phase of the performance audit, and explains how data quality issues affected the State Auditor’s decision to conclude the audit.

• Section Two discusses students with characteristics that present educational challenges in traditional settings and illustrates how ALE helps alleviate some of those challenges.

• Section Three presents approaches to delivering educational experiences that were described by programs and participants as “unique or innovative.”

Data needed to measure student outcomes:1. Individual ALE

students must be identifiable

2. ALE students must be distinguishable from other students who did not receive ALE instruction

3. A student’s record must be matched to outcome data for the same student over time

What is a “measure of student outcomes”?Examples include:

• Months of continued enrollment in school

• Graduation • College enrollment• Stable employment


Section One: Follow-up on earlier recommendations, continuing concerns, and implications for audit work

Results from the first phase of our audit found that reliably identifying ALE students using CEDARS data was not possible. The first full report made the following recommendation to OSPI:

1. Periodically evaluate whether the number of students reported by ALE programs for funding purposes is consistent with the number of students reported as ALE-funded in CEDARS, and follow up with programs where there are large discrepancies

ALE student data cannot be relied upon to accurately measure outcomes To see if OSPI had resolved this problem and if data were now usable for measuring individual student outcomes, the second (current) phase of work compared CEDARS data (29,390 students) to the monthly fiscal report of school district enrollment eligible for basic education funding through ALE, as provided by OSPI for the 2015-16 school year (35,532 students). In total, 161 school districts reported 39,819 ALE students through either reporting system.The “ALE Course Type” field in CEDARS no longer contains missing values. However, when examined on a per-district level rather than statewide level, CEDARS is far less accurate than it first appears.

• Possible over-reporting: 53 districts reported 4,287 more students taking an ALE course in CEDARS than they claimed in their fiscal reports. This accounts for 33 percent of all districts reporting to have ALE students.

• Under-reporting: 104 districts reported 10,429 more students in their fiscal reports than were reported in CEDARS. This accounts for 65 percent of districts reporting to have ALE students.

ӽ No ALE students reported in CEDARS: 43 districts reported 3,200 students in their fiscal reports but did not report any students in their districts as taking an ALE course in CEDARS. This accounts for 27 percent of all districts reporting to have ALE students.

These inconsistencies are illustrated in Exhibit 1. CEDARS data cannot be relied upon to accurately identify individual ALE students. If individual ALE students cannot be identified in the data, it is impossible to measure individual student outcomes.

Inconsistencies between �scal and CEDARS data for ALE student enrollment2015-2016 school year

4% reported no ALE students in �scal data

27% reported ALE students in �scal data but not in CEDARS

65% Fiscal reporting exceeds CEDARS

(under-reporting)

2% matched

33% CEDARS exceeds �scal

(possible over-reporting)

Source: CEDARS database and P-223 enrollment report.

Exhibit 1 – Inconsistencies between fiscal and CEDARS data for ALE student enrollmentDistricts reporting ALE students for the 2015-16 school year

Source: CEDARS database and P-223 enrollment report.

See the Background section of this report for information on the two systems for reporting ALE students.


School district reporting of ALE enrollment is inconsistentTo explore some potential causes of reporting inconsistencies between the fiscal data and the data in CEDARS, auditors contacted eight school districts of various sizes and locations that either over-reported, under-reported, or did not report having any students enrolled in ALE courses in CEDARS compared to their fiscal reports.Districts that reported more students in CEDARS than they claimed for funding (possible over-reporting)Districts can claim a student for basic education funding for no more than 1.0 student full-time equivalent (FTE). However, ALE students may take more than a full load of courses in order to retrieve credits or to make room in their schedules for elective courses. When a student’s FTE exceeds 1.0, districts must find another source of funding to cover the costs, such as grants or levies. In this situation, more students receive ALE instruction than are reported for funding. Districts also cannot claim ALE funding for a student if the program fails to meet all of the program requirements before the end of the month. One school district said it reports all students participating in its ALE program in CEDARS because they receive this form of instruction, even if the district does not plan to claim ALE funding for some of those students. So, in some cases, districts will report more ALE students in CEDARS than in their fiscal reports, and this reporting is not necessarily inaccurate. Districts that are under-reporting or not reporting ALE coursework in CEDARSThe number of students reported in CEDARS can be more, but should never be less than what was reported for funding in fiscal reports. The reasons why districts under-report ALE enrollment are complex and varied. Some misunderstood the reporting requirements:

• One district believed the new reporting requirements for the 2013-14 school year were for that year only. The person in charge of data entry noted that the district’s data system displays a message saying “this information is no longer relevant.” They could not recall which authority told them the information was no longer relevant.

Other programs use their own reporting systems, with mixed results:• Some are not linked to CEDARS at all. One district said it uses a reporting

system called Wings Northwest for its ALE program, as do 36 other ALE programs in the state. Wings Northwest is not linked to CEDARS, which causes CEDARS to show that ALE students are not enrolled in any courses.

• Some are linked to CEDARS but incompletely. Other ALE programs said they use a reporting system called Skyward that does link with CEDARS. However, courses for ALE students entered in Skyward do not show up in CEDARS until the course is finished, when it appears in their transcript (not the Student Schedule file in CEDARS that performance auditors used for comparison). The person in charge of data entry at one of the districts said that Skyward does not have a field that aligns with the “ALE Course Type” field in CEDARS, and they never identify any course as “alternative.” When asked why they did not report ALE student course enrollments in CEDARS, they replied, “We never had anybody tell us we had to do it.”


ALE program staff continue to struggle when classifying ALE instruction by course type The first report in the audit series concluded that some program staff were uncertain how to interpret the ALE course types as defined in legislation (briefly summarized in Exhibit 2). CEDARS contains a record for every course that a student enrolls in. The school district must indicate whether or not the course was an ALE course, and if so, its type. The course type selected should match how the individual student received instruction, which might differ from student to student. Without clear and consistent application of the new definitions, programs may misclassify the type of instruction a student received, which will compromise the reliability of any analyses based on ALE course type. The report made a second recommendation to OSPI:

2. Evaluate whether ALE programs report course types in a way that is consistent and comparable, and provide further clarification as needed.

Reporting guidance for ALE programs is not up to dateAudit follow-up on this recommendation found that, as of October 2017, reporting guidance for school districts on OSPI’s Alternative Learning Department’s (ALD) webpage is not up to date. The main page lists “Guidance and documentation for 2015-16,” and the last informational webinar is dated August 2014. The 2017-18 CEDARS data manual says the ALE course type field definition was last updated in January 2014. ALD representatives said they had not yet updated the guidance because OSPI was in the process of revamping its website. Therefore, the website continues to lack information to help districts and programs interpret ALE course type beyond the definitions given in statute. Questions added to our 2016 ALE Program Survey were specifically designed to determine whether program staff were interpreting ALE course types consistently. In analyzing responses from the survey, we discovered that when a student differs from fellow students taking the same course in site-time or teacher-time, course type reporting is very inconsistent across programs. Exhibit 3 on the following page sets out three hypothetical situations and how ALE programs said they would report this student’s course type.

Exhibit 2 – ALE Course-type definitions in WAC 392-121-182Site-based: Course work that is not an online course where the student has in-person instructional contact time for at least 20 percent of the total weekly time for the course.Remote: Course work that is not an online course where the student has in-person instructional contact time for less than 20 percent of the total weekly time for the course.Online: Coursework where more than half of the course content is delivered electronically and more than half the teaching is conducted from a remote location through an online course learning management system or other online or electronic tools.Note: Auditor emphasis added.Source: Washington Administrative Code.


In every instance, 45 percent or more of the respondents gave an incorrect answer. If these answers reflect the actual selections ALE program staff make in CEDARS when confronted with similar uncharacteristic student situations, there is a strong probability that the ALE course type reported for these students will be incorrect.

OSPI did not maintain a contact list of ALE programsOSPI has a list of contacts who report student enrollment counts at each school district, but does not maintain a list of contacts at the individual ALE program level. In order to conduct the survey, auditors generated an ALE program level contact list and revised it as they learned of new ALE programs or those that were no longer in operation. Auditors gave OSPI the contact list used for the survey; the agency said the information was very valuable and that staff would maintain and regularly update the list.

Exhibit 3 – ALE programs interpret course type definitions differentlyCorrect answers supplied by Director of the Alternative Learning Department, OSPI, November 2017

Jo is enrolled in an ALE course typically defined as “site-based” for his fellow students, but Jo receives in-person instruction for less than 20 percent of the total weekly time for the course. How would you report Jo’s ALE course type in CEDARS? Online Remote Site-based

47%

16%

38%

Survey responses indicate Jo would be reported as:

Correct answer: Remote

Sofia is enrolled in an ALE course typically defined as “remote” for her fellow students, but Sofia receives in-person instruction for more than 20 percent of the total weekly time for the course. How would you report Sofia’s ALE course type in CEDARS?

Online Remote Site-based

36%

14%

50%

Survey responses indicate Sofia would be reported as:

Correct answer: Site-based

Kyle is enrolled in an ALE course typically defined as “online” for his fellow students, but Kyle receives more than 50 percent of his course content on-site (for example, in a computer lab). How would you report Kyle’s ALE course type in CEDARS? Online Remote Site-based

7%

55%

38%

Survey responses indicate Kyle would be reported as:

Correct answer: Online


Auditors asked ALD management how they know when a new ALE program is forming. They stated that they do not know about new ALE programs until programs claim their first month of ALE funding, at which time the program name appears on fiscal reports. If the ALD does not know about a program while it is in development, it may not be currently playing a role in helping new ALE programs understand and apply all the applicable rules and regulations as they are forming.

OSPI has taken steps to address data issuesWhen asked what actions OSPI, and specifically the ALD, has taken in response to our performance audit recommendations, officials acknowledged continued inconsistencies in the data and stated they have developed plans to address them. Two ALD documents – a white paper developed in February 2017 and a communications plan developed in September 2017 – describe their possible solutions to reporting issues. The actions OSPI has taken or plans to take to improve ALE data quality include the following.

OSPI Action 1: Ask districts to respond to discrepancies between their CEDARS ALE data and enrollment submitted for fundingThe ALD has begun comparing ALE student headcounts in CEDARS to enrollment reports of ALE student headcounts submitted by school districts for funding, but is waiting for a report of 2016-17 school year student counts from CEDARS before continuing. The ALD says it is investigating the capabilities of its data system and exploring options to develop a process that identifies discrepancies between CEDARS and fiscal reporting at the district level.

OSPI Action 2: Ask the Legislature to consider amending ALE course type definitions OSPI believes that the existing definitions of “site-based” and “remote” ALE course types are too narrow, creating confusion that contributes to poor data reporting. OSPI is asking the Legislature to update the existing definitions during the 2018 legislative session.

OSPI Action 3: Improve communication on reporting requirementsThe ALD developed an ALE Reporting Discrepancies Communication Plan in September 2017. Proposed improvements to communication include adding more guidance to the ALD website; regular communications about reporting requirements through newsletters, webinars and other channels; and targeted communications with districts that have reporting discrepancies. The ALD also included a brief reminder about ALE reporting through CEDARS in the September 2017 Alternative Learning newsletter. The ALD is using the contact list the Auditor’s Office provided to disseminate this information. ALD management stated that it is also conducting presentations on reporting requirements at Washington Association for Learning Alternatives workshops across the state during the months of October and November 2017. Other communication efforts are still in the planning stages or have no specific implementation date.


Post-secondary outcomes tracked by ERDC in its data warehouse include:

• Two- and four-year college enrollment

• Apprenticeship enrollment

• Employment/wage data

Continued data quality issues will affect remaining work for this auditThe original plan for the third phase of the performance audit, in line with the legislative mandate, was to use data for a cohort of students from the 2013-14 school year and compare its progression over the course of four years, ending with the 2016-17 school year. However, the lack of reliable and centralized data to identify ALE students means the planned comparative, quantitative data analyses cannot be achieved. Nonetheless, the performance audit has produced rigorous qualitative measures of student outcomes that offer policy makers and stakeholders greater insight into ALE programs and the student populations they serve. The State Auditor may decide to revisit some of the remaining objectives in the future if and when student data becomes more reliable:Additional data quality issues may impact other studiesOSPI regularly sends CEDARS data to the Education Research and Data Center (ERDC). The ERDC matches student data with data on post-secondary outcomes in its data warehouse. At the present time, however, there appears to be insufficient verification of data as stored in CEDARS and how it appears once it is manipulated and loaded into ERDC’s data warehouse. Issues with data received from ERDC during this performance audit include:

• It takes several months to receive requested data• Updates to ERDC’s data warehouse or revisions to its data extraction

process requires an entirely new data set to be provided to auditors• At one point, data appeared to show there were no longer any gifted

students in the public school system. Apparently, the way this information is tracked was changed in CEDARS, but the change was not communicated to the person who regularly pulls data extracts for ERDC.

• Multiple problems with data quality, including: ӽ Students who graduated but had no courses recorded ӽ Records with missing values on required fields such as State

Course Code ӽ Elementary grade students with records of high school credit earned

It is unclear at this point whether these issues are a result of data entry errors in CEDARS or were generated during the many processes that occur before data is provided to auditors. When data quality issues are identified, it often takes many months to determine the cause since there are multiple points at which errors might be introduced into the complex process of data assembly.Audit staff have given both ERDC and OSPI feedback about various issues as they were discovered. However, auditors have not yet attempted to track a single student over time, and this process may identify additional data quality issues. Although auditors have conducted analyses on post-secondary outcomes using ERDC summary data (as published in December 2016 in Workforce Development: Identifying CTE Student Outcomes), we have not yet explored the quality of data on post-secondary outcomes from the ERDC data warehouse on an individual student level.

http://www.sao.wa.gov/state/Documents/PA_Workforce_Development_CTE_Outcomes_ar1018364.pdf


Data quality improvement is necessary for adequate monitoring and supervision of ALESufficient supervision and oversight of ALE programs cannot occur if OSPI does not know how to contact the ALE programs that are currently in operation, nor will they be able to provide guidance to new ALE programs as they are forming. Consistent and accurate data also demonstrate accountability in the use of public funds.The inability to accurately identify ALE students in the Comprehensive Education Data and Research System (CEDARS) is an issue that must be addressed for purposes beyond this legislatively mandated audit. Researchers, educators, policy makers and other members of the educational system need accurate and complete data on a per-student level to adequately monitor and evaluate student outcomes. For these reasons, this report contains recommendations to OSPI that, if implemented, will allow for future measurement of student outcomes from ALE instruction. These recommendations include: identifying and addressing factors that hinder school districts from accurate reporting of ALE students in CEDARS; engaging district leadership to increase compliance with ALE reporting; and collaborating with ERDC to ensure data in ERDC’s data warehouse is complete and accurate.


Section Two: ALE program professionals and participants report that these programs meet their intended purpose

Lacking reliable data to measure student performance, auditors looked for other ways to answer the legislative mandate. Building on earlier work, the performance audit focused on certain student populations that are challenging to serve in a traditional school setting, and auditors visited eight ALE programs in person that reported providing a unique or innovative educational experience. However, because student outcomes cannot be measured with the available data, the Offi ce cannot verify how well these ALE programs are performing or compare outcomes to students who did not receive ALE instruction.

Washington’s ALE student population is diverseTh roughout the audit, people involved with ALE programs said they want to dispel negative impressions of ALE. Nonetheless, program staff also indicated that they do see students with characteristics that present educational challenges. Th erefore, the main focus for the second phase of the long-term audit was to explore how ALE functions for students with educational challenges.As the fi rst report in this audit series described,

”Many ALE participants and staff mentioned that there’s a public perception that ALE programs are easier than traditional programs and house the students who are seen as ‘bad kids’ or ‘kids who can’t hack it’ in traditional school. Yet we met with many ALE students who are advanced in their studies, and we heard from many sources that behavioral issues are rare in ALE programs.”

In November 2016, we sent a survey to all ALE program administrators in Washington, which obtained a 100 percent response rate. Th e survey responses allowed us to describe the overall proportion of ALE students with certain characteristics, many of which are not currently tracked in any centralized data system. Exhibit 4 summarizes the characteristics auditors considered, ordered from most frequent to least frequent. Characteristics were chosen in consultation with OSPI and the Washington Association for Learning Alternatives (WALA). A detailed description of survey results in Tableau, based on each program’s self-reported data, is available on the State Auditor’s Offi ce’s website at www.sao. wa.gov/local/pages/ALEstudy_FinalResults. Appendix C provides illustrations from the Tableau presentation.

Exhibit 4 – Prevalence of selected characteristics in current ALE students, most to least commonResponses from 264 ALE program administrators, concerning students enrolled as of November 1, 2016. CharacteristicBelow grade level

Credit retrieval

Has Individualized Education Program for special education

Drug/alcohol issues

Above grade level

Has disability identifi ed under law that requires accommodation and 504 Plan

Homeless

Attended high school for more than 5 years

Also enrolled in Running Start

Also enrolled in Skills Center

Juvenile/Criminal record

Monitored for truancy issues under state’s Becca Bill

Pregnant/parenting

English Language Learner

Head of household

Diagnosed with autism

Note: Some students may have more than one of these characteristics. Source: ALE program survey.



According to the program staff auditors spoke with, recently enrolled students are a different population than those who have attended these programs historically. For example, more students are coming in with anxiety, depression, mental illness, or having experienced adverse circumstances. Conversely, they are also seeing more students with none of these issues seeking out their program as an educational choice. The survey also asked programs to report the characteristics of students who had newly enrolled in their program during the previous two school years. Exhibit 5 lists some of the many reasons why recently enrolled students chose ALE, as reported by their ALE program, ordered from most frequent to least frequent.

Traditional education does not work for every studentTraditional education can present barriers for some students. To better understand how ALE addresses the needs of students with particular characteristics, auditors conducted a literature review and reached out to parents and students, ALE program administrators, teachers and counselors, many with a background in traditional education. The comments in this section and the next resulted from online surveys, telephone focus groups and site visits to ALE programs that reported serving students with characteristics such as those listed in exhibits 4 and 5. (Appendix D has more information from the telephone focus groups regarding the student characteristics we explored.) ALE program participants described challenges with traditional education Audit outreach showed that many ALE students have experienced challenges with traditional education. These challenges include unavailability during typical school hours; needing to take breaks from their education; overstimulation presented by the sights, sounds and crowds of a typical school environment; and feeling disengaged from their education. For example, the contact time required in a traditional school becomes a barrier for students who need room in their schedule due to physical or mental health issues, those who are pregnant or parenting, or those who have external pursuits that conflict with a typical school schedule. These themes also arose during the first phase of the audit.Some parents reported that in a traditional education setting, their children were bored and not challenged enough in some areas, yet struggled without help in other areas. Parents felt traditional schools were not able to accommodate such specific or conflicting learning needs. The most common reason surveyed families gave for choosing ALE is that traditional education was just not working for them (see Exhibit 6).

[My son] requires extra time and attention in some areas to achieve mastery. He’s a grade level behind in spelling, but a grade level ahead in reading. It’s hard to meet those varying needs at a traditional school.

~ ALE parent

Exhibit 5 – Reasons for enrolling in ALE (recent student enrollments), most to least commonResponses from 264 ALE program administrators, concerning students newly enrolled in the 2015-16 and 2016-17 school yearsReasonPreference for ALE philosophy

Credit deficient

Unwilling to return to traditional school

Mental health issues

External commitments/pursuits

Friends also enrolled

Make room in schedule

Previously ALE

Physical health issues

Dropout reengagement

Take a specific course

Suspended/expelled

Want to graduate early

Note: Some students may have had more than one reason.Source: ALE program survey.

0%

5%

10%

15%

20%

25%

Traditionalschool not

working

Better/fasterlearning

Safe/less stress

School size

Schedule Learning style

Exhibit 6 – Reasons for choosing ALEResponses from 246 students and parents



For students who have experienced both traditional and ALE instruction, 91 percent of students and 85 percent of parents reported that it was easier to learn in an ALE program than a traditional school (see Exhibit 7).

This was not because they found the lessons or coursework to be less rigorous, but rather because the environment fostered by the ALE program was conducive to their learning style and/or needs. Respondents went on to describe a number of issues that made traditional schooling harder or, conversely, what made ALE a better choice for them. Their insights are provided on the following pages.Many students feel more comfortable in ALE programs Students reported that they found traditional schools to be overcrowded and anxiety inducing, and some had been bullied to the extent they no longer felt safe going to school. Students in general reported feeling safer in their ALE program than they did in traditional school, and that other students were more accepting. Students reported that there are “no cliques” or bullying at their ALE program, and that their peers accept and support each other.The students expressed a sense of belonging and pride in their schools. Students said smaller class sizes allowed for more individualized instruction and one-on-one time with their teachers. The ALE format also gave them the freedom to work at their own pace and according to their particular learning style, which they said makes it easier to learn and gain a better understanding of the material. For students diagnosed with autism in particular, fewer students and environmental noises, such as bells and hallway noise, generate less overstimulation. There are also fewer demands on them to conform to social norms — they can engage in self-stimulatory behaviors and rituals without fear of being bullied. Education professionals elaborated on challenges with traditional educationTeachers, counselors and administrators echoed the opinions held by parents and students. They offered professional insight into why certain students find the structure and environment of traditional education so challenging and when ALE instruction may be a good fit. For example, online courses mask the age of the student during electronic interactions with fellow students – students do not know that their classmate in Calculus is only in 7th grade or set to graduate early.

Both/other/no response

5%ALE easier

91%

Traditional easier4%

Students

Both/other/no response

11%

ALE easier85%

Parents/guardians

Traditional easier4%


Exhibit 7 – Is it easier for you (your child) to learn with ALE or at a traditional school?83 responses from students, 74 responses from parents/guardians

We pride ourselves on tolerance regardless of faith/ethnicity/sexual orientation. Classes are five kids, so if one is missing, it is a big impact. Even with so few kids in class, there is no sense of isolation. When I was in the hallways as a new student, I expected to be brushed off or get a ‘don’t want to talk’ vibe because I was new or infiltrating, but that never happened here. Almost every time the person is willing to let you join. If you are a [program name] student, you are already friends. You can be yourself and everyone will be okay.”

~ ALE student


Such anonymity also benefits students behind a grade level or still in high school after their fifth year, who might otherwise feel embarrassed or too old to be in a traditional high school. Conversations during the audit produced stories of students working to help support their families or regularly arriving late because they take younger siblings to school every morning; students who missed many days of school to care for a parent or to recover from illness themselves; and students trying to balance pregnancy or parenthood with completing their education. Other stories include students trying to balance education with competitive dance, competing on a national television show, or recovering from a traumatic brain injury; the last student would have missed three months of school but for the ALE program. Program staff said that the flexibility provided to these students through ALE might mean the difference between staying in school and dropping out.

ALE participants report ALE programs are more successful than they may appear

In the view of people interviewed for this audit, Washington’s ALE programs are meeting their intended purpose. Almost everyone was supportive of the innovative learning experiences and services available to students through ALE programs, emphasizing that ALE is an important scholastic option and is successful for many students. Program officials and teachers alike stressed the important role ALE programs play in helping districts meet their obligation to make education available and accessible to all students. When students were asked where they would be now if not for ALE, the most common answers were “dropping out,” “failing,” or “not graduating,” some said they would be “homeless;” and a few said “dead.” Instead, they are continuing their education and hoping to reach graduation.However, success for these youths can be difficult to quantify. For instance, a metric of on-time graduation rates, which is used to define success in the state’s educational system, can be misleading. Often students choose ALE programs because they are already behind in school, and by the time they begin their journey with ALE, may be on track to be a 5th or 6th year senior. By this common metric, some of the programs we visited said their program does not look “successful” on paper.

It keeps kids in school, over half if not more of the students would be dropouts if ALE was not serving them. It’s successful because it gives ALE kids a chance to hold their breath until they can figure it out, then a light goes on. Kids are in control of their education, and they work on relationships. It builds confidence through small successes.

~ ALE principal

Not everyone needs to hear the same thing at the same time. Traditional schools are assuming everyone learns the same way, so [you’ve] already lost half the group.

~ ALE teacher


Staff know their program is successful when they observe individual student growth in small, daily accomplishments, as well as increased student learning, engagement, and social and emotional growth. Because these reported successes are most evident in individual student improvement, program staff and parents said that student growth over time would be a more accurate metric of ALE student outcomes and program success. Key words and phrases to describe success for ALEStudents said these programs make them want to come to school every day, and overall, have prepared them for the future in a way that traditional school did not. This is in alignment with OSPI’s mission, to “ensure students […] are prepared to access post-secondary training and education, and are equipped to thrive in their careers and lives.”Program staff and parents offered a host of reasons why they believe some students respond well to ALE and why they consider it a successful model. It keeps students engaged by allowing them to take control of their own learning, and the assignments and curriculum are tailored to what fits the individual student’s interests. Progress to the next level is often a competency-based determination, which staff, parents and students all felt holds students more personally accountable for their learning. As such, multiple students compared their program to a college-like setting because of the way they felt responsible for their own progress and management of assignments.Students said the life-skills they have learned through internships and applied-learning make them feel more prepared for college and a career than their peers at the local traditional school. Program staff, parents, and students also attribute ALE’s “success” to the close relationships teachers build with the students, and to the mutual respect established that they feel students and teachers often do not receive in a traditional setting. Interviewees said these relationships have been crucial in helping a student feel safe physically and emotionally. During the audit, almost all the feedback about ALE was positive. However, a small number of students said they wanted more class options or structure to their instruction. A few parents expressed concerns about the ALE framework moving toward a more traditional model of education.

It teaches you how to keep yourself on track, a lot of self-discipline, that’s important when you get out there on your own.

~ ALE student

In education, there are all kinds of metrics in school. But knowing and seeing a group of kids that has a variety of barriers or trauma, but have a great sense of peace and safety—that is profound.

~ ALE administrator


Section Three: Several ALE programs reported providing innovative learning experiences and critical services to foster student success

This section of the report presents information gathered during site visits to ALE programs that reported using unique or innovative approaches to delivering educational experiences. The auditors would like to thank the district and program administrators, teachers and staff, as well as students and parents who shared their insights and experiences. The following discussion serves to illustrate the variety of approaches ALE programs are currently taking to educate the state’s diverse student population. Some of these approaches are educational alternatives, and others are possible solutions to common issues faced by ALE programs.

Auditor Note: The audit did not independently verify individual statements for accuracy, nor do we intend to represent the opinions expressed as factually representative of all ALE programs in the state. Because it is not possible to measure and corroborate the stated outcomes from these approaches, we do not assert these approaches represent “best practices.”

Programs apply practices that are grounded in evidence and experienceAccording to an article published on the IDEAs That Work website (see sidebar), published by U. S. Department of Education,

“Evidenced-based [instructional] practices are those ‘effective educational strategies supported by evidence and research’ (ESEA, 2002). When teachers use evidence-based practices with fidelity, they can be confident their teaching is likely to support student learning and achievement of college and career-ready standards.”

Some of the programs auditors visited explicitly mentioned they apply evidence-based instructional approaches, often building on models that have proven successful elsewhere.

• Avanti High School’s approach evolved from Alaska’s Chugach School District model that earned the Malcolm Baldrige National Quality Award and was honored by President Bush for being the first educational institution to win this award.

In the Chugach model, learning is non-graded and constant for each student, but the expected time to absorb learning is variable. (In traditional models, time is fixed – the number of days in a school year – but how much each student learns is variable.) Students select interest-based projects to help them master academic skills, and the teachers serve as facilitiators to help students stay engaged and make sure learning meets state standards. Engaging with the community through service and other learning experiences is also a main component of Chugach’s and Avanti’s model.

Programs visited for this auditProgram DistrictAvanti High School OlympiaDiscovery Alternative High School South KitsapIndependent Learning Center Methow ValleyKent Phoenix Academy KentNova High School SeattleOpportunities Program EastmontSky Valley Education Center MonroeThree Rivers Homelink Richland

A description of each program visited, including challenges and barriers to establishing similar programs elsewhere, can be found in Appendix E.

View the article at the U. S. Department of Education’s website: ccrs.osepideasthatwork.org/teachers-academic/evidence-based-practices-instruction

https://osepideasthatwork.org/


• Independent Learning Center is one of 52 Big Picture Learning (BPL) schools in the United States. Students are assigned to a mentor and participate in internships one or two days a week. Assessment involves public learning exhibitions where students present what they have learned.

BPL has received several grants to establish new schools from the Bill and Melinda Gates Foundation; in 2010, President Obama praised BPL for “engaging and relevant models of innovation worth replicating.”

Below, the report describes in more detail how some ALE programs in Washington have incorporated similar approaches.Alternative approaches for delivering instructionSeveral of the visited ALE programs use a multi-disciplinary or project-based curriculum, offering instruction in an integrated way that encourages students to make meaningful connections across all subject areas. Teachers from these programs explained that they regularly coordinate with each other to plan and present lessons or help students complete projects. Avanti High School’s “cave” is a section of hallway that students had turned into a cave-like setting. While constructing the cave, teachers integrated instruction on art, chemistry, history and archeology. Before it became a cave, the hallway was a brain. Students in the Three Rivers Homelink Science, Technology, Engineering, Art and Math (STEAM) program studied sound wave frequencies while learning to make and play a musical instrument called a marimba. In another project, the students designed chess sets while learning about geometric shapes and modeled their designs in clay. Then, in the engineering section, students used a computer design program to produce a prototype of their chess set using a 3-D printer. While all ALE programs provide individualized instruction through a written student learning plan, Avanti High School takes personalization further. New students complete a brain preferences inventory and Myers-Briggs assessment to help them identify and better understand their learning style. Instructors use the results of these assessments to tailor each student’s learning experience. Sky Valley Education Center takes parent participation a step further. Not only are parents encouraged to be closely involved in teaching their children, they can also attend onsite classes with their child. Sky Valley offers parents trainings and workshops on a variety of topics such as selecting curricula, providing instruction outside the classroom, improving reading and math skills, and strategies to help students with dyslexia, ADHD or executive function issues. Participants reported that having parents on campus provides role models to students and helps parents stay fully informed on how their child is doing.

Benefits of alternative approaches for delivering instructionStudents who received alternative approaches to delivering instruction say they are able to express their own interests, demonstrate their unique skills, and deeply understand academic material by applying a variety of learning styles. As a result, students said they have improved confidence and leadership skills. Educational professionals from some of these programs said that colleges and employers love their students because they are able to think on their own.


Alternative approaches to student assessmentSome ALE programs use alternative approaches to assess student growth and progress. These programs provide a non-competitive environment, where student progress is measured in terms of self-improvement rather than grades and recognition. ALE instructional staff, parents and students attribute reduced learning anxiety and promotion of individual learning success to such approaches. As one student said, “The emphasis is on actually knowing the information, not just cramming it and forgetting it.” Several of the programs visited apply competency or mastery-based assessment, meaning that students must demonstrate understanding of the material before they can move on to the next lesson. Most programs using this type of approach require a minimum competency level of 75 percent or 80 percent in order to progress, with no option to pass a class with D-level work. Some programs are using competency-based assessment rather than giving students grades or test scores. At Sky Valley Education Center, students receive a grade of Honors, Pass or No Pass. Honors is more than a substitute for an A; it means high performance and going above and beyond requirements. Students must teach a unit or do an extra project or other additional work to be awarded an Honors score. At the Independent Learning Center, students present exhibitions publically at the end of each trimester, with parents, teachers and members of the community invited to sit on the panel. Students’ individual learning plans and advancement are adjusted or decided based on achievements that student demonstrates and the panel’s recommendation on their readiness to proceed to the next level.Staff at programs using alternative approaches to student assessment noted that, despite a lack of emphasis on test scores and grades, their students perform well on a variety of measures of student success. Nova High School and the Kent Phoenix Academy have developed tracking systems to help them closely monitor and track student progress, and both report that their approach allows them to view and document each student’s growth over time, such as how many credits a student was behind when entering the program versus now, and number of graduation requirements completed. Several other programs noted that their students perform well on state tests, have a high percentage of students who enroll in college through Running Start programs, graduate high school with their associate degree already earned, or receive scholarships to further their education.

You don’t just get a D or F and be done. You have to keep redoing it until it’s an A or B. And then you understand what an A or B paper is, and you understand what’s expected of you.

~ ALE parent

Benefits of alternative approaches to student assessmentALE teachers said they are able to monitor students closely to ensure they are grasping course material. Students said that not having to worry about test scores or grades in their classes alleviates a lot of the pressure and stress that they felt when enrolled in a traditional school. This allows them to focus on understanding the material rather than on merely accumulating credits or passing tests.


Alternative learning spacesAlternative learning spaces provide students with the environment they need to be successful. For example, Independent Learning Center’s facility is a former residential house. In a different approach, Kent Virtual High School at Kent Phoenix Academy is located in a former junior high school. It offers a traditional school-like atmosphere for students who prefer this environment when on site, but it also offers students with significant school anxiety an opportunity to experience such an environment in small doses. After gradual exposure, they may be able to tolerate being on campus for a full day in one of the other two alternative programs also located on the same campus, or even transfer back to their local neighborhood school if desired. Several students seeking acceptance with their gender identity said they were looking for a place to fit in. Some ALE programs go a step further to provide an open and accepting learning environment for LGBTQ students: two schools, Avanti and Nova, provide gender-neutral restrooms. Nova also provides a “gender resource closet” to allow students to dress according to their preferences and feel more comfortable while attending school.Several of the ALE programs we visited expand the learning environment beyond the home and classroom. Students at the Independent Learning Center are offered internships in which community members mentor them two days a week and sometimes even beyond graduation. The Opportunities Program in Eastmont School District also has a mentorship option where high school students serve as mentors in an elementary or middle school classroom. The hours students put in as mentors qualify them to apply for paraeducator certification, and some have been hired by the school district after graduation as full-time employees.

Alternative approaches for increasing student engagement Some ALE programs are a “school of choice” where prospective students must apply and interview with program administration to demonstrate interest in attending and their compatibility with program aims. Staff and administrators at these programs said that defining their school as a school of choice has increased motivation because the student actively chooses their school rather than simply being told to go to school.

Since it’s less stressful, I feel less anxious and I actually no longer have depression since I started going here. It’s so much easier to understand and enjoying learning in an environment like this.

~ ALE student

Benefits of alternative learning spacesStudents who had been bullied at their traditional school now describe school as a safe place where they can focus on their education rather than fear for their safety. Administrators described students who had been unable to leave their mothers’ side or even come into the building, but now are able to thrive and socialize. Students who have received instruction through community service opportunities or internships report feeling better prepared for the future than they did in a traditional school environment. Auditors observed shy students who had struggled to speak in interviews gain confidence and take charge in their mentorship classrooms.

Everyone who’s here chose to be here, and even that simple thing makes it feel so much different than a normal high school.

~ ALE student

Identity was why I came here. There’s such a sense of conformity just to survive in other schools—I’ve never seen bullying here. You can dress the way you want, you can be intellectual without being judged.

~ ALE student


Students at several ALE programs play a very active role in the administration of their school. For example, each year Sky Valley Education Center surveys students, parents and staff on which courses they would like to take or teach. There are more than 350 elective courses to choose from, and the courses with the most nominations are scheduled. Students at Nova High School play an active role in many aspects of school governance, serving on committees and even on hiring panels for new teachers. Students from Eastmont Opportunities’ mentorship program say they feel motivated to come to school every day because they do not want to disappoint the young students they mentor, and it motivates them to do their own work. Similarly, students who have established a strong bond with their ALE teacher say they are motivated to complete their schoolwork because they know their teacher cares about their well-being and genuinely wants them to succeed.

Finding innovative solutions to common challenges of ALE instructionThe ALE programs we visited share many of the same challenges as programs we visited in the first phase of the audit. Some have developed ways to address limited resources, the perceived stigma of alternative education, and the needs of students who require extensive support.Stretching limited resourcesSome programs have found ways to address problems of space and money. Several shared their facility with other alternative programs, or were housed on the same campus as the local traditional school. Sharing resources not only helps hold down costs, it also allows programs to pool resources so they can offer important services that a single program cannot afford on its own, such as tutors or counselors. Other programs applied for and received community grants or used local levies and bonds to fund new facilities. For example, Independent Learning Center uses space donated by the local community, and Three Rivers Homelink moved to a new building with the use of bond dollars. Avanti plans to renovate and expand its site through bond funding. To maximize valuable teacher time, Kent Virtual High School at the Kent Phoenix Academy employs a Site Coordinator whose primary role is to closely track and monitor each student’s progress, be the local “face” for their contracted curriculum/teacher provider, and to make sure all ALE paperwork is completed. Instituting a coordinator staff role may help other ALE programs ease the workload of teachers, who are often tasked with administrative and reporting duties in addition to their teaching responsibilities.

The teacher I work with in the classroom will tell me that my little 1st-grader was crying yesterday asking why I wasn’t there. It makes me want to come to school every day.”

“Making kids feel like they should do their school [work] makes me want to do it myself or I’d feel like a hypocrite. I want them to do better so they don’t have to deal with what I did. I didn’t want them to follow in bad footsteps.

~ Eastmont Opportunities students

Benefits of alternative approaches to increasing student engagementTeachers and administrators at the ALE programs we visited believe their approaches improve student retention and graduation rates for their school districts. Staff at one program observed reductions in graffiti and other vandalism over the school year as new students develop a connection to their school and become proud of it. We saw no defaced school property or misplaced trash at any of the programs we visited.


Some ALE programs reported collaborating with community businesses and organizations to enrich the educational experience. For example, Sky Valley Education Center coordinates with employees of well-known technology companies to offer courses in computer programing and robotics and enrich learning. These community professionals volunteer under the direction and supervision of state-certificated instructors to develop and deliver the curriculum covered in the classroom.Changing the perception of alternative learningPrograms reported that communicating what they do, and how it helps students, has been key to dispelling inaccurate perceptions of ALE in their communities and school districts. Students from the Independent Learning Center gave a presentation about the program at a district school board meeting to help the superintendent understand the program’s needs. Students and administrators said that participating in community service projects or internships with local businesses gives students an opportunity to obtain real-world experiences to complement their educational framework. Such projects also help change their reputation for the better, as the community gets to know these youths as helpers and contributors rather than “delinquents.” Serving students with extensive support needsDuring both phases of the audit, teachers and administrators reported that being an impromptu counselor or social worker is becoming part of their regular required duties. Few, however, receive any specialized training to educate these students, and most have little knowledge of available community resources. ALE programs reported a need for an on-site social worker, substance abuse counselor or mental health specialist. Some ALE programs tackle this issue by providing resources and specialized expertise directly on campus; staff explained that social services designed to target student needs helps promote success. Students commend these programs for providing the framework that allows them to address their issues and be successful in school.

Auditor note: ALE programs reported providing additional services for students through the use of public/private partnerships and awarded grant funds. This audit did not verify the use of these funds, nor did we assess service delivery. We do not conclude that these services result in positive student outcomes.

Additional social servicesSeveral of the programs have food or clothing banks on site, and the Methow Valley School District hired a psychiatrist to inform instructors on how to work with students suffering from Post Traumatic Stress Disorder. Several other programs have an on-site mental health worker, or regularly bring one in to provide easy access to therapy for students. Programs said this lessens the burden on instructional staff who are generally not trained as social workers. Students at Kent Phoenix Academy can make appointments with a mental health counselor or doctor at the on-site teen health clinic, which also includes a lab and a pharmacy. Nova High School wants to re-establish the teen health center they had at their previous location to serve the school’s large population of LGBTQ students. They said it will be important to recruit a medical provider who understands what body dysphoria is, because students won’t seek medical help from someone who doesn’t understand who they are.

We are not all juvenile delinquents and criminals.

~ ALE student


Discovery Alternative High School has established several need-based partnerships within the greater South Kitsap community. One of these is a partnership with Olympic Educational Service District 114 to provide on-site services for teen parents, including day-care and instruction on life skills, parenting and kindergarten readiness. Their partnership with Coffee Oasis offers a homeless youth shelter and employment opportunities, such as vocational assistance, training, internships and other preparations for employment.Additional emotional supportResearch shows the presence of a caring, positive adult role model is associated with numerous positive outcomes such as greater resilience, lower stress, lower likelihood of arrest, reductions in homelessness, better employment outcomes, lower rates of delinquent conduct, better overall health, less suicidal ideation, reductions in rapid repeat pregnancies and better outcomes for the children of teen mothers. For students at risk of dropping out, success in school may depend on the development of caring relationships with other students, teachers, staff or administrators. The atmosphere of an ALE program may be particularly important for these students. Administrators said that many of their students feel anxious in large groups, or have experienced some sort of trauma in their young lives. They stated such issues are best served by small class sizes and strong student-teacher relationships, which are more difficult to foster in a traditional educational setting. Students and teachers explained that students in ALE settings bond and develop relationships with their teachers by having more one-on-one communication. Students are more willing to confide personal issues to a trusted teacher, and ALE teachers are therefore more likely to notice when a student is having an off day and can adjust or intervene accordingly. Students said the nature of the alternative learning style, individualized instruction, and smaller class sizes help build genuine relationships between students and teachers. Trauma-informed instructionSome ALE programs are specifically designed to help students who have endured what the federal Centers for Disease Control and Prevention terms “adverse childhood experiences (ACEs).” Neuroscience research has found that childhood trauma disrupts a child’s brain development, impairing his or her social, emotional and cognitive abilities and affecting both behavior and the ability to learn. Childhood trauma is also associated with longer-term impacts, such as chronic disease, mental illness and premature death in adulthood. Discovery Alternative High School uses a trauma-informed education practice that was first implemented at a Walla Walla high school and featured in the documentary Paper Tigers in 2015. This approach has since been applied successfully in schools across the United States. Some of the other ALE programs we visited also use trauma-informed instruction. Staff at these programs say this approach allows teachers to reduce the impact of trauma on students and improve student outcomes. Researchers report dramatic improvements in graduation and post-secondary enrollment rates, and significant reductions in behavioral incidents at schools using trauma-informed instruction.

“’Trauma-informed’ approaches are not new – they have been implemented in many fields including the medical profession and our judicial system … At the heart of these approaches is the belief that students’ actions are a direct result of their experiences, and when students act out or disengage, the question we should ask is not ‘what’s wrong with you,’ but rather ‘what happened to you?’ By being sensitive to students’ past and current experiences with trauma, educators can break the cycle of trauma, prevent re-traumatization, and engage a child in learning and finding success in school.”

~ Education Law Center

https://kpjrfilms.co/paper-tigers/


Innovation has its own challenges In addition to describing their innovative approaches to education, many ALE program staff noted that innovation has its own challenges. For example, programs that are mastery- or competency-based must find ways to calculate a grade-point average (GPA). One student said this can make applying for scholarships or showing progress to parents more difficult. Programs also emphasized that finding the right staff personalities is critical to the success of their approach. For example, staff need to be open to trying new things, understanding that their role may differ from the way they were trained. Additionally, teachers providing multidisciplinary instruction must be willing to collaborate and work closely with their peers. Teachers reported that it takes time and a lot of patience to win the trust of students who have experienced significant life events or come in mistrusting the educational system. It also takes an emotional toll on staff. For example, staff at Nova High School receive counseling for secondary trauma they may suffer in response to the difficult personal experiences students sometimes share with them.Establishing and maintaining community partnerships is not a simple task and is usually resource intensive. Teachers reported that they do not have the necessary time or training to develop relationships with community organizations. The Opportunities Program is adding a part-time staff member expressly to free up the lead ALE teacher to focus on monitoring the mentorship program and to help expand it to other high schools. Advocating for and obtaining needed resources is also an intensive effort, and even when resources are approved, they may not last. In one instance, an ALE program reported receiving approval for a counselor, but this person was repositioned to the district’s main high school. As a result, students attending the ALE program could not see this counselor for help. Some programs expressed an interest in expanding so more students could benefit from their approach. Even if all the resources for expansion were readily available, they still need to consider, how big is too big? Expansion must be carefully balanced with the very nature of what makes their program successful – a small, personal environment with substantial one-on-one interaction between students and teachers.

We have trouble deciding what level of partnership and what partnership, and how to maintain it. That’s like asking a teacher in any school to develop a partnership with a business. We’re not normally required to do that, it’s so outside of our classroom expertise. We’re expected to be a marketer/PR/counselor/advisor – not just a teacher.

~ ALE teacher


Recommendations

The inability to accurately identify ALE students in CEDARS is an issue that must be addressed for purposes beyond this legislatively mandated audit. Researchers, educators, policy makers and other members of the educational system need accurate and complete data on a per-student level to adequately monitor and evaluate student outcomes. Sufficient supervision and oversight of ALE programs cannot occur if OSPI does not know how to contact the ALE programs that are currently in operation, nor will they be able to provide guidance to new ALE programs as they are forming. Consistent and accurate data also demonstrate accountability in the use of public funds. The State Auditor’s 2018 Accountability Report, summarizing results from compliance audits of ALE programs over a four-year period, provides some recommendations to address these concerns. In order to be able to measure outcomes of ALE instruction, we recommend OSPI:

1. Identify and address factors that hinder school districts from accurately reporting ALE student coursework in CEDARS

2. Engage school district leadership to increase compliance with ALE reporting in CEDARS as required by RCW and WAC

In order to be able to measure long-term outcomes for all students, we recommend OSPI:

3. Collaborate with the Education Research & Data Center (ERDC) to ensure OSPI data in the ERDC data warehouse is complete and accurate

http://www.sao.wa.gov/local/Pages/SchoolsProgram.aspx

ALE: Student Characteristics and Innovative Approaches :: Agency Response | 35

Agency Response

ALE: Student Characteristics and Innovative Approaches :: Agency Response | 36

ALE: Student Characteristics and Innovative Approaches :: Appendix A | 37

Appendix A: Initiative 900

nitiative 900, approved by Washington voters in 2005 and enacted into state law in 2006, authorized the Office of the Washington State Auditor to conduct independent, comprehensive performance audits of state and local governments. Specifically, the law directs the Auditor’s Office to “review and analyze the economy, efficiency, and effectiveness of the policies, management, fiscal affairs, and operations of state and local governments, agencies, programs, and accounts.” Performance audits are to be conducted according to U.S. Government Accountability Office government auditing standards. In addition, the law identifies nine elements that are to be considered within the scope of each performance audit. The Office of the Washington State Auditor evaluates the relevance of all nine elements to each audit. However, because this mandated audit was descriptive in nature, several of the I‐900 elements do not apply to this particular effort. The table below indicates which elements are addressed in the audit. Specific issues are discussed in the Results and Recommendations section of this report.

I-900 element Addressed in the audit1. Identify cost savings No. Not applicable2. Identify services that can be reduced or

eliminatedNo. Not applicable

3. Identify programs or services that can be transferred to the private sector

No. Not applicable

4. Analyze gaps or overlaps in programs or services and provide recommendations to correct them

No. While the audit explored gaps and overlaps in ALE course type definitions, we did not analyze gaps or overlaps in programs or services.

5. Assess feasibility of pooling information technology systems within the department

No. Not applicable

6. Analyze departmental roles and functions, and provide recommendations to change or eliminate them

Yes. We were not directed to analyze department roles and functions in this legislatively mandated audit, but we did analyze OSPI’s role regarding supervision of data quality and made recommendations for steps the agency can take to improve student data.

7. Provide recommendations for statutory or regulatory changes that may be necessary for the department to properly carry out its functions

No. While the audit makes no recommendations for statutory or regulatory changes, we discuss issues related to ALE course type definitions that are defined in WAC and RCW.

8. Analyze departmental performance data, performance measures and self-assessment systems

Yes. The audit analyzed information gathered directly from ALE programs and those who participate in them to determine whether these programs are meeting their intent. However, we were not able to independently verify the success of these programs due to data quality concerns.

9. Identify relevant best practices No. The audit identified ALE education practices that were reported as innovative, but did not have the necessary student outcome data needed to determine whether these are indeed effective and can be cited as “best practices.”

ALE: Student Characteristics and Innovative Approaches :: Appendix B | 38

Appendix B: Methodology

Although we could not measure student outcomes quantitatively during the second phase of this performance audit, this report discusses outcomes as reported by students, parents, teachers, counselors, administrators and other support staff. In doing so, we also describe approaches that were reported as unique or innovative, intended to address the diverse needs of the state’s student population.

Quantitative analyses The Office of Superintendent for Public Instruction (OSPI) gathers a substantial amount of information on students enrolled in public schools across the state. The Education Research and Data Center (ERDC) within the Office of Financial Management (OFM) regularly matches student data with data on post-secondary outcomes (such as employment, two- and four-year college enrollment, apprenticeships and other indicators) in its data warehouse. The data we received for this audit was provided by the ERDC from its data warehouse. We requested academic records and state assessment scores for all students enrolled in public schools at any time during the 2013-14, 2014-15 or 2015-16 school years. For purposes of this audit, we defined an ALE student as anyone enrolled in one or more courses of any type, as indicated by the “ALE Course Type” field in the CEDARS Student Schedule table.To determine whether or not individual ALE students could be identified, auditors compared student headcounts as reported in CEDARS to the monthly report of school district enrollment eligible for basic education funding (P-223 report) through ALE. The annual unduplicated count of students who have taken one or more courses through ALE should be at least as high as, or higher than, the maximum number of ALE students the school district reported for funding during any particular month in the school year. However, in both the first phase and the current phase of this audit, auditors discovered that ALE student counts from CEDARS were substantially lower than they should be.

Qualitative analyses During this phase of the audit, the team employed several qualitative data collection strategies including the use of electronic surveys, interviews, and focus groups to evaluate ALE instruction. Participation in these activities was voluntary. The collection of approaches was designed to help answer the following questions:

• What are the characteristics of specific populations of students who enroll in ALE programs?• What are the characteristics of ALE programs that are implementing innovative approaches to

educating the state’s diverse student population?

ALE program survey We surveyed 264 ALE programs to obtain descriptive information not available in data systems. This information helped to broaden our understanding of ALE programs and the types of students enrolled in these programs. We received a 100 percent response rate for the survey. Information gathered in the survey served as the initial backdrop for the site visits, telephone focus groups and additional survey work described below.

Site visitsBased on the results from the ALE program survey, we identified programs that said they were providing a unique or innovative educational experience. We also considered programs recommended by leaders of OSPI’s Alternative Learning Department (ALD) and the Washington Association for Learning Alternatives (WALA), and any ALE programs that have received Washington’s Innovative Schools designation from OSPI. To further assess these programs, we developed a screening protocol and held interviews with 16 programs. Based on these interviews, we narrowed our selection to eight programs to visit in person.

ALE: Student Characteristics and Innovative Approaches :: Appendix B | 39

During each site visit, we held focus groups with district and program administrators, instructional staff, support staff, counselors, students and parents. Each of these groups provided information that allowed us to learn more about the program’s approach to delivering the educational experience and the barriers that other programs may face if they were to replicate this same approach.

Telephone focus groupsAuditors collaborated with Illuminate Evaluation Services to administer a series of telephone focus groups with a selection of ALE programs that reported having five or more students with a particular characteristic, as reported in their ALE program survey. Programs selected to participate in a focus group included the top three programs by prevalence of a certain characteristic as well as other programs having five or more students with that characteristic that were randomly selected. This approach was intended to gain the perspective of programs that focus on a particular student population, as well as programs integrating these students into a more general population. Each of these focus groups was designed to assess the benefits and challenges of working with a particular type of student and the reasons why these students seek ALE instruction.

Parent/student surveyDue to the risk of low participation from parents and students with certain characteristics during site visits and concerns about confidentiality laws regarding educational information, we developed an anonymous survey in order to collect information regarding their experiences with ALE. We asked the eight ALE programs visited to distribute a cover letter with a survey link to parents/guardians and students having one or more characteristics under study during this phase of the audit. We received anonymous responses from 115 ALE students and 131 parent/guardians of ALE students. We combined the responses by student characteristic to identify common themes.

ALE: Student Characteristics and Innovative Approaches :: Appendix C | 40

Appendix C: Statewide Results from ALE Program Survey

Figure 2 – Student characteristics

These figures are illustrations taken from an online presentation of all 2017 ALE Program Survey results from across the state. To view the full Tableau presentation, please visit the State Auditor’s Office website at www.sao.wa.gov/local/Pages/ALEstudy_FinalResults.aspx.

Figure 1 – Program characteristics


ALE: Student Characteristics and Innovative Approaches :: Appendix D | 41

Appendix D: Summary of Telephone Focus Group Findings

Auditors conducted telephone focus groups with a selection of ALE programs to assess the benefits and challenges of working with students who have particular characteristics. One goal of the focus group sessions was to learn the reasons why these students seek ALE instruction. The following is a summary of themes that emerged during those focus groups, organized alphabetically. Above Grade Level & Graduate Early – Students who are taking one or more ALE classes above grade level, or who enrolled in ALE to graduate from high school early

• Online work masks the age of the student, whether they are ahead or behind• Online programs allow access to higher level courses that might not be available at the local

traditional school• Instead of graduating early, some students participate in Running Start, so 15-year-olds who

have already accumulated 24 high school credits can still be high school students• Flexibility and self-pacing allows advanced students to stay challenged

Autism – Students with a diagnosis of Autism Spectrum Disorder• With ALE, there is less overstimulation (such as crowds, bells, social pressure)• ALE classes may be serving many more students with an undiagnosed Autism Spectrum

Disorder because parents may be trying to avoid ‘labeling’• With ALE, there is less demand to follow social norms

ӽ Teachers can let the student focus on what they’re interested in rather than a prescribed lesson plan

ӽ Rituals and self-stimulation behaviors are tolerated and accepted ӽ Teachers can use time differently, adapting to individual student pace for transitions, rather

than using a standard schedule

Becca Bill & Unwilling to Return – Students who are being monitored by the courts for truancy under the Becca Bill, or who are unwilling to return to their former school due to a highly aversive or traumatic experience

• Many of these students have social anxiety or 504 plans• Reduced in-person attendance time is helpful for students with anxiety• Smaller learning environment allows students with severe anxiety to feel more comfortable

participating

Below Grade, Credit Deficient & Credit Retrieval – Students who are taking one or more ALE classes below their grade level, have insufficient credits to graduate from high school in four years, or who are retaking a course previously failed

• ALE allows for more one-on-one help• Credit recovery adds workload and ALE provides flexibility so students can take more than a

full course load• ALE can offer below-grade-level courses to fill skill gaps

Dropout & Post 5th – Students who previously dropped out of school and have reengaged with the school system, and students still enrolled after attending high school for five or more years

• Students experienced stigma in traditional school, felt embarrassed or too old • One-on-one instruction is helpful


English Language Learner (ELL) – Students enrolled in the ELL program, formerly known as English as a Second Language (ESL)

• Programs said it was typically more difficult to educate ELL students in an ALE environment, but may be easier in online courses if the vendor offers translated curriculum or if advancements in technology for text-to-speech translation are available

• Staff/teachers receive specialized training to work with these students (Auditor’s note: One of the few characteristics where staff/teachers received specialized training)

• Inclusiveness provides greater acceptance for other cultures • More difficult to keep parents engaged, particularly if they’re monolingual

ӽ Parents play a significant role in ALE because much of the education is outside the schoolroom – language and cultural barriers make this more difficult

Head of Household & Pregnant/Parenting – Students that are the primary wage earner supporting their birth family, or who are pregnant or parenting (mother or father)

• These students need a flexible schedule to meet other demands• Programs wish they could provide quality day care for students’ children. Often students bring

their children to school with them, but this only works for infants – as young children become more mobile, they require more attention and can be disruptive

• Important for programs to keep in contact with these students if they have to be away from school for an extended period of time

• Teachers don’t know what community resources are available and need more access to social workers

Homeless – Students who do not have a permanent residence currently or within the past year• These students often remain unidentified for a long time in traditional schools• Transitioning from one traditional school to another traditional school often results in credit

loss because of differences in semester schedules• These students often have little support at home and are worried about parents, or are on

their own• These students already miss out on much of the high school experience, such as yearbooks,

prom, sports

Individualized Education Plan & 504 Plan – Students that have an IEP or 504 Plan in place for a disability that requires special education services

• The ALE model allows programs to get to know their students better and the learning style that works best for them

• Flexibility makes it easier to individualize instruction and provide accommodations for a particular student

• Students don’t miss classwork as they would when pulled from their regular classroom to receive services

• The most common accommodations in these plans (such as smaller environment, more time) are things ALE programs provide to all students


Juvenile/Criminal & Drug/Alcohol – Students who have a juvenile justice or criminal record, or who have a history of drug or alcohol issues

• When students go into juvenile detention, ALE programs need to un-enroll them. As a result: ӽ Students get behind on their work ӽ The ALE program has to reintegrate them when they return, but ALE programs can work

closely with the detention center and teachers/counselors can visit students to keep them connected

• Programs can assign partial credit for work completed before or during detention• Closer oversight of the student allows earlier detection and referral for drug or alcohol issues• Some ALE programs provide drug and alcohol counselors on site

Physical Health & Mental Health – Students with physical health or mental health issues that interfere with regular school attendance

• With ALE, mobility is less of an issue for students with physical limitations because they don’t have to move from one class to another

• Education can be put on hold for periods of time as needed – work is self-paced, staff can bring work to the student, there are multiple entry points, not just the start of semester (Auditor note: Also mentioned in Head of Household & Pregnant/Parenting, in Juvenile Justice/Criminal & Alcohol/Substance Abuse and in Homeless focus groups)

• Smaller, quieter setting with more personalized relationships and lower expectations for being on campus and doing group work fits the needs of students with school anxiety (Auditor note: High anxiety was mentioned in many of the other focus groups as well)

Room in Schedule & External Pursuits – Students who are enrolled in ALE to create room in their schedule to take electives or other educational options, or who have external pursuits or commitments that interfere with typical school hours

• These students take core courses online to make room for in-person electives• Contact time required in traditional school becomes a barrier to participating in education• Frequent communication and engagement are key for these students• State funding is limited to 1.0 student FTE so schools need levy or other dollars to offer more

than the standard student courseload

Running Start & Skills Center – Students enrolled in college courses while still in high school through the Running Start program, or who are enrolled in a preparatory program at a Skills Center

• ALE allows access to a broader range of courses typically not available in smaller schools• Easier to meet graduation requirements and schedules through the flexibility of ALE• Students can get started on college/career earlier• Transportation and travel time to colleges/Skills Centers may be an issue• Students may need to miss college classes to take state assessments -- some programs are

offering night testing to address this• Ongoing communication with colleges is a challenge; communication is better with

Skills Centers• Running Start requires independence and maturity and many students – even those in ALE –

are not ready

ALE: Student Characteristics and Innovative Approaches :: Appendix E | 44

Appendix E: ALE Program Profiles

This appendix contains short profiles of the eight ALE programs visited during the audit. Each program reported using unique or innovative approaches to delivering educational experiences. The auditors would like to thank the district and program administrators, teachers and staff, as well as students and parents, all of whom shared their insights and experiences.

Program name (District) page

Avanti High School (Olympia) __________________________________45

Discovery Alternative High School (South Kitsap) __________________46

Independent Learning Center (Methow Valley) ____________________47

Kent Phoenix Academy (Kent) _________________________________48

Nova High School (Seattle) ____________________________________49

Opportunities Program (Eastmont) _____________________________50

Sky Valley Education Center (Monroe) ___________________________51

Three Rivers HomeLink (Richland) ______________________________52

Auditor note: The audit did not independently verify individual statements for accuracy, nor do we intend to represent the opinions expressed as factually representative of all ALE programs in the state. Because it is not possible to measure and corroborate the stated outcomes from these approaches, we do not assert these approaches represent “best practices.”


Avanti High School (Olympia) What is innovative?Avanti originally began as a dropout re-engagement program. The school’s former principal visited Alaska’s Chugach School District and brought back a performance-based education model designed to sharply improve academic performance and graduation rates. Over time, staff at Avanti revised and enhanced the model by incorporating integrated art, a “Habits of the Mind and Heart” program and experiential learning to transform from a re-engagement program to a school of choice. Avanti focuses on mastery learning and requires a grade of 80 percent or higher to complete all assignments. Students receive a brain preference inventory and Myers-Briggs personality assessment to explore the way they learn and receive information. Instruction is often interdisciplinary. Avanti has “the cave,” a section of hallway that students had turned into a cave-like setting. While constructing it, teachers integrated instruction on art, chemistry, history and archeology. Before it was a cave, the hallway was a brain. Student art projects are displayed throughout the school. Teachers and administrators believe this contributes to the strong sense of connection students have with their school. Service learning is also emphasized at Avanti. Some students serve as counselors at the district’s fifth-grade Cispus outdoor learning camp, and through a partnership with GRuB (Garden-Raised Bounty, a local non-profit), students maintain a garden to provide fresh fruits and vegetables for the neighboring elementary school, which has a high proportion of low-income students. Avanti has a high LGBTQI population and is part of the Safe School Coalition. The school provides a gender-neutral bathroom.

BenefitsStudents said that beyond passing tests or accumulating credits, Avanti helps them really understand what they are learning. Parents said that redoing assignments until they get a good work product is what people are expected to do on the job. They also said it shows students how an A or B paper should look, which helps them produce high-quality work the first time around on their next assignment. Avanti reported it has won many awards for its graduation rates and state test scores: 100 percent of its Special Education students passed state tests last year, and 16 of last year’s 38 graduates went on to four-year colleges with more than $1 million in scholarships.

Challenges for establishing similar programsAccording to Avanti’s teachers and program administrator, a challenge of providing interdisciplinary learning is finding the right staff. It is difficult to find teachers who are endorsed in several subjects, and especially difficult to find those who are able to teach advanced level classes in multiple subjects. Teachers also need to feel comfortable working closely together and have the time necessary to collaborate on lesson plans.

“The cave” takes over an Avanti High School corridor.Photo courtesy of Avanti High School.


Discovery Alternative High School (South Kitsap) What is innovative?Discovery Alternative High School, established in 1973, has fostered several need-based partnerships within the greater South Kitsap community. One of these is with Olympic Educational Service District 114 (OESD 114). This partnership allows the program to enroll teen parents and provide daycare and specific instruction including life skills, parenting skills and kindergarten readiness. Discovery is also able to provide teen parents with access to sexual assault assistance, mentors, health services, financial assistance and college readiness programs. Students in the teen parent program are required to attend school four days a week. All other Discovery students must attend two days a week, and are required to complete 15 hours of schoolwork on the days they are not on campus. In addition to the partnership with OESD 114, several non-profit organizations provide additional support to homeless students at Discovery. Coffee Oasis provides vocational assistance, training, internships and other preparations for job seekers. It also offers shelter and employment opportunities to homeless youth. Discovery also has other organizations in the community that provide additional assistance. For example, Standup for Kids provides breakfast items for students. Other groups such as Girls of Color provide leadership and training for students of color, particularly girls.

BenefitsBecause this program requires most students to attend school only two days a week, the program offers smaller class sizes. According to staff, this allows them to build meaningful relationships with students, which often is a new experience for their students. They have found students become more engaged over time due to these relationships. Another benefit mentioned about the smaller class sizes is that it provides a framework for students to receive the services they need. Discovery has students who are homeless, lack food, have mental health needs, and struggle with substance abuse.

Challenges for establishing similar programsDue to the school’s schedule, staff do not see students every day. Program staff reported that this presents a challenge because students, many of whom are considered high risk, sometimes do not show up for school for extended periods. Staff explained that they have to count on the students having the drive to attend regularly. To help mitigate this challenge, prospective students are required to apply for admission to the school and go through a formal interview process. Staff explained that this process gives administrators and teachers an opportunity to assess a student’s fit for the program. This process also helps staff see how committed prospective students are to participating in the program. Administrators also expressed that budgetary constraints and allocation of district resources pose challenges.

Discovery students and children at the on-site teen parent child center.Photo courtesy of Discovery Alternative High School.


Independent Learning Center (Methow Valley) What is innovative?Almost 30 years ago, Independent Learning Center (ILC) began as a credit recovery program consisting of off-the-shelf curriculum and “packet based” learning. ILC has a new administration with a new vision and now partners with Big Picture Learning. The program consists of project-based learning and internships in which community mentors support students two days per week and even beyond graduation in some cases. Students advance only by demonstrating competency. At the end of each trimester, students present “exhibitions” to demonstrate what they have learned. Parents, teachers and members of the community are invited to participate on the evaluation panel. Students’ individual learning plans and advancement are assessed and adjusted based on the student’s demonstration and the panel’s recommendation on their readiness to proceed to the next level. Ninth and 10th grades are taught together by one instructor, and 11th and 12th grades are taught together by another. Students can choose projects based on their interests, and instructors incorporate this into assignments. Program staff stated that while most students here struggle on some level either academically or in their personal lives, ILC is open to any student who prefers this style of learning. A mental health counselor comes to ILC to work with students.

BenefitsAdministrators see students being more engaged because they have ownership over their learning and can choose topics and projects that interest them. Administrators and students said this better prepares them for college and life after graduation. In addition to helping students develop their interests, the internships help show the community that the students are not “problem kids.” Because of the change of reputation in the community, the program is now a choice program rather than a fallback option. Administrators also noted that small class sizes and one-on-one mentoring with their teachers and community mentors provide students with the kind of personalized attention that makes them feel that their instructors care. Students said that because their teachers respect them, it makes them, in turn, respect their teachers. One student said he volunteered to speak to the School District Board of Directors to demonstrate the importance of the school’s competency-based model. “This school has done so much for me,” he said, “I just wanted to give back and show people what this school is. I wouldn’t be graduating otherwise, so I was happy to give back.”

Challenges for establishing similar programsProgram staff report that funding is the biggest challenge. ILC has only two 0.8 full-time equivalent instructors and two part-time special education support staff. Staff noted that personalization requires a relatively low ratio of students to staff, and the school must use additional funding to maintain its current staffing at a 12-to-1 ratio. Transportation is also an issue, according to program staff. Students work in internships one or two days a week, and their transportation to those places must be reliable. Ideally, ILC would like to see this program cover grades 7 through 12, and while the demand exists, the school does not have enough funding. To obtain its current space and fund its Big Picture Learning support, ILC has had to obtain grants and community assistance.

An ILC student meets an owl with her mentor from the US Forest Service.Photo courtesy of Independent Learning Center.


Kent Phoenix Academy (Kent) What is innovative?Kent Phoenix Academy (KPA) consists of three distinct programs, one of which, Kent Virtual High School, is ALE funded. Virtual High School students have access to an extensive array of social service resources at their school. KPA has several partnerships, including King County Public Health, Kent Youth and Family Services, Kent Parks and Recreation, and Communities in Schools. Through these partnerships, students can access a teen health clinic (which includes a lab and pharmacy); mental health, drug and alcohol counseling; transportation assistance; a food and clothing bank; after-school activities; and academic tutoring. The school allows students to check out a laptop if they do not have access to a personal computer, and through a cooperative agreement with an internet provider, families participating in the program can receive home internet service for less than $10 per month. A private contractor provides the Virtual High School’s curriculum and teachers, but the school employs a local Site Coordinator who meets with each student weekly, helps them navigate the online instructional model, closely tracks student progress, and ensures all paperwork for ALE funding gets completed. The curriculum vendor provides training for the local Site Coordinator on supporting students in an online format, and establishes connections to Site Coordinators at other schools around the nation. KPA uses a locally developed electronic Student Learning Plan. The form gathers extensive information to closely monitor and report each student’s academic progress and growth, such as number of credits earned, progress toward meeting graduation requirements, and improvement on state tests. School administrators think these measures reflect their work more accurately than the typical metrics used to compare performance across schools.

BenefitsKPA is housed in a former junior high school, so students have a feeling of attending a traditional school when on site. The program administrator said that because the Virtual High School is housed with other site-based programs, it allows access to a variety of other supports that would not be feasible to provide for the Virtual High School alone. These wrap-around services provide necessary supports for students who need it so they can focus on their education. The Site Coordinator position provides a local “face” for Virtual High School students and eases the burden of ALE requirements for teachers, relieving them of the need to teach and keep track of all the paperwork at the same time. The Coordinator’s primary role is to support student success and make sure the program is meeting ALE requirements.

Challenges for establishing similar programsKPA is located in a metropolitan area and has good relationships with a variety of social service organizations. These resources may not be as readily available in a more rural setting. The program administrator said that establishing and maintaining these community connections is also a challenge, and having a dedicated staff member to do this work is crucial.

The Site Coordinator meets with a KPA student.Photo courtesy of Kent Phoenix Academy.


Nova High School (Seattle) What is innovative?Nova provides an academic program that features a competency- and project-based teaching and learning system, performance-based assessment, seminar-style courses, multi-level classes and independent study. Full or partial credit is given only for student work that is the equivalent of a B or higher. Nova reports it has a six-year extended graduation rate of 85 percent and won an OSPI award for extended graduation rates. To document student achievement and meet ALE funding requirements, Nova has created an internal database. Teachers track class attendance and assignments, and the database generates monthly reports on each student that calculates monthly progress and hours.Students play an active role in many aspects of school governance, serving on committees and on hiring panels for new teachers. In addition to learning individual and social responsibility, Nova students design their own programs of study within a non-graded, interdisciplinary structure. Students choose a learning project and then learn about core subjects as needed to carry out their project. For example, they might read topical literature, study the topic’s history, and utilize math and science principles to complete a project.Nova uses practices informed by trauma research; program staff stated that about two-thirds of Nova students have experienced two or more adverse childhood experiences. In addition, about 45 percent of its students identify as LGBTQ; the school has been using gender-neutral bathrooms for years without issue.

BenefitsNova consists of only 330 students. Program staff reported that this allows students to build relationships with staff, which fosters success in and outside of the classroom. Smaller class sizes also are beneficial to students who struggle with anxiety or are on the autism spectrum. Staff reported that students gain a sense of ownership and responsibility for setting their own path to success. Nova staff said they see students begin to advocate for themselves. Nova staff and students expressed that the school is a safe place. Many students said they were bullied at their previous, traditional high schools, and at Nova they get the support they need.

Challenges for establishing similar programsSchool administrators discussed several key challenges. The first challenge is that the school needs to receive an adequate level of district support. The second challenge is working with students who have been trauma-impacted, or do not trust the educational system. It takes some of these students time to recover from what they have been through. Some come in and immediately thrive, but others have to go through a process of trusting themselves and the school staff before they can succeed. Nova administrators said they are advocating for what they say is a needed teen health center. They stated this is particularly important because of the school’s large population of LGBTQ students. Many health care providers do not understand what body dysphoria is, and teenagers do not want to seek medical help from someone who does not understand them.

Nova High School in Seattle.Photo courtesy of Seattle Public Schools.


Opportunities Program (Eastmont) What is innovative?The Opportunities Program was established to serve students who have not been successful in traditional school or who require a different learning format or schedule due to personal hardships. Credit for coursework is project- and competency-based. Students must demonstrate 75 percent mastery to receive credit. The program allows teen parents to bring their infants to school with them. Eastmont Opportunities has a mentorship program, in which high school students serve as mentors in an elementary or middle school classroom to assist the certificated instructor.

BenefitsMany of the student mentors struggle with anxiety or personal problems outside of school and said that mentoring younger children provides them a sense of purpose. Those who previously had attendance issues said it makes them want to come to school every day so they do not disappoint the students they mentor. Administrators said the students think of themselves as employees who must show up to work on time. Some come in during their off time because they feel a sense of responsibility to the classroom and the children. We witnessed shy students who struggled to speak in interviews gain confidence and take charge in their mentorship classrooms. The elementary or middle school instructors whom the high school students assist said they could not get through their day without the help of these student mentors. Because of the hours they put in as mentors, students have the opportunity to get their paraeducator certification at graduation, and some have been hired by the school district after graduation as full-time employees.Six former students who returned solely for the purpose of being interviewed for this audit unanimously agreed that they would have dropped out of school if not for this program, and that it taught them responsibility and ownership for their learning. Two students said that it made them reflect on their own choices and the consequences of their actions. All six agreed that they felt like role models to the younger students and like they mattered to someone. Many students help translate, and working with the younger monolingual students helps them with their own language skills. The administrators said they know the program is working because they track attendance and graduation for these students, and they see positive changes in attitude and their overall learning progress. Administrators said that because of the mentorship program, some students now aspire to become teachers.

Challenges for establishing similar programsTeachers were skeptical at first about allowing ALE students to be with their elementary school children, but now they request the ALE students. Program administrators said this buy-in and cooperation is crucial for establishing this type of program. Resources and staffing can also present challenges. Eastmont is hiring a part-time staff member to give the lead ALE teacher more capacity to monitor the mentorship program and help expand it to other high schools. Administrators said that a person to facilitate logistics is also necessary. Finding the right personalities and passionate people to fill these positions could make the program difficult to replicate.

High school mentors in the Opportunities Program assisting a middle school class.Photo courtesy of Opportunities Program.


Sky Valley Education Center (Monroe) What is innovative?Sky Valley Education Center (SVEC) is unique in that it offers a wide variety of educational experiences in a multi-grade setting – including but not limited to the Parent Partnership Program (PPP), Excursion Program, Montessori program, Environmental Science School, STEM Academy and online curricula. Each year Sky Valley surveys students, parents and staff about what courses they would like to take, and they offer those that get the most nominations – more than 350 choices of elective courses in total. Parents are encouraged to attend classes with their child. SVEC also offers parent trainings and workshops on a variety of topics; such as, selecting curricula, providing instruction outside the classroom, improving reading and math skills, and strategies to help students with dyslexia, ADHD or executive function strategies.There are no letter grades at SVEC; coursework is assessed using a portfolio and presentation method, and students receive a grade of Honors, Pass, or No Pass. Honors is more than a substitute for an A. It means high performance and going above and beyond. Students must teach a unit or do an extra project or other additional work to be awarded an Honors score.

BenefitsAccording to program staff, the variety of classes that are offered allows any child to succeed in their education. Students are given the flexibility to pursue their particular interests. Parents are truly involved in teaching their children, and staff said having parents on campus gives students role models and helps parents see how their children are doing. Staff said that having no letter grades means that students do not have to worry about being an A, B, C, D or F student. They can focus more on learning the material. Sky Valley reported more than 90 percent of SVEC students passed the English Language Arts state assessment, and 60 percent of students access Running Start or enroll in a Skills Center. The program also reported that students’ average GPA for Running Start is 3.4, and about a quarter of graduates have earned their two-year degree, many with college honors.

Challenges for establishing similar programsStaff explained that the large number of course offerings at Sky Valley means much more preparation time for staff and that it would be easier to have a core set of classes. They went on to say that although this is a challenge, it is why many parents and students select the Sky Valley educational experience.

Students explore robotics in a Sky Valley class.Photo courtesy of Sky Valley Education Center.


Three Rivers HomeLink (Richland) What is innovative?Three Rivers HomeLink is a partnership between parents, students and staff, serving 12 school districts. It started as a resource for homeschool families to get more support. Nine years ago, the program opened with 23 students, using rented space in a church. It now has its own school building and serves nearly 500 students. Some students attend HomeLink only for extracurricular and elective activities, such as music and art. Other students attend classes where the parent wishes to receive extra support, such as higher-level math, career and technical education classes, science and foreign language. Elementary and middle school students are on site between 0 and 40 percent of the time, and high school students are on site between 0 and 60 percent of the time. The majority of 11th and 12th grade students at Three Rivers HomeLink enroll in Running Start or at Tri-Tech Skills Center. Three Rivers HomeLink serves families who choose to take an active role in their student’s learning. Parent involvement is a requirement in the program and is the primary reason students find great success. The principal told us the school logo is divided into thirds — parents, teachers and students — and all are expected to do 100 percent of their part to create a successful educational experience for the student. Three Rivers stands out most for its Science, Technology, Engineering, Art and Math (STEAM) program. Students in 7th and 8th grades can apply for admission to this project-based and interdisciplinary program. One of the projects we heard about involved students building their own musical instruments: The students studied sound wave frequencies and learned how marimbas are made while learning how to play this instrument. In another, the students designed chess sets. The students learned about geometric shapes, drew the chess pieces in three-dimensional shapes and used a 3-D printer to print their models.

BenefitsOn our site tour, the principal showed us a room full of robots and coding projects that first-graders were doing. In another classroom, elementary school students used laptops for research, and STEAM students conducted a physics “bridge test.” Students are encouraged to excel at their own pace. The school counselor said most students in the program struggle with unique challenges but often find success in the small environment, supported by the parent’s increased involvement, which creates a safe learning setting.

Challenges for establishing similar programsWorking so closely with the parents was unanimously described as an overall positive experience and the primary contributor to the program’s success; however, this also requires a high level of involvement from school personnel to intentionally seek input from families to develop a strong culture of collaboration “partnership.” A strong program requires a desire and willingness from staff to build relationships with the students’ parents. Three Rivers HomeLink regularly surveys families on topics ranging from the lunch menu to what courses will be offered for graduation. HomeLink staff take pride in being a customer service program and value the families’ feedback. When the program was developed, a rigorous curriculum approval process ensured all curriculum requests were secular and met state standards. As the program grows, another challenge is knowing how big is too big to maintain the small class sizes and personal attention that make the program successful.

Students playing the marimbas they made.Photo courtesy of Three Rivers HomeLink.