Embed Size (px)
Citation preview
Fundations® Implementation Evaluation This report was created in April 2017 for the Park City Education Foundation. In the following pages, we seek to provide an overview and evaluation of initial implementation of Fundations® at McPolin Elementary School (MPES) in the Park City School District (PCSD). The purpose of the evaluation was to assess implementation of the Fundations® program and its impact on teaching and learning at McPolin Elementary School. The evaluation required collection of both qualitative and quantitative data collected between September 2016 and March 2017. Data used to inform findings include: individual teacher reflections, grade-level team reflections, teacher, coach, and principal interview data, and student achievement data (including Dibels and iReady data). Evaluators chose to assess several aspects related to the implementation and impact of Fundations® including:
• Ease of implementation • Impact on school culture • Impact on teaching, specifically reading instruction • Impact on student achievement in literacy
Overall, the positive outcomes and impact of the program, which are described in detail throughout the report that follows, serve as indicators of the program’s success in its initial year of implementation. As teachers have become more knowledgeable about Fundations® and had time to practice implementing with ongoing support, reflection, and feedback, the program has led to increased consistency in how foundational literacy skills are taught, the amount of time spent explicitly teaching these skills, and the overall confidence of both teachers and learners. At this early stage, it is too soon to say with confidence whether the program will have substantial, long-lasting impact on student achievement, but early data suggest that Fundations® is having a positive effect on student learning. Feedback from teachers and school leadership overwhelmingly support the idea that the school’s culture of collaboration, data use, and evidence-based instruction have improved because of implementation of the Fundations® program. For questions about the information contained in this report, please contact the program implementation evaluator, Allison Miller. Allison Miller Ed Direction [email protected]
2
Project Overview The Need Park City School District believes that teachers have Core Promises they must keep for all students. One of these Core Promises is to provide effective reading instruction for all students. In addition, the district’s newly-formed Dyslexia Initiative1 seeks to empower teachers with additional tools and insights to better support any student who struggles with reading by providing access to targeted instruction, and/or intervention sooner, more effectively, and with less protentional turmoil. In the spring of 2016 PCSD stakeholders, in partnership with the Park City Education Foundation, PC READS, and the Ty and Karen Hall Foundation, began considering how literacy instruction in early grades (K-2) could be improved to prevent the compounding effects of children developing weaknesses in foundational skills, which over time give rise to deficits in higher-level reading skills. A steering committee was formed to further consider the need for an explicit, systematic program for teaching literacy in the lower grades. The committee agreed that it would be informative to pilot a program at one elementary school in the district so that they could have a deeper understanding of the feasibility and impact of implementing a literacy program aimed at providing students in K-2 classrooms with more explicit instruction on the foundational skills for reading and spelling.
The Response Recognizing the need for more explicit instruction in reading and spelling, the steering committee considered multiple program options for implementation. After extensive research exploring the feasibility of implementation and potential impact on student learning, the committee agreed to pilot the Fundations® reading program at McPolin Elementary in the 2016-2017 school year. Beginning in August of 2016, MPES introduced Fundations® to all students in kindergarten, first, and second grades, with a plan to extend implementation to third grade in the fall of 2017.
Why Fundations®? “Fundations® is a “multisensory and systematic phonics, spelling, and handwriting program that benefits all K-3 students.”
Fundations® aligns with PCSD’s ELA curriculum maps2 and provides a comprehensive approach to teaching reading. It is integral to McPolin’s RTI/MTSS model, providing scientifically based instruction in Tier 1 (whole group) as well as repeated exposure to accurate, repetitive phonics instruction in small groups (Tier 2) for students needing additional support in phonemic awareness and/or phonics. Fundations® allows teachers to narrow their focus for those students who need additional tools and time to read. The table below provides an overview of the model currently being implementation at MPES.
Tier 1: Universal Instruction at McPolin Elementary
• Delivered to all students in general education classrooms for 20-30 minutes per day; led by classroom teacher
• Flexible based on the needs of students; allows teachers to extend lessons from one day to the next to ensure that all skills and concepts are introduced
• Accessible to all students because concepts are taught and practiced in multiple learning modalities
• Includes end of unit of unit assessments for monitoring progress
1 More information about PCSD’s Dyslexia Initiative can be found here. 2 More information about Park City School District’s ELA Curriculum Maps can be found here.
3
Tier 2: Small Group Differentiated Instruction at McPolin Elementary
• Targeted lessons in small groups for additional 30 minutes per day, 3-5 times per week; led by classroom teacher
• Differentiated for students based on their individual needs Fundations® acknowledges the diversity of reasons for why a child might struggle to learn, with a specific learning disability being only one of several possible causes. Teachers can identify and intervene early to prevent students from developing more invasive reading deficits, and more accurately identify those students who struggle to learn because of a specific learning disability. Fundations®, combined with PCSD’s Core-aligned English Language Arts Curriculum Maps, makes for an integrated and comprehensive approach to reading, writing and spelling.
The Pilot3
Overview Implementation of Fundations® at MPES was designed to target teachers and students in kindergarten, first and second grades for the pilot. In year one, a total of 186 students received instruction using the Fundations® program. A total of seven teachers are using the program as their primary model for providing systematic, phonics, based instruction. Beginning in the fall of 2017, two third grade teachers implement the program as well.
Grade # of Teachers Implementing Fundations®
Kindergarten 3
First 2
Second 2
Total 7
Training As part of the pilot, kindergarten, first, and second grade teachers at McPolin Elementary School participated in a one-day summer training. Julie Hastings, the district’s program manager who oversees implementation of Fundations® also participated in this training. Additional trainings—including on-site coaching and feedback from a Fundations® specialist and Orton-Gillingham training—were also provided. The table below outlines each training session that was offered.
Date Training Topic
June 2016 Wilsons Fundations® Language Training
Introduction to Fundations®
August 2016 MPES Professional Development MPES teachers review materials
September 2016 MPES Professional Development MPES update on implementation; Share ideas on organization and how to best organize classroom and materials
September 2016 Fundations® Coaching Fundations® coach on -site to model whole group Tier 1 instruction and lead teachers in study group
3 A complete overview of Fundations 2016-2017 Working Plan can be found here.
4
October 2016 MPES Professional Development Planning Fundations® Time; Exploring the online support component, using data trackers, implementing Unit Test data trackers, and shared strategies for teaching Trick Words
November 2016 PC READS Dyslexia Awareness Presentation4 to MPES staff
December 2016 Orton-Gillingham (O-G) Training5
January 2017 MPES Professional Development Planning Fundations and integrating multisensory strategies from O-G
February 2017 Fundations® Coaching Fundations® coach on site to model Tier 2 small group instruction, observe teachers and offer feedback, and lead teachers in study group
March 2017 MPES Professional Development Planning instruction with Fundations®
March 2017 MPES Professional Development Plan for district roll out
Materials:
• The Wilson Coaching Framework • PAST and Core Phonics Assessments • Wilsons Fundations® Materials • Orton-Gillingham Materials • Tier 1 Walkthrough Protocol • iReady • Research on Foundational Skills by What Works
Clearinghouse
Stakeholder Engagement
Date Meeting Summary
June 2016 Pilot rollout
August 2016 Creation of pilot implementation plan
September 2016 Steering Committee6 update and planning
November 2016 Steering Committee update and planning
February 2017 Steering Committee update and planning
February 2017 School Board presentation on Dyslexia Initiative
March 2017 Meeting with principals and team to discuss rollout challenges, questions, champions
March 2017 Steering Committee discussion about success factors
March 2017 School Board presentation to respond to questions about Dyslexia Initiative
March 2017 Curriculum Director and discuss rollout, implementation plan and budget
May 2017 Board Meeting Presentation: Implementation Plan for 2017-2018 and Pilot Assessment
Budget7 4 As part of the Dyslexia Initiative, each elementary school in the PCSD will receive dyslexia awareness training from PC READS. 5 To further enhance teacher training, a representative from each elementary school attended an intensive training on the Orton-Gillingham approach, “an instructional approach intended primarily for use with persons who have difficulty with reading, spelling, and writing of the sort associated with dyslexia.” This training was specifically selected by the steering committee for proven track record helping students with language processing problems associated with dyslexia. More information about the Orton-Gillingham approach can be found on the organization’s website. 6 Steering Committee members: Elissa Aten, PC READS; Bob Edmiston, MPES Principal; Karen and Ty Hall, funders; Julie Hastings, Program Manager; Abby McNulty, PCEF; Allison Miller, outside evaluator 7 Complete budget can be accessed here.
5
Line Item Cost
Fundations® Materials: Fundations® Classroom Sets (1 Teachers’ Kit and materials for 20 students) $9,272.00
Fundations® Summer Training (7 teachers) $1,813.00
Fundations® Out of State Travel $6,476.64
Fundations® Coaching (Coach on Site at MPES 2 times) $3,400.00
Orton-Gillingham (O-G) Training (5 teachers) $5,375.00
Sub Costs (5 days for O-G Training) $550.00
Additional Classroom Instructional Materials (MPES funds used to purchase) $233.63
Total Expenditure $27,120.27
Funding
Source Amount
Ty and Karen Hall Foundation, distributed by PCEF $15,000.00
Park City School District $12,120.27
Total Funding $27,120.27
Current State
Implementation To create a full picture of implementation of Fundations® at McPolin Elementary School, evaluators collected
qualitative data from all participating teachers, Julie Hastings and the Bob Edmiston, principal. In addition, student
achievement data including Dibels data from 2015-2016 and 2016-2017 and iReady data (2016-2017) were
collected. The summary below is intended to provide a look at implementation in year 1 from the perspective of
teachers, the instructional coach, and the school principal. The student achievement data is included to provide
stakeholders a quick look at the current state of student achievement related to literacy. The data do not
necessarily represent a direct correlation between implementation of Fundations® and student achievement. As is
the nature of a mid-year implementation review, total impact cannot yet be determined. End of year student
achievement data will provide additional insights into the impact of Fundations® on student learning at MPES.
Current State of Implementation
Teachers at MPES completed two separate reflections about the implementation of specific components of the
Fundations® program. The charts below show increases in all areas of implementation from the first reflection
(November 2016) to the second (March 2017). Most notable are the increases seen in classroom setup, lesson
planning, learning activities being presented in a prescribed order, and the use of Unit Tests and Trackers.
6
Impact on Teaching When interviewed, teachers voiced that Fundations® is set apart from other approaches used in the past in the following ways:
• Targeted, ongoing professional development that includes time for coaching, feedback, and collaboration was provided to participating teachers
• Program promotes vertical alignment between grades K-2 and encourages increased collaboration among teachers both within and across grade levels
• Instruction is explicit and structured to ensure that all students have opportunities to engage in learning activities and demonstrate their understanding
• Rationale for language rules is provided for the students • Teachers have access to real-time student data that helps them strategically group their students and plan
for small group instruction • Tier 2 instruction is more streamlined and intentional • Teachers can group students based on needs, not just who is in their classroom. Teachers feel confident
moving students to different groups (often in other classrooms) to ensure that each student is getting the differentiated instruction he/she needs
7
School leadership (including the principal and instructional coach) report that the implementation of Fundations® has impacted teaching at MPES this year in the following ways:
• The program has provided on-going professional learning for teachers that is job-embedded • There is a more focus on explicit, systematic phonics instruction and Fundations® has empowered the
teachers to improve their instruction related to foundational skills of literacy • Implementation has created alignment and a consistent focus for instructional practice in kindergarten,
first, and second grades • There is fidelity to implementation amongst the whole group of teachers (grades k-2) • Teachers are fully responsible for and capable of teaching Tier 2 differentiated instruction to students in
their classroom • Fundations® has created a unified conversation around teaching • Teachers are regularly collecting student achievement data and using the data to inform instruction and
student grouping • Teachers have more understanding into why specific skill deficits exist and the time & resources necessary
to offer the 2nd dose of instruction for the students who need it based on Tier 1 instruction and student data
• Teachers can identify and intervene early to prevent students from developing more invasive reading deficits
Julie Hastings shared that “collaboration among the teachers has been the fuel that drives the enthusiasm with this pilot. With Fundations®, teachers have a common language with a detailed understanding of the process that is systematic and vertical. Because of the additional resources and techniques, teachers feel more confident in working together and problem-solving to identify next steps with instruction when they encounter a student who is struggling. Common tools better encourage collaboration during this learning phase.” Mr. Edmiston also adds that the biggest benefit of having Fundations® at MPES is the impact that it has had on school culture. He shared that the program has "empowered the teachers to be directly involved in the success of implementation. It has created a level of trust between teachers, administration and parents.”
Impact on Learning When interviewed, teachers reported the following positive impacts that Fundations® in having on student learning at MPES:
• Students are learning the rules of the English language, rather than just memorizing spelling and learning to site read (a skill that are difficult for approximately 40% of all learners8)
• Students are transferring learning to other portions of their day. Teachers report that students are using strategies learned during Fundations® instruction in other aspects of their learning.
• Students appear to be making greater gains in literacy in a shorter amount of time in comparison to previous years. Teachers report that day-to-day student progress seems to be more advanced this year than in years past.
• Students have more confidence in their abilities. Teachers report that students are exhibiting greater confidence in decoding, spelling, and writing and note that all students are exhibiting increased willingness to participate in whole-class instruction.
• Whole-class engagement has increased substantially. All students are aware of expectations and are participating in a variety of modalities, including, visual, auditory, kinesthetic, and tactile. All students are given equal access to participate in learning.
• 100% of teachers report that “most” students are engaged in lessons in the following ways: o Students smoothly transition from one activity to the next
8 Research reports that 40% of students require explicit instruction (including phonics) to become capable readers. http://files.eric.ed.gov/fulltext/ED498005.pdf
8
o Students follow directions throughout the lesson o Students tap, as directed o Students manipulate letter tiles, as directed o Students actively participate by responding and doing
• Students have access to a variety of resources that allow them to be more active learners. • 100% of teacher report that “most” students are effectively using resources in the following ways:
o Resources (i.e. posters/notebooks) are visible and referenced by students as needed o Students manage materials, following effective routines
As part of the quarterly check-in report (collected in quarter 1 and quarter 2, teachers reported that Fundations® is having a positive impact on students at MPES. In both reports teachers rated the programs impact on students a 4 on a 5-point scale.
Teachers also report the following in relation to student mastery of specific components of Fundations®:
Total Number of Students
Very Few
Some Approx.
Half Most
Students demonstrate mastery of previously taught concepts for decoding Students demonstrate mastery of previously taught concepts for spelling Students demonstrate success with new material and challenges
Students demonstrate application of taught skills in Composition Books
Students’ letter formation demonstrates mastery and neatness
School leadership (including the principal and instructional coach) report that implementation of Fundations® has impacted student learning at MPES this year in the following ways:
• Using Fundations®, with the current Problem Solving Process (RTM30), has provided an explicit way of documenting teacher decisions around individual student needs. With this initiative, McPolin has been able to focus their efforts in kindergarten, 1st & 2nd exclusively on what students need and what is truly, most effective for helping them learn to read.
• Fundations® has impacted learning in that it has offered explicit instructional practices that have been transferred to student behaviors. For example, “students are tapping out sounds as a strategy.” – Bob Edmiston, principal
iReady Data iReady data from beginning of year (BOY) and middle of year (MOY) show improvement in phonics placement and phonological awareness placement across all three grades. A complete data set for these two areas can be accessed here. Increases in student achievement on iReady include:
Phonological Awareness
9
Kindergarten 29 students in kindergarten were on benchmark for Phonological Awareness at the beginning of the year. The number of students on benchmark at middle of year increased to 45, with 16 of those students performing above grade level.
1st Grade 19 students in first grade were on benchmark for Phonological Awareness at the beginning of the year. This number increased to 36 students at benchmark on MOY testing, with 15 students (up from 7 students at BOY) achieving the max score.
2nd Grade 47 second graders tested out at the beginning of the year, and increased to 57 students testing out in the middle of the year. The number of students below grade level in Phonological Awareness decreased from 22 (BOY) to 14 (MOY).
10
Phonics
Kindergarten 24 students in kindergarten were on benchmark for Phonological Awareness at the beginning of the year. The number of students on benchmark at middle of year decreased to 23, however, the number of students above grade level increase from 5 (BOY) to 10 (MOY).
1st Grade 24 first graders were on benchmark for Phonics at the beginning of the year, and increased to 25 students on benchmark for the middle of the year. Additionally, 19 students scored above grade level on MOY testing compared to 5 students achieving this level on BOY testing.
11
2nd Grade 18 second graders were on benchmark for Phonics at the beginning of the year, and 17 second graders were on benchmark in the middle of the year. MOY data show that 13 students were above grade level on MOY testing, an increase from 4 students above grade level on BOY testing.
Dibels9
Kindergarten Composite scores from DIBELS show that kindergarten students increased from 72% at or above benchmark composite score to 85% at or above benchmark composite score in the middle of the year. This 13% increase from BOY to MOY is consistent with the increase seen in last year’s composite score for kindergarten students at McPolin. 2016-2017 McPolin Elementary School
The tables below show how McPolin’s data compare to other elementary schools in the district.
9 McPolin’s Dibels Data for grade K-2 can be accessed here.
12
Percent of Kindergarten Students at Benchmark (Green): COMPOSITE SCORES
School BOY Composite Score MOY Composite Score Change
Jeremy Ranch 69% 58% -11%
McPolin 72% 85% +13%
Parley’s Park 65% 77% +12%
Trailside 81% 75% -6%
Percent of Kindergarten Students at Benchmark (Green): NONSENSE WORD FLUENCY (NWF) CLS (Correct Letter Sounds)
School MOY Nonsense Word Fluency CLS
Jeremy Ranch 69%
McPolin 87%
Parley’s Park 86%
Trailside 75%
First Grade First grade students decreased composite scores from 86% at or above benchmark to 74% at or above benchmark in the middle of the year. This decrease is typical for first graders because of the introduction of story-read. Last year’s decrease from BOY to MOY in first grade was 5%. The most substantial increases were seen in students’ fluency and accuracy.
2016-2017 McPolin Elementary School
13
The tables below show how McPolin’s data compare to other elementary schools in the district.
Percent of 1st Grade Students at Benchmark (Green): COMPOSITE SCORE
School BOY Composite Score MOY Composite Score Change
Jeremy Ranch 73% 72% -1%
McPolin 86% 74% -12%
Parley’s Park 69% 71% +2%
Trailside 75% 62% -13%
Percent of 1st Grade Students at Benchmark (Green): Nonsense Word Fluency (NWF) WHOLE WORDS READ (WWR)
School BOY NWF WWR Score MOY NWF WWR Score Change
Jeremy Ranch 74% 63% -9%
McPolin 81% 84% +3%
Parley’s Park 76% 70% -6%
Trailside 78% 83% +5%
Second Grade Second grade students’ composite scores decreased by 4% from BOY to MOY, compared to a 6% increase seen in the 2015-2016 school year. 2016-2017 McPolin Elementary School
The tables below show how McPolin’s data compare to other elementary schools in the district. Percent of 2nd Grade Students at Benchmark (Green)
School BOY Composite Score MOY Composite Score Change
Jeremy Ranch 70% 69% -1%
McPolin 61% 57% -4%
Parley’s Park 67% 69% +2%
14
Trailside 79% 72% -7%
Percent of 2nd Grade Students at Benchmark (Green) DORF FLUENCY
School BOY DORF Fluency Score MOY DORF Fluency Score Change
Jeremy Ranch 57% 57% ----
McPolin 53% 56% +3%
Parley’s Park 57% 59% +2%
Trailside 73% 68% -5%
Percent of 2nd Grade Students at Benchmark (Green) DORF ACCURACY
School BOY DORF Accuracy Score MOY DORF Accuracy Score Change
Jeremy Ranch 61% 59% -2%
McPolin 58% 53% -3%
Parley’s Park 62% 65% +3%
Trailside 65% 64% -1%
Highlights of Fundations® Overall, teachers expressed appreciation for the program. Comments from interviews voiced that the program was systematic in its approach, created a common language for all teachers, and provided explicit, direct instructions. When asked about the success of Fundations®, all teachers interviewed voiced that the program was either successful or very successful.10 The principal added, “As an overall score in reference to any other implementation that has happened recently…it’s a 5 plus” (exceeding the very successful rating). Mr. Edmiston contributes the success the school has experienced with implementation in year one to planning that has been done and the buy-in teachers have felt from day one.
• Student manipulatives are very engaging for all students, including “high performers” and “struggling”
readers. • Teachers report increased active student engagement during instructional time. • Fundations® provides an easy-to-access, systematic approach for teaching phonics. Teachers report
increased confidence in their ability to teach of phonics in direct response to students’ needs. • Fundations® helps teachers identify gaps in student learning. Teachers report that the program helps
surface specific skill deficits for individual students. The unit assessments that are included help teachers identify which students need additional support from the teacher.
• Fundations® provides materials for use during small group instruction time. Teachers appreciate having materials directly linked to Tier 1 instruction that they can use during Tier 2 instruction.
Challenges Teachers acknowledge that there are some challenged to implementing a new program. Overall, teachers expressed that the biggest challenge of the program was initial implementation- it took time and practice. Additionally, teachers voiced that it is necessary to “stick with it” to improve student growth. From the first quarter to the second quarter teachers report increased confidence in their ability to implement the routines and procedures effectively. They reported the following as the primary challenges faced this year:
• Unpacking & organizing the Fundations® materials takes a lot of time • Learning the routines and procedures for teaching lessons takes time and ongoing coaching and feedback
are needed • Teachers and school leaders must make time in the daily schedule for Fundations® instruction • Daily collaboration time for team members is essential
10 Teachers were asked to rate implementation on a 5-point scale from “1= Very Unsuccessful to 5= Very Successful”
15
Summary Overall, teachers and students have been positively impacted by the implementation of Fundations® at McPolin Elementary. The program is easy to successfully implement with minimal training. McPolin teachers and leadership agree that the success of implementation is due in part to teachers’ willingness to learn all that they could about a program they believed would have a positive impact on student learning. Teachers also voice the need for consistent support (from both Fundations® and a school “champion”) and adherence to the program (including consistency in when and how the lessons are taught). Teachers also report that successful implementation of Fundations® requires buy-in and ownership from all participating teachers. Because it is still in initial stages of implementation, it is too early to determine the exact impact of the program on student achievement. Qualitative data reveal, overwhelmingly, that teachers and school leaders see the value of Fundations® and its impact on both teaching and learning. Continued coaching, feedback and refinement of implementation has the potential to further impact student achievement. K-2 teachers and administration see the value in continuing Fundations® at McPolin and throughout the entire Park City School District.
