Embed Size (px)
Citation preview
1 / 27
California Educational Research Association 88th Annual Conference Formative Assessment: Implications for Student LearningSan Francisco, CA November 18, 2009
Jessica Ulloa
POWERSOURCE© Formative Assessment Practices Using the POWERSOURCE© Algebra Intervention: A View from the Classroom and Teacher Perceptions
CRESST/UCLA
2 / 27
1. POWERSOURCE© Overview and Implementation
2. Teacher Interview & Classroom Observations
a. Overview
b. Sample
c. Data Collection Methods
d. Results
3. Conclusion
PRESENTATION OVERVIEW
3 / 27
POWERSOURCE© INTERVENTION:
OVERVIEW
4 / 27
POWERSOURCE© Background
• Five-year research project
• Funder by the U.S. Department of Education, Institute of Education Sciences (National Assessment Center Grant)
• An algebra formative assessment intervention.
• SAMPLE:
150 Teachers , approximately 6,000 students
6th- 8th grade
8 participating School Districts : CA, AZ
Two Randomized control led groups
5 / 27
THE POWERSOURCE© INTERVENTION
6 / 27
POWERSOURCE© INTERVENTION: Professional
Development Component
GOALS :
Develop content knowledge
Develop skills in examining student work and explore how student thinking develops
Establish teacher-learner community
STRUCTURE: Three 90-minute sessions per domain (two-and-a half
month intervals)
Small groups (5-20 teachers)
7 / 27
POWERSOURCE© : Classroom Observations
& Teacher Interviews
8 / 27
TEACHER INTERVIEWS & CLASSROOM OBSERVATIONS
RESEARCH QUESTIONS:
1. Are teachers using POWERSOURCE© assessment data in a formative way?
2. How are teachers using POWERSOURCE© formative assessment information to guide their instruction?
3. What instructional practices are teachers implementing in the classroom to present POWERSOURCE© Big Ideas?
9 / 27
POWERSOURCE© Teacher Interviews & Classroom
Observations: METHODOLOGY:
10 / 27
Teacher Interviews & Classroom Observations
SAMPLE CHARACTERISTICS Conducted over a two year period for the 07/08 and 08/09 school year: Participation is voluntary and teachers receive a small stipend
Total Sample Size: n=15 Teachers
Total School Districts: 5 (CA, AZ)
Focus of Observations: Solving Equations Review & Application 6th grade/Treatment (POWERSOURCE©) Teachers
11 / 27
Interviews & ObservationsSAMPLE CHARACTERISTICS
Teacher Background:
1-2 years in PowerSource, 5+years of teaching experience (2 first year teachers) Hold clear and multiple subject credentials Most possess Bachelor Degrees 2 teachers with Masters degrees
Classroom composition: ELL, G.A.T.E, R.S.P Students Average classroom size: 25 students
12 / 27
Teacher Interviews & Classroom Observations
Data Collection Methods
OBSERVATION MEASURE : Semi-structured measure 40-50 minute classroom observation Teachers observed twice (29 separate observations) Paired Observations
OBSERVATION COMPONENTS:1. Implementation of Big Ideas2. Teacher Behaviors3. Lesson Structure4. Use of POWERSOURCE© materials
13 / 27
Teacher Interviews & Classroom Observations
DATA COLLECTION METHODS
INTERVIEW MEASURE Semi-Structured measure One 15-30 minute audio recorded interview
INTERVIEW COMPONENTS :1.Teacher Lesson Reflections2. Formative Use of Assessments3. Lesson Structure4. Professional Development
DATA ANALYSIS: Qualitative data was coded and analyzed using Atlas Ti. Interviews coded independently by two coders Observation frequency data counted and coded across all four observation components.
14 / 27
POWERSOURCE© Teacher Interviews &
Classroom Observations:
FINDINGS & TRENDS
15 / 27
POWERSOURCE© Teacher Interviews: FINDINGS & TRENDS
SAMPLE INTERVIEW QUESTIONS:Category: Use of Formative Assessment:
1.“How did you use the 1st (or last) Check for Understanding to plan for today’s lesson?”
2.“Based on today’s lesson, what do you think your steps will be instructionally?
3. “What other sources of information, other than the POWERSOURCE © assessments, do you typically use to identify your student strengths and areas of difficulties in math?
16 / 27
Interview Measure Category: Use of Formative Assessment
The majority of teachers (n=12) reported using the checks for understanding to identify student areas of difficulty & areas of strength.
Teachers (N=15) reported centering subsequent POWERSOURCE© lessons around student’s areas of difficulty—as evidenced by the Checks for Understanding .
Teachers (n=12) reported “quickly glancing through” student assessments (Checks for Understanding) to identify gaps in student understanding of POWERSOURCE © math concepts.
POWERSOURCE © Teacher Interviews Preliminary Results:
17 / 27
POWERSOURCE © Teacher Interviews Preliminary Results:
LIMITATIONS: Use of Formative Assessments
Based on interview responses (n=12) , teachers’ appear to struggle with articulating how to adjust instruction based on formative assessment information.
Teachers did not identify specific actions or strategies used to plan instruction and/or to present the POWERSOURCE© big ideas.
Common teacher responses tied to POWERSOURCE© instruction planning and methods sound similar to practices already being used for district modules.
18 / 27
Classroom Observations: Teacher Behaviors: RESULTS YEAR 1
MOST Observed Teacher Behaviors
Uses guided problem solving
Questions for Understanding
Provides procedural feedback
Provides procedural feedback + correct solution.
LEAST Observed Teacher Behaviors
References Checks for Understanding, quiz/test/
References homework results
Discussion of misconceptions
Demonstrates incorrect solution methods
19 / 27
Classroom Observations: Teacher Behaviors: RESULTS YEAR 2
MOST Observed Teacher Behaviors :Questions for Understanding
Uses guided problem Solving
Demonstrates correct solution methods
Provides Procedural feedback
Provides Procedural feedback plus correct answer
LEAST Observed Teacher Behaviors:References Checks for Understanding, quizzes/test or homework results
Discussed Misconceptions
Demonstrates incorrect solution method
20 / 27
Classroom Observations & Teacher Behaviors Year 1 and Year 2 Comparison
21 / 27
POWERSOURCE © Classroom Observations & Teacher Interviews
CONCLUSIONS: Teacher interview responses and observation data, indicate that teachers are using formative assessment to gauge student understanding of math concepts.
Observation data shows that teachers consistently and over-time apply similar instructional strategies to present the Big Ideas in math.
Teachers seem to struggle with using Formative Assessment as an instrument to guide their instruction.
Results suggest that teachers may be at the beginning stages of understanding the use of formative assessment
Teachers may still be struggling with transferring and applying content knowledge taught at professional development trainings into the classroom.
22 / 27
Teacher Interviews & Classroom Observations: NEXT STEPS
Questions to Further Research:
1. Is nine hours of professional development training sufficient to produce change in teacher’s pedagogical methods and math content knowledge?
2. Is the appeal of the Big Ideas overshadowing the POWERSOURCE© Formative Assessment Piece?
24 / 27
