Upload
kyle-hahn
View
214
Download
0
Tags:
Embed Size (px)
Citation preview
Collecting and Analyzing Collecting and Analyzing Student Engagement Data as Student Engagement Data as A Key Component for School A Key Component for School Improvement, Organizational Improvement, Organizational
Learning, and Increased Learning, and Increased AchievementAchievement
West Virginia School Improvement SpecialistsAugust 18, 2010Jerry Valentine
Professor EmeritusUniversity of Missouri
Copy, print, or use only with written permission (via email) from Dr. Valentine ([email protected]).
Criteria for these slides…Criteria for these slides…
Sound, research based knowledgeSound, research based knowledge Practical use in schoolsPractical use in schools Practical use for you as a SI Practical use for you as a SI
specialist…you can use all, most, specialist…you can use all, most, some, a few of the slides with your some, a few of the slides with your schools.schools.
Perspectives on Student Perspectives on Student EngagementEngagement
Addressing Engagement is Addressing Engagement is critical to learning…here are critical to learning…here are
some basics:some basics:
Three Broad Forms of Student Three Broad Forms of Student Engagement in the ClassroomEngagement in the Classroom
Cognitive
Physical Emotional
Physical and Emotional Support CognitivePhysical and Emotional Support Cognitive
OUR FOCUS TODAY IS COGNITIVE OUR FOCUS TODAY IS COGNITIVE ENGAGEMENTENGAGEMENT
Logical Perspectives on Physical Logical Perspectives on Physical Presence, Mental Attention, and Presence, Mental Attention, and
Cognitive EngagementCognitive Engagement Attendance is a precursor to attention in class… Attention is a precursor to cognitive engagement… Cognitive engagement is a precursor to new learning for most
students (building knowledge, understanding and ability). A few students are physically present in class but consistently
cognitively absent. All students are, occasionally, physically present in class but
cognitively absent. For most students…attendance, attention, and cognitive
engagement are linked to learning through student motivation
For most students…motivation to cognitively engage is a function of:
Teacher-Student Relationships Emotional Security Content Relevance Challenging and Realistic Learning Experiences
Students are motivated to engage Students are motivated to engage when…when…
MotivationMotivation
See Value inSee Value in
the Learningthe Learning
Can make an Emotional/PersonalCan make an Emotional/Personal
Link to the Content/ProcessLink to the Content/Process
(Relate to Prior Knowledge/Experiences)(Relate to Prior Knowledge/Experiences)
Believe TheyBelieve They
Can Learn ItCan Learn It
Components of Meaningful Components of Meaningful Cognitive EngagementCognitive Engagement
Necessary ComponentsNecessary Components AttendanceAttendance in School in School AttendanceAttendance in Class in Class AttentionAttention during Learning during Learning
ExperiencesExperiences Cognitive Engagement Cognitive Engagement during during
Learning ExperiencesLearning Experiences Relevant and Challenging Relevant and Challenging
Learning ExperiencesLearning Experiences Student Self-Reflection/ Student Self-Reflection/
Assessment/Goal SettingAssessment/Goal Setting Student MotivationStudent Motivation to Attend and to Attend and
Cognitively EngageCognitively Engage
Supportive Processes Teacher builds Teacher-
Student Relationships School Protection of
Classroom Learning Time Teacher Use of Allocated
Classroom Learning Time Teacher Design of
Relevant Challenging Lesson
Teacher is Enthusiastic Supportive, Facilitative toward Student Effort
ATTENDANCE: SCHOOL-CLASSROOMINTEREST IN CONTENT AND
ACTIVITIES
Emotional Security to Engage
Physical Presence Necessary to Engage
Emotional Commitment to Engage with Effort
ENGAGEMENT AND ACHIEVEMENT: THE BIG ENGAGEMENT AND ACHIEVEMENT: THE BIG PICTUREPICTURE
R E L A T I O N S H I P S
CHALLENGING RELEVANT LEARNINGKNOWLEDGE
DEVELOPMENT“SURFACE”
EMBED BASIC FACTS/SKILLSCOMPREHENSIONAPPLICATION
THINKING DEVELOPMENT“H-O/DEEPER”ANALYSIS
PROBLEM-SOLVINGCRITICAL THINKINGCREATIVITY/INNOV.SYNTHESIS
Cognitive Engagement
Terminology and Basic Numbers… Academic School Day
Beginning to ending time for school--e.g. 8:00-3:25 (445 minutes) Allocated Learning Time
Scheduled in-class time What % of the Academic School Day is Allocated Learning Time? (85%-90%) (375-400 minutes)
Engaged Learning Time Students are truly cognitively engaged in the learning experience What % of the Allocated Learning Time is Engaged Learning Time? (80-90%) (300-360 minutes) Surface Learning Time
Simple comprehension, fact-finding, recall, and skill development/practice
What % of Engaged Learning Time is Surface Learning? (80-85%) (250-300 minutes)
Deeper Learning Time Analytical, critical, creative, innovative, and synthesis types of
thinking What % of Engaged Learning Time is Higher-Order Deeper Learning? (15-20%) (30-60 minutes)
Table Reflective Discussion The basics of engagement are neither
complex nor new… What information was affirming of
what you already knew? Did you hear or read something that
expanded your perspective about, or understanding of, student engagement? If so, what?
What stands out in your mind right now about student engagement??????
What stands out the most in What stands out the most in your mind right now about your mind right now about
engagement?engagement?
Engagement and Engagement and Achievement: Achievement:
The Classic StudyThe Classic Study(Yair, 2000)(Yair, 2000)
Yair Study of Cognitive Yair Study of Cognitive EngagementEngagement
Individual Student Engagement in Grades 6-8-10-12 Grade 6 in K-6 Schools and Grades 6 and 8 in K-8
Schools Grades 6 and 8 in Middle Schools Grades 10 and 12 in High Schools Students selected randomly and stratified by gender,
race, and ability level Data Collection Design
Digital wristbands Randomly buzzed 8 times daily for a week >3500 self-reports during in-class (allocated) learning
time Students Described:
Where are you? What are you doing? Who are you interacting with? What is on your mind?
How much are you concentrating? How challenging is it? How difficult is it? How interesting is it? How important is it to you? What else are you doing?Source: Yair, Educational Administration Quarterly, Vol. 36, #4 (October 2000)
Cognitive Engagement andCognitive Engagement andStudent CharacteristicsStudent Characteristics
Boys were 21% more likely to be engaged than girls.
Asian and white students were about 25% more likely to be engaged than African American and Hispanic students
Sixth graders are 29% more likely to be engaged than 12th graders; 23% more than 10th graders, and 6% more than eighth graders
The more students are engaged, the higher are their grade point averages
Source: Yair, Educational Administration Quarterly, Vol. 36, #4 (October 2000)
In which subject is cognitive In which subject is cognitive engagement the highest?engagement the highest?
Subjects: Mathematics English Reading Science Social Science
Rank order:
14325
Compared to Math and Science and Reading, students in English and Social Studies are more than 40% less likely to be engaged cognitively with the content
Source: Yair, Educational Administration Quarterly, Vol. 36, #4 (October 2000)
In which instructional method In which instructional method is cognitive engagement the is cognitive engagement the
highest?highest?Instructional Method
Teacher lecture Class discussions
(whole group T-led)
Laboratory (Hands-on work)
Group work Individualized
(seat) work Presentation
(creating/making)
EngagementRank
Passive for all
Active for all
Is the Learning Active/Passive?
Sources: Yair, Educational Administration Quarterly, Vol. 36, #4 (October 2000); Valentine (NSDC Conference (December, 2010)
6
5
1
2
4
3
Passive for most
Active for all
Active for all
Passive for all
Engagement and Instructional Engagement and Instructional MethodsMethods
When compared to teacher lecture learning experiences, the odds that students will be cognitively engaged are: 125% higher during group learning
experiences 115% higher during a laboratory learning
experiences 90% higher during class presentation
learning experiences 70% higher during individualized learning
experiencesSource: Yair, Educational Administration Quarterly, Vol. 36, #4 (October 2000)
Engagement in Relevant and Engagement in Relevant and Challenging Learning Challenging Learning
ExperiencesExperiences The odds of students being engaged during
the “most relevant” lessons were 108% higher than the lessons the students described as “least relevant.”
The odds of students being engaged during the “most challenging” lessons were 90% greater than when students identified the lessons as “least challenging.”
Relevance and challenge were the two most important lesson design strategies linked to student engagement in the study.
Source: Yair, Educational Administration Quarterly, Vol. 36, #4 (October 2000)
Engagement and Student Engagement and Student Recall…Recall…
The following slides are interesting, but the exact information reported is unique to the setting about which they are reporting. Do not take the findings literally across the board. Realize that they are being used herein as a way to make a point about engagement and recall, rather than as an empirical finding. You will see what I mean when I explain them…
And, as you look at the slides, realize that we all know this, both intuitively as well as through our own knowledge of how students learn and through our knowledge of general research about instructional methods.
Learning Pyramid: Recall in Two Learning Pyramid: Recall in Two Weeks from Weeks from Audio-VisualAudio-Visual Instruction Instruction
(Edgar Dale, Audio-Visual Methods in Teaching)(Edgar Dale, Audio-Visual Methods in Teaching)
Active
Passive
Read
HearLook at Pics
View movie, exhibit, demonstration
Participate in discussion; give talk
Do dramatic presentation; simulation, do real thing
10%
50%
70%
90%
20%
30%
Caveat: pyramid is considered more theoretical than empirical…use only to stimulate discussion about Passive and Active forms of learning.
How much we remember when…
1. We Say It 2. We Hear It and See
It 3. We See It 4. We Say and Do It 5. We Read It 6. We Hear It
Robert Pike, 1989; public speaker and adult trainer (Very similar to the Edgar Dale report)
412365 ---PASSIVE--- -ACTIVE-
Retention Rates and Instructional Methods
1. Practice by Doing 2. Reading 3. Lecture 4. Demonstration 5. Discussion Group 6. Audio-Visual 7. Teach Others and/or
Immediate Use of Learning
Warren (1989) “New Movement Seeks to Replace Rivalry in Class with Team Spirit,” Education.
Instructional Method
71542 63 --PASSIVE-- -ACTIVE-
Retention Rates and Instructional Methods
Instructional MethodInstructional Method Lecture Reading Audio-Visual Demonstration Discussion Group Practice by Doing Teach
Others/Immediate Use of Learning
Retention RateRetention Rate 5%5% 10%10% 20%20% 30%30% 50%50% 75%75% 90%90%
Warren (1989) “New Movement Seeks to Replace Rivalry in Class with Team Spirit,” Education.
Facts, Facts, Facts…Facts, Facts, Facts…
The previous slides were loaded with facts about engagement, most of which are research-based…
Which information surprised you the most? Why?
What surprised you the What surprised you the most?most?
The IPI Process for The IPI Process for Collecting and Collecting and
Collaboratively Studying Collaboratively Studying School-Wide Engagement School-Wide Engagement
Data Data
What is The IPI Process?What is The IPI Process? The Instructional Practices Inventory Process is
a set of faculty-led strategies for collecting valid/reliable student engagement data and for collaboratively studying the data with the goal of increasing and enriching learning experiences throughout the school.
The process serves a school best when teacher-leaders are the data collectors and the facilitators of the faculty collaborative study of the data.
When implemented with integrity, data analyses document that the process fosters instructional change and organizational learning.
Rationale for Collecting and Rationale for Collecting and Studying Engagement As A Studying Engagement As A
FacultyFaculty Student Engagement is clearly linked
to student achievement Collaborative Conversations are a
cornerstone to establishing a learning organization
Organizational Learning is a key to maintaining currency of knowledge and application of best practices
Development of the Instructional Development of the Instructional Practices Inventory ProcessPractices Inventory Processfor Profiling and Studyingfor Profiling and Studying
Student EngagementStudent Engagement When: 1995-96 Who: Bryan Painter co-developer Why: Originally to understand change in
instruction and engagement during a two-year comprehensive, systemic school improvement project of 30 schools
Data today available from thousands of schools representing hundreds of thousands of classroom observations
Data Collection Mental Image:Data Collection Mental Image:Collect Large Volume of “Snap Shots” Collect Large Volume of “Snap Shots”
of Student Engagement…of Student Engagement…
Collaborative Study Mental Image: Collaborative Study Mental Image: Faculty Analyze the Data, Problem Faculty Analyze the Data, Problem
Solve, and Design for ChangeSolve, and Design for Change
The IPI process…The IPI process…
provides the opportunity to provides the opportunity to create an OPTIMUM profile of create an OPTIMUM profile of student engagement in learning…student engagement in learning…
that teachers will view as fair and that teachers will view as fair and accurate, and thus…accurate, and thus…
use as a basis for periodic use as a basis for periodic reflective, problem-solving, reflective, problem-solving, collaborative conversations.collaborative conversations.
The IPI does not The IPI does not profile the types of profile the types of instructional instructional activities in which activities in which students are students are engaged.engaged.
The IPI profiles how The IPI profiles how students are students are engaging in engaging in learning during the learning during the instructional instructional activities.activities.
The Six IPI Student The Six IPI Student Engagement CategoriesEngagement Categories
H-0/Deeper Verbal Learning Conversations
Synthesis, Creativity/InnovationCritical AnalysisProblem SolvingAnalysis
All Other H-O/D Engagement
Pre-2004:18-20% Post-2004:
13-15%
APA 14 Learner-Centered Principles; ES: Construct Meaning; HS: Socratic
3-5%
18-20%
Students are attentive to teacher-led instruction
Independent or group work designed to build basic understanding, new knowledge or pertinent skills
3: Teacher is attentive to, engaged with, or supportive of students
2: Teacher is not attentive to, engaged with, or supportive of students
25-30%
5-10%
30-40%
30-40%
Students are not engaged in learning directly related to the curriculum
3-5% High Achiev: 0-1%Low Achiev: 8+; 10+; 20+
RecallMemorizationSkill PracticeFact-Finding
Disengagement
6 5 4 3 2 1
What does this visual imply about the six IPI What does this visual imply about the six IPI Categories ? Categories ?
NOT A NOT A HIERARCHYHIERARCHY
Six distinct categories…ways of classifying how students are engaged. A 6 is not better than a 5; a 5 is not better than a 4; etc. A 5 is not better than a 3, it is different than a 3 and there are times when the 3 is the most appropriate learning experience for the students.In a lesson or a unit, categories 6, 5, 4, 3 are all valuable learning experiences for the students.
Influences on Achievement?Influences on Achievement?
6 5 4 3 2 6 5 4 3 2 1 1Which category has the single-most
influence on student achievement? 1
Which two categories when combined have the most negative impact on student achievement?
1 2
Which two categories when combined have the most positive impact on student achievement?
5 6
The relative impact of 1-2 vs 5-6 is devastating… Categories 1-2 have slightly more than 2 times the influence of categories 5-6 on achievement in most grades.
H-0/Deeper Verbal Learning Conversations
Synthesis, Creativity/InnovationCritical AnalysisProblem SolvingAnalysis
All Other H-O/D Engagement
Students are attentive to teacher-led instruction
Independent or group work designed to build basic understanding, new knowledge or pertinent skills
3: Teacher is attentive to, engaged with, or supportive of students
2: Teacher is not attentive to, engaged with, or supportive of students
Students are not engaged in learning directly related to the curriculum
RecallMemorizationSkill PracticeFact-Finding
Typical Engagement Typical Engagement Percentages from the IPI Percentages from the IPI Data before IPI Process Data before IPI Process
ImplementationImplementation(Data from 2004-2009(Data from 2004-2009
…the NCLB era)…the NCLB era)
What schools (grade-levels) have What schools (grade-levels) have the highest percentages of Higher-the highest percentages of Higher-
Order/Deeper Engagement? Order/Deeper Engagement?
Early Early ChildhoodChildhood
ElementaryElementary MiddleMiddle HighHigh Voc-TechVoc-Tech Alternative HSAlternative HS
Typical Percentages:
18.4%
17.4%
16.5%
15.5%
35.1%
21.8%
What schools (grade-levels) have What schools (grade-levels) have the lowest percentages of Student the lowest percentages of Student
Disengagement? Disengagement?
Early Early ChildhoodChildhood
ElementaryElementary MiddleMiddle HighHigh Voc-TechVoc-Tech Alternative HSAlternative HS
Typical Percentages:
1.3%
3.1%
3.9%
6.2%
2.2%
2.3%
Typical Percentage--2004-2009 (> 46,000 Class Typical Percentage--2004-2009 (> 46,000 Class Obs.)Obs.)
IPICategory EC ES ML HS V-T-C Alt.
6 13.37 13.29 11.45 11.30 29.99 15.39
5 5.06 4.09 5.06 4.24 5.14 6.43
4 46.88 44.12 39.71 37.60 25.51 32.88
3 28.53 28.62 30.71 29.08 31.58 35.54
2 4.86 6.80 9.13 11.62 5.62 7.49
1 1.30 3.08 3.93 6.16 2.16 2.27
5-6 18.43 17.38 16.51 15.54 35.13 21.82
2-3 30.69 35.42 39.85 40.70 37.20 43.03
4-5-6 65.31 61.50 56.22 53.14 60.64 54.70
1-2-3 34.69 38.50 43.78 46.86 39.36 45.30
Practice Coding Examples Practice Coding Examples to Develop an to Develop an
Understanding of the Understanding of the Categories in the Categories in the
ClassroomClassroom
Classroom Coding Classroom Coding Examples…Examples…
Read the example and chat briefly Read the example and chat briefly with your neighbor…what do you with your neighbor…what do you think is the correct code?think is the correct code?
1—2—3—4—5—61—2—3—4—5—6
H-0/Deeper Verbal Learning Conversations
Synthesis, Creativity/InnovationCritical AnalysisProblem SolvingAnalysis
All Other H-O/D Engagement
Students are attentive to teacher-led instruction
Independent or group work designed to build basic understanding, new knowledge or pertinent skills
3: Teacher is attentive to, engaged with, or supportive of students
2: Teacher is not attentive to, engaged with, or supportive of students
Students are not engaged in learning directly related to the curriculum
RecallMemorizationSkill PracticeFact-Finding
IPI Example AAs you enter the 8th grade science classroom,
the students are listening attentively to the teacher give them simple directions for the litmus experiment they will begin in a few minutes. The teacher explains the process step by step. You learn from two students that the class also did litmus tests last month. As you begin to leave the room the teacher is re-explaining the main points of the process.
Which of the IPI categories should be coded for this example?
IPI Example AAs you enter the 8th grade science classroom, the
students are listening attentively to the teacher give them simple directions for the litmus experiment they will begin in a few minutes. The teacher explains the process step by step. You learn from two students that the class also did litmus tests last month. As you begin to leave the room the teacher is re-emphasizing the main points the students are to recognize and the steps they are to take when they do the litmus test.
IPI Code: Category 4—Students are attentive to Teacher-Led Instruction
H-0/Deeper Verbal Learning Conversations
Synthesis, Creativity/InnovationCritical AnalysisProblem SolvingAnalysis
All Other H-O/D Engagement
Students are attentive to teacher-led instruction
Independent or group work designed to build basic understanding, new knowledge or pertinent skills
3: Teacher is attentive to, engaged with, or supportive of students
2: Teacher is not attentive to, engaged with, or supportive of students
Students are not engaged in learning directly related to the curriculum
RecallMemorizationSkill PracticeFact-Finding
IPI Example BAs you enter the sophomore English
classroom, the students are creating (original) poems. The teacher is moving among the students encouraging them as they work. They have a rubric on their desks that clarifies expectations about rhyme, meter, imagery, content, emotion, and length. The students are defining a topic and creatively explaining it through poem. As you read their work, you are impressed with their depth of thought and emotion.
Which of the IPI categories should be coded for this example?
IPI Example BAs you enter the sophomore English
classroom, the students are creating (original) poems. The teacher is moving among the students encouraging them as they work. They have a rubric on their desks that clarifies expectations about rhyme, meter, imagery, content, emotion, and length. The students are defining a topic and creatively explaining it through poem. As you read their work, you are impressed with their depth of thought and emotion.
IPI Code: Category 6—Students are engaged in Higher Order/Deeper, Creative thought
H-0/Deeper Verbal Learning Conversations
Synthesis, Creativity/InnovationCritical AnalysisProblem SolvingAnalysis
All Other H-O/D Engagement
Students are attentive to teacher-led instruction
Independent or group work designed to build basic understanding, new knowledge or pertinent skills
3: Teacher is attentive to, engaged with, or supportive of students
2: Teacher is not attentive to, engaged with, or supportive of students
Students are not engaged in learning directly related to the curriculum
RecallMemorizationSkill PracticeFact-Finding
IPI Example CAs you enter the 4th grade math class
the students are seated at their tables completing a workbook assignment. When you look at their work you see they are independently computing practice division problems. They have been doing these types of problems off and on for months. The teacher is working at her computer creating a test and has her back to the students.
Which of the IPI categories should be coded for this example?
IPI Example CAs you enter the 4th grade math class
the students are seated at their tables completing a workbook assignment. When you look at their work you see they are independently computing practice division problems. They have been doing these types of problems off and on for months. The teacher is working at her computer creating a test and has her back to the students.
IPI Code: Category 2—Students are engaged in practice seatwork and the teacher is not attentive to, engaged with, or supportive of their work.
H-0/Deeper Verbal Learning Conversations
Synthesis, Creativity/InnovationCritical AnalysisProblem SolvingAnalysis
All Other H-O/D Engagement
Students are attentive to teacher-led instruction
Independent or group work designed to build basic understanding, new knowledge or pertinent skills
3: Teacher is attentive to, engaged with, or supportive of students
2: Teacher is not attentive to, engaged with, or supportive of students
Students are not engaged in learning directly related to the curriculum
RecallMemorizationSkill PracticeFact-Finding
IPI Example D
As the you enter the freshman honors history class, the students are watching selected segments of the movies Pearl Harbor and Midway. The students are not taking notes, just watching. The teacher is standing by the DVD/VCR player watching the segments with the students. You can tell from the books on the students’ desks that the class is studying WWII. You are in the room about two minutes.
Which of the IPI categories should be coded for this example?
IPI Example D
As the you enter the freshman honors you enter the freshman honors history class, the students are history class, the students are watching selected segments of the watching selected segments of the movies movies Pearl HarborPearl Harbor and and MidwayMidway. The . The students are not taking notes, just students are not taking notes, just watching. The teacher is standing by watching. The teacher is standing by the DVD/VCR player watching the the DVD/VCR player watching the segments with the students. You can segments with the students. You can tell from the books on the students’ tell from the books on the students’ desks that the class is studying WWII. desks that the class is studying WWII. You are in the room about two minutes.You are in the room about two minutes.
IPI Code: Category 3—Students attentive to video resource and teacher is attentive to students
H-0/Deeper Verbal Learning Conversations
Synthesis, Creativity/InnovationCritical AnalysisProblem SolvingAnalysis
All Other H-O/D Engagement
Students are attentive to teacher-led instruction
Independent or group work designed to build basic understanding, new knowledge or pertinent skills
3: Teacher is attentive to, engaged with, or supportive of students
2: Teacher is not attentive to, engaged with, or supportive of students
Students are not engaged in learning directly related to the curriculum
RecallMemorizationSkill PracticeFact-Finding
IPI Example EAs the you enter the high school art class,
the students are in small groups of four or five. Each group has a print of a classic painting and the students are discussing their analyses of the paintings. The discussions are stimulated by two “why” and “what if” questions written on the board that require collective analysis of the artwork. One student in each group is taking notes for the group. As you begin to leave the room two minutes after entering, you hear the teacher say that it is time to explain their group analyses and defend them to the whole class.
Which of the IPI categories should be coded for this example?
IPI Example EAs the you enter the high school art class, you enter the high school art class,
the students are in small groups of four the students are in small groups of four or five. Each group has a print of a classic or five. Each group has a print of a classic painting and the students are discussing painting and the students are discussing their analyses of the paintings. The their analyses of the paintings. The discussions are stimulated by two “why” discussions are stimulated by two “why” and “what if” questions written on the and “what if” questions written on the board that require collective analysis of board that require collective analysis of the artwork. One student in each group is the artwork. One student in each group is taking notes for the group. As you begin taking notes for the group. As you begin to leave the room two minutes after to leave the room two minutes after entering, you hear the teacher say that it entering, you hear the teacher say that it is time to explain their group analyses is time to explain their group analyses and defend them to the whole classand defend them to the whole class
IPI Code: Category 5—Students engaged in higher-order verbal learning conversations
H-0/Deeper Verbal Learning Conversations
Synthesis, Creativity/InnovationCritical AnalysisProblem SolvingAnalysis
All Other H-O/D Engagement
Students are attentive to teacher-led instruction
Independent or group work designed to build basic understanding, new knowledge or pertinent skills
3: Teacher is attentive to, engaged with, or supportive of students
2: Teacher is not attentive to, engaged with, or supportive of students
Students are not engaged in learning directly related to the curriculum
RecallMemorizationSkill PracticeFact-Finding
IPI Example FAs the you enter the first grade class, 12 of
the 17 students are organized in two reading groups of 6 each on one side of the room. Each student is reading silently. A parent volunteer is circulating among the two groups to help as needed with difficult words. When you talk quietly with a few of those students and the parent, you learn that the students have challenging stories to read and they are working on developing reading skill and story comprehension. The teacher is in the opposite side of the room reading softly to the other five students.
Which of the IPI categories should be coded for this example?
IPI Example FAs the you enter the first grade class, 12 of
the 17 students are organized in two reading groups of 6 each on one side of the room. Each student is reading silently. A parent volunteer is circulating among the two groups to help as needed with difficult words. When you talk quietly with a few of those students and the parent, you learn that the students have challenging stories to read and they are working on developing reading skill and story comprehension. The teacher is in the opposite side of the room reading softly to the other five students.
IPI Code: Category 6—Non readers learning to read and comprehend requires at least analysis to obtain meaning from the sentences.
H-0/Deeper Verbal Learning Conversations
Synthesis, Creativity/InnovationCritical AnalysisProblem SolvingAnalysis
All Other H-O/D Engagement
Students are attentive to teacher-led instruction
Independent or group work designed to build basic understanding, new knowledge or pertinent skills
3: Teacher is attentive to, engaged with, or supportive of students
2: Teacher is not attentive to, engaged with, or supportive of students
Students are not engaged in learning directly related to the curriculum
RecallMemorizationSkill PracticeFact-Finding
IPI Example GAs you enter the fourth grade language As you enter the fourth grade language
arts classroom it is obvious that the 24 arts classroom it is obvious that the 24 students are taking a paper and pencil students are taking a paper and pencil test. The teacher is seated at a student test. The teacher is seated at a student desk near the back of the room observing desk near the back of the room observing the students as they take the test. You the students as they take the test. You walk among the students for a few walk among the students for a few moments to read some of the test items. moments to read some of the test items. The test is a “recall” test, of multiple The test is a “recall” test, of multiple choice and fill in the blank items. The choice and fill in the blank items. The test is still in progress when you leave test is still in progress when you leave the classroom two minutes after entering the classroom two minutes after entering the room. the room.
Which of the IPI categories should be coded for
this example?
IPI Example GAs you enter the fourth grade language As you enter the fourth grade language
arts classroom it is obvious that the 24 arts classroom it is obvious that the 24 students are taking a paper and pencil students are taking a paper and pencil test. The teacher is seated at a student test. The teacher is seated at a student desk near the back of the room desk near the back of the room observing the students as they take the observing the students as they take the test. You walk among the students for a test. You walk among the students for a few moments to read some of the test few moments to read some of the test items. The test is a “recall” test, of items. The test is a “recall” test, of multiple choice and fill in the blank multiple choice and fill in the blank items. The test is still in progress when items. The test is still in progress when you leave the classroom two minutes you leave the classroom two minutes after entering the room. after entering the room.
IPI Code: Category 3—Students are engaged in recall-level seatwork with the teacher attentive to the students.
Collecting Engagement Collecting Engagement Data with Validity and Data with Validity and
ReliabilityReliability
Procedures for Collecting Engagement Data Procedures for Collecting Engagement Data with Validity and Reliability so Teachers with Validity and Reliability so Teachers
View the Data as Fair and AccurateView the Data as Fair and Accurate
Systematically move from classroom to classroom based upon the floor plan of the school.
Observe all learning settings proportionately across the school.
Code student learning during the first moments of initial entry into classroom as if you took a snapshot upon entry.
Focus on students, not the teacher. Code the predominant engagement pattern Collect a large volume of data points
throughout the school day
Procedures for Collecting Engagement Data Procedures for Collecting Engagement Data with Validity and Reliability so Teachers with Validity and Reliability so Teachers
View the Data as Fair and AccurateView the Data as Fair and Accurate Conduct data observations on “typical”
school days. Have candid faculty discussions about
“jazzing-it up” on data collection days. Select higher-numbered code when a clear
picture between two codes is not evident. Code during regular learning time, not
during transitions between content areas. Protect anonymity of all observations; never
link an observation with a teacher.
Who Should Collect the Data?Who Should Collect the Data? Teacher-Leaders Should Collect the DataTeacher-Leaders Should Collect the Data
Observations provide teachers with broader perspective Observations provide teachers with broader perspective about learningabout learning
Teachers are not evaluatorsTeachers are not evaluators Faculty embrace data more quickly when teachers collect Faculty embrace data more quickly when teachers collect
the data and facilitate conversations about the datathe data and facilitate conversations about the data Each school should develop an IPI Team of 3-5 teacher Each school should develop an IPI Team of 3-5 teacher
leadersleaders Regional Office and State Support Staff Can HelpRegional Office and State Support Staff Can Help
Short-term strategy to build interest/comfortShort-term strategy to build interest/comfort Most impact occurs when a school builds internal capacity Most impact occurs when a school builds internal capacity
to collect and study the datato collect and study the data Long-term support in design of faculty work sessions Long-term support in design of faculty work sessions
Principals are capable of Collecting Data for Profiles, Principals are capable of Collecting Data for Profiles, however, the process usually fails because…however, the process usually fails because… Teachers link principal’s observations with evaluationTeachers link principal’s observations with evaluation Teachers perceive the IPI process as an “administrator-Teachers perceive the IPI process as an “administrator-
driven” processdriven” process Teacher ownership is critical to consistent day-by-day Teacher ownership is critical to consistent day-by-day
changes in instructional design…and teacher ownership changes in instructional design…and teacher ownership does not occur if staff view the process as administrator-does not occur if staff view the process as administrator-drivendriven
Collaborative Faculty Collaborative Faculty Study of the IPI DataStudy of the IPI Data
Collaborative Faculty Study of the IPI Collaborative Faculty Study of the IPI Data…Core, Non-Core and Total Pie Data…Core, Non-Core and Total Pie
ChartsChartsCore ClassesCore Classes Non-Core ClassesNon-Core Classes
Collaborative Faculty Study of the IPI Collaborative Faculty Study of the IPI Data…Core, Non-Core and Total Pie Data…Core, Non-Core and Total Pie
ChartsChartsTotal for All ClassesTotal for All Classes
Keys to Studying the Data Keys to Studying the Data Use multiple short (45-50 min) study sessions
Study in a timely manner…compared to waiting weeks for a professional development day
Control the study environment… arrange setting with tables for small groups permit faculty to be seated with friends after beginning the session, number-off the teachers
to mix them randomly create small groups of 5-6 teachers per group
Use Whole Group and Small Group Strategies Think of the faculty meeting as an opportunity to
model good “category 5” engagement in learning
Common Elements for Each IPI Common Elements for Each IPI Faculty Collaborative Faculty Collaborative
ConversationConversation Discuss Typicality Of School day Of Instructional classroom practices (Jazz-it-up effect?)
Compare current and prior profiles…define issues to celebrate and issues of concern that need to be addressed
Build new knowledge about engagement strategies
Conclude session with discussion of value/worth of session…reflection/meta-cognition
Facilitate the collaborative study in whole group setting and small group settings with whole group share-out (make the learning a “5”)
Can You Interpret this Image of a Highly Can You Interpret this Image of a Highly Collaborative Faculty Study Session???Collaborative Faculty Study Session???
4>5 4>55
5 5
55 5
55
5
5
55
555
55
5 5
5 5 5<4 5<4
Higher-Order Faculty Collaborative Higher-Order Faculty Collaborative Learning Conversations?Learning Conversations?
Teacher
Teacher
Teacher
TeacherTeacher
Teacher Teacher
Teacher
Powerful learning yet under-utilized in faculty study
Collaborative Conversations We learn to study and problem-solve together as a faculty
Collective Collective Commitment Grows Commitment Grows We are getting on the same We are getting on the same page and learning from each page and learning from each otherother
Faculty CollectiveEfficacy GrowsWe believe we can and we believe we are “making a difference”
Student Academic Success is more Prevalent An expectation that students will succeed becomes the norm throughout all classrooms
Students Realize They Are More Successful Learners I can see my successes and I like it
Student Self-Efficacy Increases I believe I can learn this and I will learn it
Student Student AcademiAcademi
c c SuccessSuccess
ORGANIZATIONAL LEARNING
INSTRUCTIONAL BEST PRACTICES
PSYCHOLOGY OF LEARNING
Trust Grows
Collaborative Conversations Impact Student Learning (Valentine, 7-2010)
Instructional Activities that Engage Students more Effectively Increase Disengagement declines; Higher- Order/Deeper Engagement Increases
Thinking and Talking about Engagement Strategies become more Common Consciously design units and lessons with engagement in mind
Heartland Heartland SchoolSchool
Learning Communities can be like Silos…where
knowledge, like grain, is isolated, stored.
knowledge is not transferred to other silos. Collaboration/sharing/cohesion are missing.
RtI
CoopLrng
DiffInst
Etc.
PLC Study Group
IPI Longitudinal StudyIPI Longitudinal Study(1996-2008)(1996-2008)
Overview of 2009 StudyOverview of 2009 Study Data from 1996 to 2008 were compiled in 2008-2009
and analyzed using descriptive statistics, correlations and regressions, hierarchical linear modeling, and structural equation modeling. 243 schools from 105 school districts More than 550 days of data collection produced more than
70,000 classroom observations 125 of the 243 Schools completed a detailed “process” survey
allowing us to study “integrity to the recommended procedures”
Factors studied included Variables beyond the control of the school (e.g. FRL, %
minority) Variables within the control of the school (e.g. teacher
educational level, teacher experience, instructional practices) Dependent Variables
Student Achievement (Math and Communication Arts) Instructional Practices (higher-level and non-higher-level
engagement)
Higher-Order/Deeper Higher-Order/Deeper Engagement and AchievementEngagement and Achievement
A 20% increase in Higher-order/Deeper levels of engagement (categories 5-6) is projected to produce, on the state’s high-stakes achievement test, a: 6% increase in Communication Arts scores
i.e. a school with 50% of students passing the CA portion of the test would have 56% passing
7.3% increase in Mathematics scores i.e. a school with 50% of students passing the
Math portion of the test would have 57.3% passing
Disengagement and Disengagement and Seatwork with Achievement Seatwork with Achievement A 20% increase in categories 1-2-3 is
projected to produce, on the state’s high-stakes achievement test, a:7% decrease in Mathematics
i.e. a school with 50% of the students passing the Math portion of the state test would go down to 43% pass rate
Interesting Middle School DataInteresting Middle School Data
Reducing student and teacher disengagement in middle schools is twice as impactful on achievement as is increasing higher-order thinking. In middle schools, an increase in student
disengagement and teacher disengagement during seatwork (Categories 1-2) by 20 percent reduces the percent of students passing the state test by approximately 10%...
Changing HO/D (Categories 5-6) engagement enough to make a 10% upward swing in achievement pass rates would require an increase of approximately 41 percent
Interesting High School DataInteresting High School Data
For high schools, reducing student and teacher disengagement is even more impacting on achievement than in middle schools, while increasing HO/D is comparable on state achievement tests. In high schools, an increase in student/teacher
disengagement and seatwork (Categories 1-2) to 20 percent reduces the percent of student passing the state test by approximately 14%...
Changing HO/D (Categories 5-6) engagement enough to make a 14% upward swing in Communication Arts achievement pass rates would require an increase of approximately 45 percent
High Implementation Integrity, High Implementation Integrity, FRL & Achievement FRL & Achievement
Schools that implemented the IPI with integrity did not see the same degree of negative influence of FRL on state achievement scores compared to schools that implemented the IPI with low integrity. In short, achievement in schools with
high levels of IPI integrity are not as negatively impacted by poverty as would otherwise be the case.
Key Components of High Key Components of High Implementation of the IPI Implementation of the IPI
ProcessProcess Multiple data collections per year Collaborative faculty study of the
data following each data collection Data collection by teachers Faculty collaborative study of the
data led by teachers Level of faculty receptivity
(openness) to the IPI process during initial stages and subsequent stages
Faculty Receptivity to the IPI Process Faculty Receptivity to the IPI Process before Beginning Data Collectionsbefore Beginning Data Collections
Using a five-point scale to measure faculty receptivity to the IPI process: Low L-M Moderate M-H High 1 2 3 4 5
A school with the highest receptivity to the IPI data collection and collaborative conversations process before the first data were collected is projected to have 6% higher levels of HO/D engagement over schools with low-moderate receptivity
Faculty Receptivity to the IPI Process Faculty Receptivity to the IPI Process after the First Data after the First Data
Collection/ConversationCollection/Conversation Using a five-point scale to measure faculty
receptivity to the IPI process: Low L-M Moderate M-H High 1 2 3 4 5
A school with the highest receptivity to the IPI data collection and collaborative conversations process after their first data collection and collaborative study is projected to have 12% higher levels of HO/D engagement and teacher directed learning compared to schools with low-moderate receptivity
Keys to Early Faculty Keys to Early Faculty ReceptivityReceptivity
Develop a team of teacher leaders to collect data
Teacher leaders explain, and engage the faculty in discussing, the process before beginning the IPI process
Faculty are aware of the dates when IPI data will be collected
Teacher leaders maintain integrity during and after data collection
Teacher leaders facilitate the faculty study of the data
Principal facilitates and actively supports and encourages the process
Frequency of Data Collection and Frequency of Data Collection and Collaborative Study Make a Collaborative Study Make a
DifferenceDifference10 data collections
followed by collaborative conversations
over 3 years equated to…
14% increase in higher-order/deeper engagement (Categories 5-6)
AND 13% decrease in disengagement and
seatwork (Categories 1-2-3)
Keys to Data CollectionsKeys to Data Collections Collect data quarterly Build the 4 dates/times for faculty
study into school calendar Work backward from the dates when
data can be studied by the faculty as the basis for selecting the days data will be collected
Collect data approx. one week prior to time when faculty will study the data
The True Value of the IPI The True Value of the IPI Process Resides in the…Process Resides in the…
faculty collaborative conversations following each data profiling in which
teachers constructively study the data, problem solve, strategize, and learn together
as they collectively work to raise the bar of instruction across the whole school.
Let’s Sum it up…Let’s Sum it up…ENGAGEMENTENGAGEMENT
Document engagement using a valid and reliable observation process
Categories 3-4-5-6 are each important for Learning throughout a unit and/or lesson
Eliminate Disengagement (1) Reduce Teacher Disengagement (2) Reduce Seatwork (2-3) Increase HO/D Engagement (5-6) Increase HO/D Verbal Learning
Conversations (highly powerful process for learning) (5)
Let’s Sum it up…Let’s Sum it up…Collaborative Learning Collaborative Learning
ConversationsConversations Teacher leaders are the appropriate data collectors and should collect the engagement data in their own schools
Data collectors must be certificated to ensure validity and reliability.
Teacher leaders, with support from principals, should “facilitate” the study of the data
Multiple data collections and collaborations per year are necessary for meaningful, long-term change
Each faculty collaborative study session should: Discuss typicality during the data collection process Study data profiles and define issues to address Build new knowledge per defined issues Reflect on the value of the collaborative learning experience Develop commitment and learning by engaging staff
meaningfully in small/whole group learning conversations!
Recommended Annual Recommended Annual EventsEvents Based upon our most recent research about
successful use of the IPI, collect and study data three or four times a year.
The following is a recommended timeline: Early Fall—review process and categories; prep new
faculty; revisit “jazzing-it-up” Between school start and holiday break—collect
profiles twice and have collaborative conversations as soon as possible after each collection
Between holiday break and spring break, collect profile data and have collaborative conversation
Near the end of school year, collect profile data and have collaborative conversation.
Key IPI Reflective Questions After Year Key IPI Reflective Questions After Year One of ImplementationOne of Implementation
How many times have IPI data been collected? How many times have IPI data profiles been studied/discussed by the
faculty? Before the first IPI data collection, did the faculty discuss the IPI
process? If so, who led the discussion? How receptive to the use of the IPI was the faculty before the first data
collection? How receptive to the use of the IPI was the faculty after the first data
collection? How receptive to the use of the IPI is the faculty today? Did the faculty study/discuss the IPI profiles after each data collection? Who has been leading the study/discussion of the profiles? When the faculty discussed the data, how were the discussions
organized? Generally, who collects the data for the IPI profiles? Overall, who has provided the leadership for the faculty
study/discussions of the data? Overall, what have been the outcomes from the faculty’s
study/discussion of the data? Overall, what has been the faculty’s attitude/receptivity toward the IPI
process?
Longitudinal Effect…Longitudinal Effect…
.05
.35
.30
.25
.20
.15
.10
.45
.40
Sept
MayFebOctApril
Dec
Categories 2-3
Categories 5-6
Collaborative Higher- Collaborative Higher- Order Learning Order Learning Conversations Conversations
prepare students for prepare students for life! life!
If time…discuss the following about school
change.
ReferencesReferences Collins, J. (2009). Collins, J. (2009). Higher-order thinking in the high-stakes accountability era: Linking student Higher-order thinking in the high-stakes accountability era: Linking student
engagement and test performanceengagement and test performance. Unpublished doctoral dissertation, University of Missouri. . Unpublished doctoral dissertation, University of Missouri. Gauen, K. (2009). Gauen, K. (2009). The Impact of the Instructional Practices Inventory on an Illinois Middle The Impact of the Instructional Practices Inventory on an Illinois Middle
SchoolSchool. Unpublished doctoral dissertation, Lindenwood University.. Unpublished doctoral dissertation, Lindenwood University. Painter, B. (1998). Painter, B. (1998). The Impact of Student Engagement on Student Achievement and The Impact of Student Engagement on Student Achievement and
Perceptions of Student-Teacher Relationships.Perceptions of Student-Teacher Relationships. Unpublished doctoral dissertation, University of Unpublished doctoral dissertation, University of Missouri.Missouri.
Valentine, J., Clark, D., Hackmann, D., and Petzko, V. (2004Valentine, J., Clark, D., Hackmann, D., and Petzko, V. (2004 ). A National Study of Leadership in ). A National Study of Leadership in Middle Level Schools, Volume II: Leadership for Highly Successful Middle Level SchoolsMiddle Level Schools, Volume II: Leadership for Highly Successful Middle Level Schools . Reston, . Reston, VA: National Association of Secondary School Principals. VA: National Association of Secondary School Principals.
Valentine, J. (2005Valentine, J. (2005). Statistical Differences for the Percentages of Student Engagement as ). Statistical Differences for the Percentages of Student Engagement as Measured by IPI Categories between Very Successful and Very Unsuccessful Middle SchoolsMeasured by IPI Categories between Very Successful and Very Unsuccessful Middle Schools . . University of Missouri, Columbia, MO: Middle Level Leadership Center.University of Missouri, Columbia, MO: Middle Level Leadership Center.
Valentine, J. and Collins, J. (2009a). Valentine, J. and Collins, J. (2009a). Improving Instruction by Profiling Student Engaged Learning Improving Instruction by Profiling Student Engaged Learning and Creating Collaborative Teacher Learning Conversations.and Creating Collaborative Teacher Learning Conversations. National Association of Secondary National Association of Secondary School Principals, Annual Conference, March 1, 2009. School Principals, Annual Conference, March 1, 2009.
Valentine, J. and Collins, J. (2009b). Valentine, J. and Collins, J. (2009b). Analyzing the Relationships among the Instructional Analyzing the Relationships among the Instructional Practices Inventory, School Climate and Culture, and Organizational LeadershipPractices Inventory, School Climate and Culture, and Organizational Leadership . American . American Educational Research Association, Annual Meeting, April 14, 2009, San Diego, CA.Educational Research Association, Annual Meeting, April 14, 2009, San Diego, CA.
Valentine, J. (December, 2009). Valentine, J. (December, 2009). The Instructional Practices Inventory: Using a Student Learning The Instructional Practices Inventory: Using a Student Learning Assessment to Foster Organizational LearningAssessment to Foster Organizational Learning. National Staff Development Council, Annual . National Staff Development Council, Annual Convention, December 8, 2009, St. Louis, MO.Convention, December 8, 2009, St. Louis, MO.
Valentine, J. (2010). ) Valentine, J. (2010). ) Establishing a Faculty-wide Collaborative Study of Student EngagementEstablishing a Faculty-wide Collaborative Study of Student Engagement , , National Association of Secondary School Principals, Annual Conference, San Diego, CA March National Association of Secondary School Principals, Annual Conference, San Diego, CA March 14, 2010.14, 2010.
Yair, G. (2000). Not just about time: Instructional practices and productive time in school. Yair, G. (2000). Not just about time: Instructional practices and productive time in school. Educational Administration Quarterly, 36(4), 485-512. Educational Administration Quarterly, 36(4), 485-512.
Suggested Resources…Suggested Resources… For more detailed information about the IPI see For more detailed information about the IPI see
Instructional Practices Inventory: Profiling Student Instructional Practices Inventory: Profiling Student Engagement for School ImprovementEngagement for School Improvement (Valentine, 2005) (Valentine, 2005) available from available from [email protected]@missouri.edu..
For more detailed information about Project ASSIST see For more detailed information about Project ASSIST see Frameworks for Continuous School Improvement: A Frameworks for Continuous School Improvement: A Synthesis of Essential ConceptsSynthesis of Essential Concepts (Valentine, 2001) and (Valentine, 2001) and Project ASSIT Research paper presented at available at AERA, Project ASSIT Research paper presented at available at AERA, April 2006 in San FranciscoApril 2006 in San Francisco) available from ) available from [email protected]@missouri.edu..
Also see The Instructional Practices Inventory: Using a Also see The Instructional Practices Inventory: Using a Student Learning Assessment to Foster Organizational Student Learning Assessment to Foster Organizational Learning Learning Valentine, 2007, also 2009 NSDC Annual Convention Valentine, 2007, also 2009 NSDC Annual Convention paper) available by request from paper) available by request from [email protected]@missouri.edu.
For information about IPI Level I (Coder Training) and Level II For information about IPI Level I (Coder Training) and Level II (Advanced Faculty Work Session) workshops(Advanced Faculty Work Session) workshops email email [email protected]@missouri.edu..
IPI Coder Reliability is IPI Coder Reliability is Developed through IPI Level I Developed through IPI Level I
WorkshopWorkshop
Minimum Reliability for user endorsement:Minimum Reliability for user endorsement: .80 for site-based school improvement data.80 for site-based school improvement data .90 for research.90 for research
Coder Reliability StudyCoder Reliability Study w/o Workshop .05-.20 Reliability avg: .17w/o Workshop .05-.20 Reliability avg: .17 With Workshop .80-1.0 Reliability avg: .93With Workshop .80-1.0 Reliability avg: .93
If you would like a copy of If you would like a copy of this PowerPoint this PowerPoint presentation…presentation…
Email me at Email me at [email protected]@Missouri.edu Request the PPT by date and location Request the PPT by date and location
of the presentationof the presentation I will reply and attach a copy of the I will reply and attach a copy of the
PPT plus the handouts and a couple of PPT plus the handouts and a couple of manuscripts about the IPI.manuscripts about the IPI.
For information about IPI Workshops For information about IPI Workshops in your area, contact me by email.in your area, contact me by email.
Website: www.MLLC.org
Email: [email protected]
Phone: (573) 882-0944
Questions, contact the IPI developers at:
Additional Slides of Additional Slides of Interest…Lack of time Interest…Lack of time prevented these slides prevented these slides from being used in the from being used in the
presentationpresentation The following slides may have some The following slides may have some
value for understanding the IPI value for understanding the IPI process. They have been used in process. They have been used in different presentations over time different presentations over time about the IPI and/or student about the IPI and/or student engagement in learning.engagement in learning.
Recommended ResourcesRecommended Resources www.APA.org “Learner-Centered www.APA.org “Learner-Centered
Psychological Principles (1997)Psychological Principles (1997) ““Powerful Learning (ASCD and Powerful Learning (ASCD and
www.ascd.org; (Brandt 1998)www.ascd.org; (Brandt 1998) Marzano, et al. (ASCD):Marzano, et al. (ASCD):
The Art and Science of Teaching (2007)The Art and Science of Teaching (2007) Classroom Instruction that Works (2001)Classroom Instruction that Works (2001) Designing and Assessing Educational Designing and Assessing Educational
Objectives (2008)Objectives (2008)
What does it take to change What does it take to change engagement?engagement?
Sequence of Payoff
What it Takes
6 (3) Combined Categories 5-6:
All HO/D
New KnowledgeImplementation
SkillCommitment
5
43
2(2) Categories
One Plus Two: All Disengagement
AwarenessCommitment
1(1) Most Direct:
Stu. Disengagement
AwarenessCommitment
Overall Implementation of the IPI Process Overall Implementation of the IPI Process Recommended Practices for Data Recommended Practices for Data
Collection and Collaborative Collection and Collaborative Conversations Conversations
Using a five-point scale to measure school Using a five-point scale to measure school implementation of the recommended practices:implementation of the recommended practices: Low L-M Moderate M-H HighLow L-M Moderate M-H High 1 2 3 4 51 2 3 4 5
A school implementing the IPI A school implementing the IPI recommended practices with high integrity recommended practices with high integrity is projected to have 8.4% higher levels of is projected to have 8.4% higher levels of HO/D engagement over schools with low-HO/D engagement over schools with low-moderate implementationmoderate implementation
Higher Order/Deeper EngagementHigher Order/Deeper Engagementvs. vs.
Not Higher Order/Surface Not Higher Order/Surface EngagementEngagement
Analysis, Critical Thinking, Problem Analysis, Critical Thinking, Problem Solving, Decision Making or Solving, Decision Making or Application from Analysis, Application from Analysis, Creativity, Innovation SynthesisCreativity, Innovation Synthesis
------------------------------------------------------------------------------------------
Recall, Memorization, Fact Finding, Recall, Memorization, Fact Finding, Simple Understanding, Practice to Simple Understanding, Practice to Internalize Skills or ProcessesInternalize Skills or Processes
6 – Student Active Engaged 6 – Student Active Engaged Learning (HO/D)Learning (HO/D)
Higher-Order/Deeper thinking through Higher-Order/Deeper thinking through analysis, problem solving, critical analysis, problem solving, critical thinking, creativity, innovation, and thinking, creativity, innovation, and synthesis.synthesis.
Common Examples (if HO/D): Common Examples (if HO/D): Inquiry-based approaches such as project Inquiry-based approaches such as project
and problem-based learning, research and and problem-based learning, research and discovery learningdiscovery learning
Authentic demonstrationsAuthentic demonstrations Independent metacognition, reflective Independent metacognition, reflective
journaling, and self-assessmentjournaling, and self-assessment Higher-order responses to higher-order Higher-order responses to higher-order
questions.questions.
5—Student Learning 5—Student Learning Conversations (HO/D)Conversations (HO/D)
Higher-Order Student-Student Verbal Higher-Order Student-Student Verbal Learning Conversations constructing Learning Conversations constructing deeper meaning and understanding deeper meaning and understanding through the conversationsthrough the conversations
Common examples (if HO/D):Common examples (if HO/D): collaborative or cooperative learningcollaborative or cooperative learning Peer tutoring, debate, and questioningPeer tutoring, debate, and questioning Partner research and discovery/exploratory Partner research and discovery/exploratory
learninglearning Socratic learningSocratic learning Small group or whole class analysis and Small group or whole class analysis and
problem solving, metacognition, reflective problem solving, metacognition, reflective discussions or writing, and self assessmentdiscussions or writing, and self assessment
4—Teacher-Led Instruction 4—Teacher-Led Instruction (Not HO/D)(Not HO/D)
Students are attentive to teacher-led Students are attentive to teacher-led instruction as the teacher leads the instruction as the teacher leads the learning experience by disseminating learning experience by disseminating content knowledge and/or directions for content knowledge and/or directions for learninglearning
Common Examples:Common Examples: Teacher-directed Q/A, lecture, explanationsTeacher-directed Q/A, lecture, explanations Teacher direction givingTeacher direction giving Teacher demonstrationsTeacher demonstrations
3—Student Work with 3—Student Work with Teacher Engaged (Not Teacher Engaged (Not
HO/D)HO/D) Students engaged in independent or Students engaged in independent or
group work designed to build basic group work designed to build basic understanding, new knowledge, pertinent understanding, new knowledge, pertinent skills. Teacher is attentive to, engaged skills. Teacher is attentive to, engaged with, or supportive of the students’ work.with, or supportive of the students’ work.
Common Examples: (Teacher Common Examples: (Teacher Engaged)Engaged)
Fact findingFact finding Building skill or understanding through Building skill or understanding through
practice, seatwork, worksheets, chapter practice, seatwork, worksheets, chapter review questionsreview questions
Multi-mediaMulti-media
2—Student Work with 2—Student Work with Teacher Not Engaged (Not Teacher Not Engaged (Not
HO/D)HO/D) Same as Category Three except the Same as Category Three except the
teacher is not attentive to, engaged with, teacher is not attentive to, engaged with, or supportive of the students’ work.or supportive of the students’ work.
Common Examples: (Teacher Not Common Examples: (Teacher Not Engaged)Engaged)
While students are working, teacher is:While students are working, teacher is:Out of the roomOut of the roomWorking at computerWorking at computerGrading papersGrading papers
1—Students Not Engaged 1—Students Not Engaged in Learningin Learning
Students are not engaged in learning Students are not engaged in learning directly related to the curriculumdirectly related to the curriculum
Common Examples:Common Examples: Students talking, daydreaming, or otherwise Students talking, daydreaming, or otherwise
inattentiveinattentive Students misbehavingStudents misbehaving Students not doing their assigned workStudents not doing their assigned work
Activities that are typicallyActivities that are typicallyHigher-Level Learning Higher-Level Learning
Common Examples:Common Examples: Project-based learningProject-based learning Research/Hypothesizing/Testing/Concluding/Research/Hypothesizing/Testing/Concluding/
DefendingDefending Questioning…Why, What if, Compare/ContrastQuestioning…Why, What if, Compare/Contrast Socratic SeminarsSocratic Seminars Thinking about thinking...metacognitionThinking about thinking...metacognition Cooperative LearningCooperative Learning Problem-based learningProblem-based learning Reflective Discussions and Writing TasksReflective Discussions and Writing Tasks
Less common examples…how do you make Less common examples…how do you make these higher-order?these higher-order? Watching and analyzing a movieWatching and analyzing a movie Paper/pencil testsPaper/pencil tests Dodge ball! Dodge ball!
Why are Higher-Order Learning Why are Higher-Order Learning ConversationsConversations so Significant? so Significant?
Learning is enhanced through social Learning is enhanced through social interaction. When students engage in H-interaction. When students engage in H-O learning conversations the benefits O learning conversations the benefits are: are: Motivation to learn (social nature)Motivation to learn (social nature) Depth of knowledge and understandingDepth of knowledge and understanding Breadth of knowledge and understandingBreadth of knowledge and understanding Recall of knowledge and conceptsRecall of knowledge and concepts Transfer of knowledge and conceptsTransfer of knowledge and concepts
Recommended Annual Recommended Annual EventsEvents Based upon our most recent research about Based upon our most recent research about
successful use of the IPI, collect and study data successful use of the IPI, collect and study data three or four times a year.three or four times a year.
The following is a recommended timeline:The following is a recommended timeline: Early Fall—review process and categories; prep new Early Fall—review process and categories; prep new
faculty; revisit “jazzing-it-up” effectfaculty; revisit “jazzing-it-up” effect Between school start and holiday break—collect Between school start and holiday break—collect
profiles twice and have collaborative conversations profiles twice and have collaborative conversations as soon as possible after each collectionas soon as possible after each collection
Between holiday break and spring break, collect Between holiday break and spring break, collect profile data and have collaborative conversationprofile data and have collaborative conversation
Near the end of school year, collect profile data and Near the end of school year, collect profile data and have collaborative conversation.have collaborative conversation.
Recommended Year One Recommended Year One Timeline/TasksTimeline/Tasks
Aug/SeptAug/Sept Sept/OctSept/Oct Nov/DecNov/Dec Jan/FebJan/Feb Apr/MayApr/May
Orient new Orient new faculty; faculty; remind oldremind old
Data Data collection; collection; collaborative collaborative conversationconversation
Data Data collection; collection; collaborative collaborative conversationconversation
Data Data collection; collection; collaborative collaborative conversationconversation
Data Data collection; collection; collaborative collaborative conversationconversation
Discuss:Discuss:CategoriesCategoriesProcessProcessJazz-it-upJazz-it-upNot Not EvaluationEvaluation
Discuss:Discuss:Typicality Typicality CelebrationCelebrationIssuesIssuesValueValue
Discuss:Discuss:Typicality Typicality CompareCompareCategories Categories 5-6 5-6 ExamplesExamplesHomeworkHomeworkValueValue
Discuss:Discuss:TypicalityTypicalityCompare Compare longitudinallylongitudinallyCategories Categories 5-6 5-6 ExamplesExamplesValueValue
Discuss:Discuss:TypicalityTypicalityCompare Compare longitudinallylongitudinallySet goals for Set goals for next yearnext yearValueValue
Commonalities to Each Faculty Commonalities to Each Faculty Collaborative IPI ConversationCollaborative IPI Conversation
TypicalityTypicality School daySchool day Instructional classroom practicesInstructional classroom practices
Comparison of profilesComparison of profiles Substantive learning experiencesSubstantive learning experiences Value or worth…reflection, meta-cognitionValue or worth…reflection, meta-cognition Facilitation of issues from whole group to Facilitation of issues from whole group to
small group to whole group share-outsmall group to whole group share-out
Higher-Order Student Classroom Higher-Order Student Classroom Learning Conversations?Learning Conversations?
Teacher
Student
Student
StudentStudent
Student Student
Student
Powerful learning yet under-utilized
Faculty Work Session: Analysis and Faculty Work Session: Analysis and Discussion of the Profile DataDiscussion of the Profile Data
Small and Whole Group Analyses and Small and Whole Group Analyses and DiscussionDiscussion What do we see in the profiles that we can What do we see in the profiles that we can
feel good about or celebrate?feel good about or celebrate? What do we see in the data profiles that we What do we see in the data profiles that we
should be concerned about and thus study should be concerned about and thus study and discuss more deeply?and discuss more deeply?
How do we build a cache of good ideas on How do we build a cache of good ideas on engagement, especially good HO/D engagement, especially good HO/D engagement?engagement?
Faculty Discussion: Are these types of Faculty Discussion: Are these types of data valuable to us?data valuable to us?
Faculty Work Session: Post-Faculty Work Session: Post-session Requestssession Requests
Request for sub-group analyses…Request for sub-group analyses… Can we have a profile for the math Can we have a profile for the math
program? program? Individual teacher self-Individual teacher self-
assessment…assessment… Can I build a profile of my students’ Can I build a profile of my students’
engagement using this process?engagement using this process? Value/benefit of self-ratings vs. Value/benefit of self-ratings vs.
accuracy/reliability of self-ratings?accuracy/reliability of self-ratings?
Faculty Work Session: Deeper Faculty Work Session: Deeper Analyses with Longitudinal Analyses with Longitudinal
Perspective and Goal SettingPerspective and Goal Setting How do we begin to share knowledge How do we begin to share knowledge
about effective strategies that will about effective strategies that will change the percentages?change the percentages? Small groups collaboratively brainstorm Small groups collaboratively brainstorm
good examples of categories 5-6 used in good examples of categories 5-6 used in past week (create examples from within…)past week (create examples from within…)
Move conversation to whole faculty sharingMove conversation to whole faculty sharing Move conversations after faculty meeting to Move conversations after faculty meeting to
sub-groups such as content areas, teams, or sub-groups such as content areas, teams, or grade levelsgrade levels
Type and share all examples with all facultyType and share all examples with all faculty
Faculty Work Session: Goal Faculty Work Session: Goal Setting after three or four Setting after three or four
data collections…data collections… For each category percentage, should For each category percentage, should
we:we: Increase?Increase? Maintain?Maintain? Decrease?Decrease?
If change is appropriate…If change is appropriate… How much?How much? By when?By when?
What do we address first that will have What do we address first that will have the most direct impact on student the most direct impact on student learning?learning?
““Typical” Profiles…Typical” Profiles…notnot normsnorms
Are there differences between Are there differences between typical profiles by grade levels, typical profiles by grade levels, (elem., middle, and high school)?(elem., middle, and high school)?
Are there differences between Are there differences between typical profiles for core and non-typical profiles for core and non-core classes? core classes?
Are there differences between Are there differences between profiles for more effective and less profiles for more effective and less effective schools?effective schools?
IPI Protocol Examples for Data IPI Protocol Examples for Data CollectionCollection
Data observations on “typical” daysData observations on “typical” days Systematically move from classroom to Systematically move from classroom to
classroom based upon the floor plan of the classroom based upon the floor plan of the schoolschool
Focus on students, not the teacherFocus on students, not the teacher Code student learning during first moments of Code student learning during first moments of
observationobservation When observation is borderline between two When observation is borderline between two
codes, select higher codecodes, select higher code Code during regular learning time, not Code during regular learning time, not
transitions between content areastransitions between content areas Classrooms of special education and student Classrooms of special education and student
teachers are observed and codedteachers are observed and coded Classrooms of substitute teachers are coded Classrooms of substitute teachers are coded
and included in profile if higher-order learningand included in profile if higher-order learning All observations are anonymousAll observations are anonymous
Active-Passive EngagementActive-Passive Engagement
What are some examples of What are some examples of ACTIVEACTIVE STUDENT ENGAGEMENT in learning?STUDENT ENGAGEMENT in learning?
What are some examples of What are some examples of PASSISVEPASSISVE STUDENT ENGAGEMENT in learning?STUDENT ENGAGEMENT in learning?
Being actively engaged does not Being actively engaged does not necessarily mean good learning!necessarily mean good learning!
IPI ProcessIPI Process
Fits Concept of Learning Fits Concept of Learning OrganizationsOrganizations Faculty Discussions/Analysis of Data Faculty Discussions/Analysis of Data
about Teaching/Learningabout Teaching/Learning ““Teachers engaged in the data collection”Teachers engaged in the data collection” ““Teachers engage in regular, reflective collaborative Teachers engage in regular, reflective collaborative
conversations about the data profiles.”conversations about the data profiles.” ““On-going collection and collaborative problem-On-going collection and collaborative problem-
solving conversations over time.”solving conversations over time.”
should…should…
Be associated with Be associated with staff evaluationstaff evaluation
Identify individual Identify individual teachers or classesteachers or classes
Be collected by Be collected by administratorsadministrators
Reflect the activity in Reflect the activity in which the students are which the students are engagedengaged
Create an optimum profile of Create an optimum profile of engagement teachers will view as fair engagement teachers will view as fair and accurateand accurate
Provides the basis for reflective, Provides the basis for reflective, collaborative conversationscollaborative conversations
Provide baseline data and insight Provide baseline data and insight about subsequent changes in about subsequent changes in engagementengagement
Support school improvement and Support school improvement and professional development plansprofessional development plans
Serve as a basis for action researchServe as a basis for action research
Be used in context with multiple Be used in context with multiple measures of student success measures of student success
Reflect how students are engaging in Reflect how students are engaging in learninglearning
should not…An effective student engagement profiling process:An effective student engagement profiling process:
The following four slides are The following four slides are data from the Justin Collins data from the Justin Collins
study, 2009study, 2009
Number of Collections and Number of Collections and Collaborative Conversations with Collaborative Conversations with
Engagement (Q5)Engagement (Q5) If a typical school in this study If a typical school in this study
engaged in 10 data collections and engaged in 10 data collections and collaborative conversations over the collaborative conversations over the course of three years, the school course of three years, the school would have a 13 point decrease in would have a 13 point decrease in categories 1-2-3 from current categories 1-2-3 from current average of 38% and a 14 point average of 38% and a 14 point increase in higher level thinking as increase in higher level thinking as measured by categories 5-6 from measured by categories 5-6 from current average of 22%.current average of 22%.
Engagement and Student Engagement and Student
AchievementAchievement If a school in the study had a 20% increase in If a school in the study had a 20% increase in
categories 1-2-3 it would have had a decrease categories 1-2-3 it would have had a decrease in achievement of 7% on Math performancein achievement of 7% on Math performance In our study that meant a school at the 42% In our study that meant a school at the 42%
pass rate would go down to 35% pass ratepass rate would go down to 35% pass rate If a school in the study had a 20% increase in If a school in the study had a 20% increase in
categories 5-6 it would have had a 7.3% categories 5-6 it would have had a 7.3% increase on Mathematics performanceincrease on Mathematics performance In our study that meant a school at the 42% In our study that meant a school at the 42%
pass rate would go to 49.3% pass rate pass rate would go to 49.3% pass rate N=135
3-level analysis
Faculty Receptivity andFaculty Receptivity andEngagement (Q6b)Engagement (Q6b)
If a typical school in this study If a typical school in this study introduced the use of the IPI in a introduced the use of the IPI in a manner that resulted in a level of manner that resulted in a level of receptivity (openness on a 5-unit receptivity (openness on a 5-unit scale) to the IPI process that was scale) to the IPI process that was high (5) compared to low-moderate high (5) compared to low-moderate (2), the school would see an increase (2), the school would see an increase of 6 points of categories 5-6. of 6 points of categories 5-6.
Faculty Receptivity after Initial Faculty Receptivity after Initial Analysis and Student Engagement Analysis and Student Engagement
(Q11)(Q11) If a faculty’s level of receptivity If a faculty’s level of receptivity
on a 5-unit scale to the IPI on a 5-unit scale to the IPI process after the faculty’s first process after the faculty’s first profile analysis was high (5) profile analysis was high (5) compared to low-moderate (2) compared to low-moderate (2) the school would see a 12 point the school would see a 12 point gain in the engagement for gain in the engagement for categories 4-5-6.categories 4-5-6.
Overall Benefits of IPI Process with Overall Benefits of IPI Process with Engagement (Q14)Engagement (Q14)
If a typical school in this study engaged in If a typical school in this study engaged in the use of the IPI to the degree that the the use of the IPI to the degree that the overall benefits of collaborative overall benefits of collaborative conversations and deeper understanding conversations and deeper understanding about instructional design increased 3 units about instructional design increased 3 units on a 5-unit scale assessing overall impact of on a 5-unit scale assessing overall impact of the IPI on the school culture and instruction, the IPI on the school culture and instruction, the resultant increase in Core Higher-level the resultant increase in Core Higher-level engagement would go up 8.4 points from 23 engagement would go up 8.4 points from 23 to 31.4%.to 31.4%.
Note that the following data Note that the following data are Pre-2005 Data…review are Pre-2005 Data…review
and use with cautionand use with caution
Typical Percentages by Grade Typical Percentages by Grade LevelsLevels
E. S.E. S. M. S.M. S. H. S.H. S.
Student Active Engaged Student Active Engaged LearningLearning 15-2515-25 15-2015-20 15-2015-20
Student Learning Student Learning ConversationsConversations 3-53-5 3-53-5 3-53-5
Teacher-Led InstructionTeacher-Led Instruction 35-4035-40 35-4535-45 30-4030-40
Student Work with Student Work with Teacher EngagedTeacher Engaged 20-3020-30 20-3020-30 15-2015-20
Student Work with Student Work with Teacher not EngagedTeacher not Engaged 5-105-10 10-2010-20 15-2015-20
Complete Complete DisengagementDisengagement 3-83-8 5-105-10 5-155-15
Jerry Valentine 2004
Typical Percentages by Core/Non-Typical Percentages by Core/Non-CoreCore
All LevelsAll Levels
All All ContentContent
All All LevelsLevels
CoreCore
All LevelsAll Levels
Non-CoreNon-Core
Student Active Engaged Student Active Engaged LearningLearning 15-2015-20 <15<15 <25<25
Student Learning Student Learning ConversationsConversations 3-53-5 5-105-10 <5<5
Teacher-Led InstructionTeacher-Led Instruction 30-4530-45 >40>40 <40<40
Student Work with Student Work with Teacher EngagedTeacher Engaged 20-3020-30 >25>25 <25<25
Student Work with Student Work with Teacher not EngagedTeacher not Engaged 10-2010-20 >20>20 <20<20
Complete Complete DisengagementDisengagement 5-105-10 >5>5 <5<5
Jerry Valentine 2004
Typical Percentages by School Typical Percentages by School EffectivenessEffectiveness
All LevelsAll Levels
All All ContentContent
MoreMore
EffectiveEffectiveLessLess
EffectiveEffective
Student Active Engaged Student Active Engaged LearningLearning 15-2015-20 >25>25 15-2015-20
Student Learning Student Learning ConversationsConversations 3-53-5 5-105-10 <5<5
Teacher-Led InstructionTeacher-Led Instruction 30-4530-45 35-4535-45 30-4030-40
Student Work with Student Work with Teacher EngagedTeacher Engaged 20-3020-30 15-2515-25 >25>25
Student Work with Student Work with Teacher not EngagedTeacher not Engaged 10-2010-20 5-105-10 10-2010-20
Complete Complete DisengagementDisengagement 5-105-10 <3<3 >5>5
Jerry Valentine 2004
Percentages for High Achieving and Low Achieving Middle SchoolsPercentages for High Achieving and Low Achieving Middle Schools
HighlyHighly
AchievingAchievingLowLow
AchievingAchieving
***.001***.001
**.05**.05
*.10*.10
Student Active Engaged Student Active Engaged LearningLearning 29.329.3 16.016.0 **Student Learning Student Learning ConversationsConversations 3.33.3 0.20.2 ******Teacher-Led InstructionTeacher-Led Instruction 40.540.5 33.233.2
Student Work with Student Work with Teacher EngagedTeacher Engaged 17.317.3 28.428.4 ******Student Work with Student Work with Teacher not EngagedTeacher not Engaged 8.58.5 13.613.6
Complete Complete DisengagementDisengagement 1.01.0 8.48.4 ******Jerry Valentine 2004
Percentages for High Achieving and Low Achieving Middle SchoolsPercentages for High Achieving and Low Achieving Middle Schools
HighlyHighly
AchievingAchievingLowLow
AchievingAchieving
***.001***.001
**.05**.05
*.10 *.10
Student Active Engaged Student Active Engaged LearningLearning
32.632.6 16.216.2 ****Student Learning Student Learning ConversationsConversations
Teacher-Led InstructionTeacher-Led Instruction57.857.8 61.661.6 **Student Work with Student Work with
Teacher EngagedTeacher Engaged
Student Work with Student Work with Teacher not EngagedTeacher not Engaged
9.59.5 22.022.0 ****Complete Complete DisengagementDisengagement
Jerry Valentine 2004
Percentages for High Achieving and Low Achieving Middle SchoolsPercentages for High Achieving and Low Achieving Middle Schools
HighlyHighly
AchievingAchievingLowLow
AchievingAchieving
***.001***.001
**.05**.05
*.10*.10
Student Active Engaged Student Active Engaged LearningLearning
73.173.1 49.449.4 ******Student Learning Student Learning ConversationsConversations
Teacher-Led InstructionTeacher-Led Instruction
Student Work with Student Work with Teacher EngagedTeacher Engaged
26.826.8 50.450.4 ******Student Work with Student Work with Teacher not EngagedTeacher not Engaged
Complete Complete DisengagementDisengagement
Jerry Valentine 2004
Activities that Activities that TypicallyTypically produce produce Higher-Order/Deeper Engagement Higher-Order/Deeper Engagement
Common Examples:Common Examples: Project-based learningProject-based learning Research/Hypothesizing/Testing/Concluding/Research/Hypothesizing/Testing/Concluding/
DefendingDefending Questioning…Why, What if, Compare/ContrastQuestioning…Why, What if, Compare/Contrast Socratic SeminarsSocratic Seminars Thinking about thinking...metacognitionThinking about thinking...metacognition Cooperative LearningCooperative Learning Problem-based learningProblem-based learning Reflective Discussions and Writing TasksReflective Discussions and Writing Tasks
Less common examples…how do you make Less common examples…how do you make these higher-order?these higher-order? Watching and analyzing a movieWatching and analyzing a movie Paper/pencil testsPaper/pencil tests Dodge ball! Dodge ball!
Organizational Learning Organizational Learning Reduces the Knowledge--Reduces the Knowledge--Implementation GapImplementation Gap
Society and Students We Serve Change Continuously
Expert Knowledge of Best Practices
Our Knowledge of Best Practices
Our Implementation if We Maintain Knowledge
Our Implementation w/ Moderate Knowledge
Our Implementation with No New Knowledge No Knowledge, Low Attitude, Low Effort