Tripod Student Surveys Ronald F. Ferguson, PhD Co-Founder, Tripod Education Partners, Inc., and Faculty Director, the Achievement Gap Initiative at Harvard

Tripod Student Surveys Ronald F. Ferguson, PhD Co-Founder, Tripod Education Partners, Inc., and Faculty Director, the Achievement Gap Initiative at Harvard University September 9, 2014 Discussion with Massachusetts Association of School Business Officials

Ron Ferguson Founder of the Tripod Project and its student and teacher surveys Leads all research and analysis of Tripod survey data Eighteen years building bridges between researchers and professional K-12 educators Over thirty years teaching and research at Harvard University Director of the Achievement Gap Initiative, Harvard University #2

Why might student surveys be relevant from a school finance perspective? Turnover. Productivity. Survey patterns can: 1. supplement test scores as a way of tracking instructional quality improvementsthey can help inform judgments on whether expenditures on professional development are producing value. 2. help reduce turnover by more accurately identifying teachers who are struggling and might leave without targeted support. 3. help cut turnover through reducing frustration among struggling teachers without a diagnosis of what they could do better. 4. help identify teachers whose ideas and practices could be harvested for sharing, thereby increasing payoffs to professional development expenditures 5. help instructional coaches and administrators more efficiently differentiate professional supports.

Four Key Messages Students are good observers There is a great deal of measurable variation in teaching quality Multiple measures, multiple times, over multiple years While our surveys are also appropriate for evaluation, our primary purpose has always been professional growth #4

Why the focus on student surveys? Source: www.metproject.org In the Gates Measures of Effective Teaching study a single administration of Tripod surveys was found to be a reliable measure and predictive of student achievement gains. #8

Nine Principles from the Gates Measures of Effective Teaching Source: www.metproject.org

Vision Universal reliance on student feedback as a respected basis for professional reflection on ways to provide enriched learning experiences in the near term, to enable high quality life outcomes over the long term. Role in evaluation systems? #12

Tripod

Content What should we teach? Does each teacher have deep knowledge of the curriculum? Pedagogy How should we teach? Does each teacher use effective instructional techniques? Relationships Are we a community? Do students and teachers care about, inspire, and motivate each other? Improved student performance depends on strengthening three legs of our teaching practice: Why the name Tripod? #14

Dozens of Districts and Several States are Using Tripod 1/17/2014

Some Background Typically administered as an organized school and/or district activity Multiple versions: K-2, 3-5, 6-12, measuring - Student Engagement- Teaching Effectiveness - School Climate- Youth Achievement Culture Online or paper-based Approximately 20 minutes to complete, on average Response rates of over 95% are common #16

Capturing Valuable Insights from Students The Framework Matters #17 Grounded in decades of teaching quality research Easy to understand and remember Excellent coverage of key education issues Ideas are integrated coherently as a system of concepts Easy to align and cross-walk with prominent observational frameworks (such as those by Danielson, Marshall, Pianta, and Marzano.)

Survey Items Organized in a Framework Refined over a Decade: Informed by Research and Input from Teachers #18

Components of the Tripod 7Cs Framework 1.Care: Show concern and commitment. 2.Confer: Invite ideas and promote discussion. 3.Captivate: Inspire curiosity and interest. 4.Clarify: Cultivate understanding and overcome confusion. 5.Consolidate: Integrate ideas and make learning coherent. 6.Challenge: Press for rigor and persistence. 7.Control: Sustain order, respect and focus. SUPPORTSUPPORT PRESSPRESS #19

Sample Survey Items Clarify If you dont understand something, my teacher explains it another way. My teacher explains difficult things clearly. Challenge Control Care Confer Captivate Clarify Consolidate #20

Sample Survey Items Control Students in this class treat the teacher with respect. My classmates behave the way the teacher wants them to. Our class stays busy and doesnt waste time. Challenge Control Care Confer Captivate Clarify Consolidate #21

Challenge My teacher asks students to explain more about the answers they give. My teacher doesnt let people give up when the work gets hard. In this class, we learn to correct our mistakes. My teacher wants us to use our thinking skills, not just memorize things. Sample Survey Items Challenge Control Care Confer Captivate Clarify Consolidate #22

Percentage Desirable Responses

Percent agreement (i.e., Mostly or Totally True), for: "In this class, we learn a lot almost every day." (One rural district: each bar is a class.)

Early Elementary Items, e.g.: My teacher is nice to me when I ask questions. (Care) To help us remember, my teacher talks about things that we already learned. (Consolidate) My teacher makes sure that I try to do my best. (Challenge) #25

Kindergarten First Second

Chart for illustrative purposes: NOT real data #27

Agreement with selected statements by students in 25 th and 75 th percentile classrooms from the MET secondary school sample. ( From among 2985 classrooms, each with at least 5 students responding) 25 th Percentile 75 th Percentile 1. CARE: My teacher in this class makes me feel that s/he really cares about me 4073 2. CONTROL: Our class stays busy and doesnt waste time. 3669 3. CLARIFY: My teacher explains difficult things clearly. 5079 4. CHALLENGE: My teacher wants me to explain my answers why I think what I think. 5983 5. CAPTIVATE: My teacher makes learning enjoyable. 3372 6. CONFER: My teacher wants us to share our thoughts. 4779 7. CONSOLIDATE: My teacher takes the time to summarize what we learn each day. 3867

29 Predicted differences in months of learning for classrooms at the 25th versus 75th percentiles of the composite 7Cs distribution for the MET sample. (Estimated using surveys from one class and gains from another class, taught by the same teacher.) Using student survey responses in one section to predict learning gains in another. Predicted difference per school year* On state math tests 4.8 months On the Balanced Assessment in Math 3.7 months On state English Language Arts (ELA) tests 2.3 months On the Stanford 9 Open Ended ELA 2.9 months *Based on Table 9, p. 26: Bill and Melinda Gates Foundation, Learning about Teaching: Initial findings from the Measures of Effective Teaching Project. December 2010.

Teaching Practices: What matters most? #30 Care Control Clarify Challenge Captivate Confer Consolidate

Simple 7Cs correlations with previous-year value-added ratings for 233 teachers of math, science, and English language arts in one Tripod district. (Not MET)

Some Things Tripods 7Cs Predict (MET data for 4 th through 8 th Graders; Multivariate Estimation.) SupportPress Value-Added Test Score Gains -- Inspiration to Attend College -- Happiness in Class -- Effort in Class #32

Tripod composite 7Cs favorability percentages for one district, by year and season, for each level of schooling, including only teachers with at least one class per deployment. (N>800 teachers for each level.)

Improvement

A Personalize Report to Each Teacher Surveys are organized to help focus goal-setting discussions and decisions about professional development. Progress can be tracked over time.

Four Possible Scenarios Surveys are scored and results are used for performance evaluations Results are used as a check on the accuracy of the other ratings Teacher receives results, reflects, sets some goals, and is evaluated on their approach and follow through on this process. Some (which?) or all of the above #36

School One (N=356 students) ControlChallengeCareConferCaptivateConsolidateClarifyRow Mean A11111111.0 B1111111 C11121111.1 D11212111.3 E22111211.4 F12212221.7 G22212221.9 H22312222.0 I42123232.4 J42333222.7 K24444443.7 L15534554.0 M55444444.3 N55555444.7 School Two (N=354 students) A44111212.0 B45111322.4 C4144211 D52313322.7 E52433223.0 F33233443.1 G5332243 H55142333.3 I54243343.6 J55354544.4 K54445554.6 L5555354 Variation Tables for Two Secondary Schools Numbers and colors indicate quintiles in the secondary school distribution for this state.

School One (N=356 students) Con- trol Chal- lengeCare Con- fer Capti- vate Consoli -date Clar- ify Row Mean A11111111.0 B1111111 C11121111.1 D11212111.3 E22111211.4 F12212221.7 G22212221.9 H22312222.0 I42123232.4 J42333222.7 K24444443.7 L15534554.0 M55444444.3 N55555444.7 School Two (N=354 students) A44111212.0 B45111322.4 C4144211 D52313322.7 E52433223.0 F33233443.1 G5332243 H55142333.3 I54243343.6 J55354544.4 K54445554.6 L5555354 Four Possible Scenarios Surveys are scored and results are used for performance evaluations Results are used as a check on the accuracy of the other ratings Teacher receives results, reflects, sets some goals, and is evaluated on their approach and follow through on this process. Some (which?) or all of the above OTHERS?

Each student engagement target has a unique relationship to the 7Cs. So, we may need different professional learning priorities, depending upon which aspects of student engagement seem most critical to improve.

Trust Cooperation Ambitiousness Diligence Satisfaction Care Confer Captivate Clarify Consolidate Challenge Control 7Cs ComponentsEngagement Targets

Four Key Messages Students are good observers There is measurable variation in teaching quality Multiple measures, multiple times, over multiple years While our surveys are useful for evaluation, our primary purpose is professional growth #45

For info on Tripod, contact: Rob Ramsdell Tel: 617-645-2725 [email protected] #46

Documents

Tripod Student Surveys Ronald F. Ferguson, PhD Co-Founder, Tripod Education Partners, Inc., and Faculty Director, the Achievement Gap Initiative at Harvard