1. Aligning Learning Incentives of Students andTeachers: Results from a Social Experiment in Mexican High SchoolsJere Behrman, Susan Parker, Petra E, Todd, Kenneth I. Wolpin

2. OverviewStudy the effects of a performance incentive program aimed atimproving mathematics knowledge in Mexican high schools. 3. OverviewStudy the effects of a performance incentive program aimed atimproving mathematics knowledge in Mexican high schools. Mexico ranked last of 34 OECD countries in the 2009 rankings ofPISA test sores in mathematics. 4. OverviewStudy the effects of a performance incentive program aimed atimproving mathematics knowledge in Mexican high schools. Mexico ranked last of 34 OECD countries in the 2009 rankings ofPISA test sores in mathematics. Less than 10 % of students score at or above the Proficient levelon the 2008 national 9th grade Mathematics test and over 50%score at the Pre-Basic level. 5. OverviewStudy the effects of a performance incentive program aimed atimproving mathematics knowledge in Mexican high schools. Mexico ranked last of 34 OECD countries in the 2009 rankings ofPISA test sores in mathematics. Less than 10 % of students score at or above the Proficient levelon the 2008 national 9th grade Mathematics test and over 50%score at the Pre-Basic level.ALI program designed to promote mathematics achievementthrough monetary incentives for performance on curriculum-basedtests. 6. Empirical Literature on SchoolPerformance Incentives1. Teacher Incentives: Glewwe et. Al. (2003) in Kenya,Springer et. al. (2010) in Tennessee, Muralidharan andSundararaman (2011) in India. 7. Empirical Literature on SchoolPerformance Incentives1. Teacher Incentives: Glewwe et. Al. (2003) in Kenya,Springer et. al. (2010) in Tennessee, Muralidharan andSundararaman (2011) in India.2. Student Incentives: Angrist and Lavy (2009) in Israel,Kremer et. al. (2009) in Kenya, Fryer (2010) in Chicago,Dallas, New York and Washington D.C., Levitt et. al.(2010) in a Chicago suburb. 8. Empirical Literature on SchoolPerformance Incentives1. Teacher Incentives: Glewwe et. Al. (2003) in Kenya,Springer et. al. (2010) in Tennessee, Muralidharan andSundararaman (2011) in India.2. Student Incentives: Angrist and Lavy (2009) in Israel,Kremer et. al. (2009) in Kenya, Fryer (2010) in Chicago,Dallas, New York and Washington D.C., Levitt et. al.(2010) in a Chicago suburb.3. Student and Teacher Incentives: Jackson (2010) inDallas. 9. Empirical Literature on SchoolPerformance Incentives1. Teacher Incentives: Glewwe et. Al. (2003) in Kenya,Springer et. al. (2010) in Tennessee, Muralidharan andSundararaman (2011) in India.2. Student Incentives: Angrist and Lavy (2009) in Israel,Kremer et. al. (2009) in Kenya, Fryer (2010) in Chicago,Dallas, New York and Washington D.C., Levitt et. al.(2010) in a Chicago suburb.3. Student and Teacher Incentives: Jackson (2010) inDallas. Effect sizes on test scores generally .10-.25 sd. 10. Some Facts About Education in MexicoSchool Completion Rates (1996 1st grade entry cohort):87% complete 6th grade 82% enter 7th grade65% complete 9th grade 62% enter 10th grade47% complete 10th grade39% complete 11th grade38% complete 12th grade28% enter college 11. Some Facts About Education in MexicoSchool Completion Rates (1996 1st grade entry cohort):87% complete 6th grade 82% enter 7th grade65% complete 9th grade 62% enter 10th grade47% complete 10th grade39% complete 11th grade38% complete 12th grade28% enter collegeFederal high schools (1,000 schools):Per-pupil expenditure - 21,000 pesosAverage teacher monthly salary - 20,000 pesosPct. of high school students attending - 25%Average annual tuition 1,200 pesos 12. ALI ProgramPilot program period: AY 2008/09, 2009/10 and 2010/11.Program participants: all students in 88 Federal highschools in Mexico 24 of 31 states.Overall design: Random assignment to three treatment groups of 20 schools each and 28 control schools 13. TreatmentsTreatments:T1 (20 schools)Payment provided to students related to theirindividual performance. 14. TreatmentsTreatments:T1 (20 schools)Payment provided to students related to theirindividual performance.T2 (20 schools)Payment provided to mathematics teachers basedon the performance of the students in their classes. 15. TreatmentsT3 (20 schools)1. Payment to students based on individualperformance and on performance of classmates. 16. TreatmentsT3 (20 schools)1. Payment to students based on individualperformance and on performance of classmates. 2. Payment to mathematics teachers based onperformance of students in their classes and of studentsin all other mathematics classes. 17. TreatmentsT3 (20 schools)1. Payment to students based on individualperformance and on performance of classmates. 2. Payment to mathematics teachers based onperformance of students in their classes and of studentsin all other mathematics classes.3. Payment to non-mathematics teachers based onperformance of students in all mathematics classes. 18. TreatmentsT3 (20 schools)1. Payment to students based on individualperformance and on performance of classmates. 2. Payment to mathematics teachers based onperformance of students in their classes and of studentsin all other mathematics classes.3. Payment to non-mathematics teachers based onperformance of students in all mathematics classes.4. Payment to principals and other administratorsbased on performance of students in all mathematicsclasses. 19. RandomizationSchool-based block randomization design.Nine blocks characterized by school size and graduationrates prior to the initiation of the program.Within each block, schools are allocated randomly to thethree treatment groups and the control group. 20. ENLACE scores are reported both standardized (mean=500, sd=100) andin four categories.National figures include students who never attended high school. 21. ALI TestsThe tests are based on the standardized curriculum foreach grade and were produced especially for this projectby a Mexican educational testing service (CENEVAL). 22. ALI TestsThe tests are based on the standardized curriculum foreach grade and were produced especially for this projectby a Mexican educational testing service (CENEVAL).Grade 10: Algebra, Geometry and Trigonometry (classhours - 4hrs/wk) 2.5 hour ALI examination 23. ALI TestsThe tests are based on the standardized curriculum foreach grade and were produced especially for this projectby a Mexican educational testing service (CENEVAL).Grade 10: Algebra, Geometry and Trigonometry (classhours - 4hrs/wk) 2.5 hour ALI examinationGrade 11: Analytical Geometry, Calculus (class hours -4hrs/wk) 2.5 hour ALI examination 24. ALI TestsThe tests are based on the standardized curriculum foreach grade and were produced especially for this projectby a Mexican educational testing service (CENEVAL).Grade 10: Algebra, Geometry and Trigonometry (classhours - 4hrs/wk) 2.5 hour ALI examinationGrade 11: Analytical Geometry, Calculus (class hours -4hrs/wk) 2.5 hour ALI examinationGrade 12: Probability and Statistics, Applied Statistics(class hours - 5hrs/wk) 2.5 hour examination on 12thgrade material, 1.25 hours each on 10th and 11th gradematerial. 25. Incentive Schedules : Teachers (T2, T3) 26. Incentive SchedulesIncentive schedules are based on the categorical scoreson an initial test (grades 10 and 11) and on the end-of-year ALI test (grades 10,11,12). 27. Incentive SchedulesIncentive schedules are based on the categorical scoreson an initial test (grades 10 and 11) and on the end-of-year ALI test (grades 10,11,12).The initial test score for the tenth grade is thenational 9th year mathematics ENLACE (curriculum-based test). 28. Incentive SchedulesIncentive schedules are based on the categorical scoreson an initial test (grades 10 and 11) and on the end-of-year ALI test (grades 10,11,12).The initial test score for the tenth grade is thenational 9th year mathematics ENLACE (curriculum-based test).The initial test score for the eleventh grade is the10th grade ALI curriculum test (except in first year 9th grade ENLACE). 29. Incentive SchedulesThe 10th grade test score cutoffs mimic the controlgroups distribution of categorical scores on the 9th grademathematics ENLACE. 30. Incentive SchedulesThe 10th grade test score cutoffs mimic the controlgroups distribution of categorical scores on the 9th grademathematics ENLACE.The 11th grade test score cutoffs mimic the controlgroups distribution on the 9th grade ENLACE in year 1and on the 10th grade ALI test in years 2 and 3. 31. Incentive SchedulesThe 10th grade test score cutoffs mimic the controlgroups distribution of categorical scores on the 9th grademathematics ENLACE.The 11th grade test score cutoffs mimic the controlgroups distribution on the 9th grade ENLACE in year 1and on the 10th grade ALI test in years 2 and 3.The 12th grade test score cutoffs mimic the controlgroups distribution on the 12th grade mathematicsENLACE. 32. Incentive Schedules: Students (T1, T3) 33. Incentive Schedules: Students (T1, T3) 34. Incentive Schedules : Students (T1, T3) 35. Incentive Schedules : Students (T1, T3) 36. Incentive Schedules : Students (T1, T3) 37. Incentive Schedules : Students (T1, T3) 38. Incentive Schedules : Teachers (T2, T3) 39. Incentive Schedules : Teachers (T2, T3) 40. Incentive Schedules : Teachers (T2, T3) 41. Incentive Schedules: Performance of Others (T3)In addition to the incentives based on own performance,Students receive an additional payment of one percent of thetotal amount received by all of the students in their class. 42. Incentive Schedules: Performance of Others (T3)In addition to the incentives based on own performance,Students receive an additional payment of one percent of thetotal amount received by all of the students in their class.FTE mathematics teachers receives an additional payment of25 percent of the average (FTE) amount earned by the othermathematics teachers. 43. Incentive Schedules: Performance of Others (T3)In addition to the incentives based on own performance,Students receive an additional payment of one percent of thetotal amount received by all of the students in their class.FTE mathematics teachers receives an additional payment of25 percent of the average (FTE) amount earned by the othermathematics teachers.A FTE non-mathematics teacher receives a payment of 25percent of the average (FTE) amount earned by themathematics teachers. 44. Incentive Schedules: Performance of Others (T3)In addition to the incentives based on own performance,Students receive an additional payment of one percent of thetotal amount received by all of the students in their class.FTE mathematics teachers receives an additional payment of25 percent of the average (FTE) amount earned by the othermathematics teachers.A FTE non-mathematics teacher receives a payment of 25percent of the average (FTE) amount earned by themathematics teachers.The principal of the school receives a payment of 50 percent ofthe average (FTE) amount earned by the mathematicsteachers. 45. AttritionThere are existing incentive programs that pay students for attendanceand the bonus from the ALI program is uncertain. 46. Testing Protocol One external monitor per classroom one overall externalsupervisor in school. 47. Testing Protocol One external monitor per classroom one overall externalsupervisor in school. Teachers not present during test administration. 48. Testing Protocol One external monitor per classroom one overall externalsupervisor in school. Teachers not present during test administration. Test answer sheets and test booklets collected by monitors atthe end of the exam and returned to testing agency for scoring. 49. Testing Protocol One external monitor per classroom one overall externalsupervisor in school. Teachers not present during test administration. Test answer sheets and test booklets collected by monitors atthe end of the exam and returned to testing agency for scoring. Despite these measures, we found evidence that led to asuspicion of student cheating. In some treatment schools, students and teachers received unusually highlevels of incentive payments. Some answer sheets of students within the same classroom exhibitedstrings of matching correct and incorrect answers. 50. Analysis of Student CopyingAnalysis performed by George Wesolowsky (professor emeritus,McMaster University) uses method described in his J. of AppliedStatistics (2000) article.1. Statistical model determining probability that student i answers multiplechoice question j incorrectly Incorporates a parametric function of the difficulty of the question and the ability of the student.2. Determine for every pair of students and for each question, the probabilitythat the two students will have the same answer (assume, e.g., that all wronganswers are equally likely).3. The probability distribution of the number of matches is a compoundbinomial; approximated as normal.4. Choose a critical value for the number of observed matches. Reject the nullof no copying if the number of matches exceeds the critical value. A Bonferronicorrection is used with a critical value such that the probability is one that atleast one pair of students is falsely accused. 51. A Caveat: Lack of Test-Taking Effort by Control StudentsAssumption 1:a. test-taking effort of T1 students no less than that of T3 studentsb. T1 effect is zero in all yearsLower Bound Estimate of Treatment Effect in Year 3:a T3:31.1 standardized points for 10th grade16.9 for 11th grade34.0 for 12th gradea. Adjusted for copying. 52. A Caveat: Lack of Test-Taking Effort by Control StudentsAssumption 2:a. test-taking effort of C students same in all years.b. T1 effect is zero in year one onlyLower Bound Estimate of Treatment Effect in Year 3a: T3: 46.5 standardized points for 10th grade28.5 for 11th grade47.1 for 12th grade T1: 15.4 standardized points for 10th grade11.6 for 11th grade13.1 for 12th gradea. Uses treatment effects adjusted for copying. 53. Payment Outcomes 54. Payment Outcomes 55. Payment Outcomes 56. Conclusions Evaluated effects of student and teacher incentiveprograms, in isolation and combined Student incentives alone increased mathematics testscores by 0.2-0.3 std. dev. (adjusted estimates) Teacher incentives alone had no effect Combined student and teacher incentives increasedscores by 0.3-0.6 std. dev. Positive impacts over entire baseline test scoredistribution, with slightly larger impacts at higherquantiles. 57. TESTIMONIOS ALUMNOS Sulem Hernndez Gonzlez Alfajayucan, Hidalgo Estudio para ser Tcnico en Informtica y con el dinero queme gan, pude comprar mi computadora 58. TESTIMONIOS ALUMNOSSulem Hernndez GonzlezErika del Carmen Oln MagaaAlfajayucan, Hidalgo Comalcalco, TabascoEstudio para ser Tcnico enInformticaque me dieron en el Con lo y con el dinero que programagan, pude comprar mi me apoy a mi familia paracomputadora construir los cimientos de mi casa 59. TESTIMONIOS ALUMNOS Cristian Prez Prez Santiago Papasquiaro,Guanajuato Con lo que me gan compr una vaca y hace poco tuvo un becerro.Con la venta de la leche que da lavaca, me ayudo para mis tiles de la escuela