View
1
Download
0
Category
Preview:
Citation preview
Randomized Experiments
11-07-15 2:02 PMMore Than 1 Billion People Are Hungry in the World - By Abhijit Banerjee and Esther Duflo | Foreign Policy
Page 1 of 15http://www.foreignpolicy.com/articles/2011/04/25/more_than_1_billion_people_are_hungry_in_the_world?print=yes&hidecomments=yes&page=full
More Than 1 Billion People AreHungry in the WorldBut what if the experts are wrong?
BY ABHIJIT BANERJEE, ESTHER DUFLO | MAY/JUNE 2011
For many in the West, poverty is almost synonymous with hunger. Indeed, the
announcement by the United Nations Food and Agriculture Organization in
11-07-15 2:02 PMMore Than 1 Billion People Are Hungry in the World - By Abhijit Banerjee and Esther Duflo | Foreign Policy
Page 2 of 15http://www.foreignpolicy.com/articles/2011/04/25/more_than_1_billion_people_are_hungry_in_the_world?print=yes&hidecomments=yes&page=full
2009 that more than 1 billion people are suffering from hunger grabbed
headlines in a way that any number of World Bank estimates of how many poor
people live on less than a dollar a day never did.
But is it really true? Are there really more than a billion people going to bed
hungry each night? Our research on this question has taken us to rural villages
and teeming urban slums around the world, collecting data and speaking with
poor people about what they eat and what else they buy, from Morocco to
Kenya, Indonesia to India. We've also tapped into a wealth of insights from our
academic colleagues. What we've found is that the story of hunger, and of
poverty more broadly, is far more complex than any one statistic or grand
theory; it is a world where those without enough to eat may save up to buy a TV
instead, where more money doesn't necessarily translate into more food, and
where making rice cheaper can sometimes even lead people to buy less rice.
But unfortunately, this is not always the world as the experts view it. All too
many of them still promote sweeping, ideological solutions to problems that defy
one-size-fits-all answers, arguing over foreign aid, for example, while the facts
on the ground bear little resemblance to the fierce policy battles they wage.
Jeffrey Sachs, an advisor to the United Nations and director of Columbia
University's Earth Institute, is one such expert. In books and countless speeches
and television appearances, he has argued that poor countries are poor because
they are hot, infertile, malaria-infested, and often landlocked; these factors,
however, make it hard for them to be productive without an initial large
investment to help them deal with such endemic problems. But they cannot pay
for the investments precisely because they are poor -- they are in what
economists call a "poverty trap." Until something is done about these problems,
neither free markets nor democracy will do very much for them.
But then there are others, equally vocal, who believe that all of Sachs's answers
are wrong. William Easterly, who battles Sachs from New York University at the
other end of Manhattan, has become one of the most influential aid critics in his
11-07-15 2:02 PMMore Than 1 Billion People Are Hungry in the World - By Abhijit Banerjee and Esther Duflo | Foreign Policy
Page 2 of 15http://www.foreignpolicy.com/articles/2011/04/25/more_than_1_billion_people_are_hungry_in_the_world?print=yes&hidecomments=yes&page=full
2009 that more than 1 billion people are suffering from hunger grabbed
headlines in a way that any number of World Bank estimates of how many poor
people live on less than a dollar a day never did.
But is it really true? Are there really more than a billion people going to bed
hungry each night? Our research on this question has taken us to rural villages
and teeming urban slums around the world, collecting data and speaking with
poor people about what they eat and what else they buy, from Morocco to
Kenya, Indonesia to India. We've also tapped into a wealth of insights from our
academic colleagues. What we've found is that the story of hunger, and of
poverty more broadly, is far more complex than any one statistic or grand
theory; it is a world where those without enough to eat may save up to buy a TV
instead, where more money doesn't necessarily translate into more food, and
where making rice cheaper can sometimes even lead people to buy less rice.
But unfortunately, this is not always the world as the experts view it. All too
many of them still promote sweeping, ideological solutions to problems that defy
one-size-fits-all answers, arguing over foreign aid, for example, while the facts
on the ground bear little resemblance to the fierce policy battles they wage.
Jeffrey Sachs, an advisor to the United Nations and director of Columbia
University's Earth Institute, is one such expert. In books and countless speeches
and television appearances, he has argued that poor countries are poor because
they are hot, infertile, malaria-infested, and often landlocked; these factors,
however, make it hard for them to be productive without an initial large
investment to help them deal with such endemic problems. But they cannot pay
for the investments precisely because they are poor -- they are in what
economists call a "poverty trap." Until something is done about these problems,
neither free markets nor democracy will do very much for them.
But then there are others, equally vocal, who believe that all of Sachs's answers
are wrong. William Easterly, who battles Sachs from New York University at the
other end of Manhattan, has become one of the most influential aid critics in his
11-07-15 2:02 PMMore Than 1 Billion People Are Hungry in the World - By Abhijit Banerjee and Esther Duflo | Foreign Policy
Page 3 of 15http://www.foreignpolicy.com/articles/2011/04/25/more_than_1_billion_people_are_hungry_in_the_world?print=yes&hidecomments=yes&page=full
books, The Elusive Quest for Growth and The White Man's Burden.
Dambisa Moyo, an economist who worked at Goldman Sachs and the World
Bank, has joined her voice to Easterly's with her recent book, Dead Aid. Both
argue that aid does more bad than good. It prevents people from searching for
their own solutions, while corrupting and undermining local institutions and
creating a self-perpetuating lobby of aid agencies. The best bet for poor
countries, they argue, is to rely on one simple idea: When markets are free and
the incentives are right, people can find ways to solve their problems. They do
not need handouts from foreigners or their own governments. In this sense, the
aid pessimists are actually quite optimistic about the way the world works.
According to Easterly, there is no such thing as a poverty trap.
This debate cannot be solved in the abstract. To find out whether there are in
fact poverty traps, and, if so, where they are and how to help the poor get out of
them, we need to better understand the concrete problems they face. Some aid
programs help more than others, but which ones? Finding out required us to
step out of the office and look more carefully at the world. In 2003, we founded
what became the Abdul Latif Jameel Poverty Action Lab, or J-PAL. A key part of
our mission is to research by using randomized control trials -- similar to
experiments used in medicine to test the effectiveness of a drug -- to understand
what works and what doesn't in the real-world fight against poverty. In practical
terms, that meant we'd have to start understanding how the poor really live
their lives.
Take, for example, Pak Solhin, who lives in a small village in West Java,
Indonesia. He once explained to us exactly how a poverty trap worked. His
parents used to have a bit of land, but they also had 13 children and had to build
so many houses for each of them and their families that there was no land left
for cultivation. Pak Solhin had been working as a casual agricultural worker,
which paid up to 10,000 rupiah per day (about $2) for work in the fields. A
recent hike in fertilizer and fuel prices, however, had forced farmers to
economize. The local farmers decided not to cut wages, Pak Solhin told us, but to
11-07-15 2:02 PMMore Than 1 Billion People Are Hungry in the World - By Abhijit Banerjee and Esther Duflo | Foreign Policy
Page 3 of 15http://www.foreignpolicy.com/articles/2011/04/25/more_than_1_billion_people_are_hungry_in_the_world?print=yes&hidecomments=yes&page=full
books, The Elusive Quest for Growth and The White Man's Burden.
Dambisa Moyo, an economist who worked at Goldman Sachs and the World
Bank, has joined her voice to Easterly's with her recent book, Dead Aid. Both
argue that aid does more bad than good. It prevents people from searching for
their own solutions, while corrupting and undermining local institutions and
creating a self-perpetuating lobby of aid agencies. The best bet for poor
countries, they argue, is to rely on one simple idea: When markets are free and
the incentives are right, people can find ways to solve their problems. They do
not need handouts from foreigners or their own governments. In this sense, the
aid pessimists are actually quite optimistic about the way the world works.
According to Easterly, there is no such thing as a poverty trap.
This debate cannot be solved in the abstract. To find out whether there are in
fact poverty traps, and, if so, where they are and how to help the poor get out of
them, we need to better understand the concrete problems they face. Some aid
programs help more than others, but which ones? Finding out required us to
step out of the office and look more carefully at the world. In 2003, we founded
what became the Abdul Latif Jameel Poverty Action Lab, or J-PAL. A key part of
our mission is to research by using randomized control trials -- similar to
experiments used in medicine to test the effectiveness of a drug -- to understand
what works and what doesn't in the real-world fight against poverty. In practical
terms, that meant we'd have to start understanding how the poor really live
their lives.
Take, for example, Pak Solhin, who lives in a small village in West Java,
Indonesia. He once explained to us exactly how a poverty trap worked. His
parents used to have a bit of land, but they also had 13 children and had to build
so many houses for each of them and their families that there was no land left
for cultivation. Pak Solhin had been working as a casual agricultural worker,
which paid up to 10,000 rupiah per day (about $2) for work in the fields. A
recent hike in fertilizer and fuel prices, however, had forced farmers to
economize. The local farmers decided not to cut wages, Pak Solhin told us, but to
11-07-20 1:00 AMSearch | The Abdul Latif Jameel Poverty Action Lab
Page 1 of 1http://www.povertyactionlab.org/search/apachesolr_search?view=map&filters=type:evaluation
The Abdul Latif Jameel Poverty Action Lab Contact J-PAL | Subscribe
TRANSLATING RESEARCH INTO ACTION
Legend Education Finance & Microfinance Environment & Energy Health Political Economy &Governance
Labor Markets Agriculture
Standard Search Region-Theme Grid
Search
To refine displayed results, select one ormore of the categories below:
Keyword:
Themes all...
Policy Goals all...
Region all...
Country all...
Researchers all...
Status all...
Data all...
11-07-15 2:02 PMMore Than 1 Billion People Are Hungry in the World - By Abhijit Banerjee and Esther Duflo | Foreign Policy
Page 6 of 15http://www.foreignpolicy.com/articles/2011/04/25/more_than_1_billion_people_are_hungry_in_the_world?print=yes&hidecomments=yes&page=full
But what if the poor are not, in general, eating too little food? What if, instead,
they are eating the wrong kinds of food, depriving them of nutrients needed to
be successful, healthy adults? What if the poor aren't starving, but choosing to
spend their money on other priorities? Development experts and policymakers
would have to completely reimagine the way they think about hunger. And
governments and aid agencies would need to stop pouring money into failed
programs and focus instead on finding new ways to truly improve the lives of
the world's poorest.
Consider India, one of the great puzzles in this age of food crises. The standard
media story about the country, at least when it comes to food, is about the rapid
rise of obesity and diabetes as the urban upper-middle class gets richer. Yet the
real story of nutrition in India over the last quarter-century, as Princeton
professor Angus Deaton and Jean Drèze, a professor at Allahabad University and
a special advisor to the Indian government, have shown, is not that Indians are
becoming fatter: It is that they are in fact eating less and less. Despite the
country's rapid economic growth, per capita calorie consumption in India has
declined; moreover, the consumption of all other nutrients except fat also
appears to have gone down among all groups, even the poorest. Today, more
than three-quarters of the population live in households whose per capita
calorie consumption is less than 2,100 calories in urban areas and 2,400 in rural
areas -- numbers that are often cited as "minimum requirements" in India for
those engaged in manual labor. Richer people still eat more than poorer people.
But at all levels of income, the share of the budget devoted to food has declined
and people consume fewer calories.
What is going on? The change is not driven by declining incomes; by all
accounts, Indians are making more money than ever before. Nor is it because of
rising food prices -- between the early 1980s and 2005, food prices declined
relative to the prices of other things, both in rural and urban India. Although
food prices have increased again since 2005, Indians began eating less precisely
Notes from: “Using Randomization in Development Economics Research: A Toolkit” (Duflo, Glennerster, Kremer 2008)
Randomization is now integral part of development economics
Over last ten years has grown to dominate the field
Diverse topics: education, health, technology adoption, micro-credit…
Randomized evaluations in developing countries – relatively cheap
Working with local partners allows for flexibility to researchers
Why Randomize?
Problem of causal inference
What is causal impact of specific program requires a counterfactual
At a given point in time, an individual is either exposed to the program or not
Comparing same individual over time does not work because other factors which affect outcomes have changed --- cannot get reliable estimate of program’s impact
Cannot obtain impact of program on a given individual – can obtain average impact of program on a group of individuals by comparing them to similar group of individuals who were not exposed to program
Need a comparison group – group of people who in absence of treatment would have had similar outcomes to those who received treatment
Reality- individuals exposed to treatments differ from those not exposed
programs placed in specific areas (poorer)
individuals screened for participation
decision to participate is voluntary, creating self-selection
Any difference between groups can be attributed to both impact of program and pre-existing differences (selection bias) – cannot decompose
Randomized evaluation removes selection bias
N individuals selected from population (not a random sample)
Divide N individuals into treatment and control randomly
Average treatment effect is difference in empirical means across two groups
Had the treatment not occurred – their outcomes would have been in expectation the same (selection bias equal to zero)
Provides and unbiased estimate of the impact of the program in the sample under study – estimate is internally valid
Many ways in which the assumptions in the simple framework may fail when randomized evaluations are implemented in the field in developing countries
Practicalities
Partners: governments, NGOs, private companies
Implementing real-world programs and interested in finding out whether they work or how to improve them
Partner is always in charge of implementing program
Government programs:
Meant to serve entire eligible population -- pilot programs run before
Most famous example in development: PROGRESA (grants to women) – randomized evaluation by government of Mexico --- due to budgets constraints could not implement in all communities at once
Randomized evaluations in collaboration with governments still relatively rare (require cooperation at high political levels – difficult to generate consensus for implementation)
Recent spread of randomized evaluations owes much to move towards working with non-governmental organizations (NGOs)
NGOs not expected to serve entire population
Even small organizations can substantially affect budgets for households, schools, or health clinics
Often eager to work with researchers to assess effectiveness of operations
For profit firms also started getting interested in randomized evaluations, goal is to understand their business and better serve their clients
Karlan et al. worked with private lender in South Africa – able to test models of lending
Design
Sample size and other design choices will affect the power of the experiment
Tradeoffs between how expensive data collection is compared to treatment – determines size of comparison group (data collection cheap if using administrative data)
Control Variables:
Controlling for covariates affected by the treatment would bias the estimate of the treatment effect by capturing part of the impact
Information on covariates should be collected in the baseline surveys (pre-treatment value of the outcome is of special interest)
Can reduce sample size requirement needed – e.g. controlling for baseline test scores in evaluations of education interventions greatly improves precision of estimates
Practical design choice is whether to randomize the intervention at the level of the individual, family, village, district etc.
Conducting baseline survey – in theory randomization renders baseline surveys unnecessary since it ensures treatment and comparison group are similar in expectation
Still want to collect:
Control variables will reduce sample size requirements
Check randomization was done appropriately
Notes from: “The Experimental Approach to Development Economics” (Banerjee and Duflo 2009)
Veritable explosion of randomized experiments in development economics
Rising tide of criticism
(most found in Heckman 1992 – mainly commenting on field of labour economics)
Promise of experiments:
Surprisingly positive results
o (Miguel/Kremer 2004) showed that deworming treatment (costs 49 cents/child per year) can reduce abesenteeims from by school by one-quarter
o In terms of increasing attendance – deworming is 20 times as effective as hiring an extra teacher, even though both work in the sense of generating statistically significant improvements
o Economic intuition would not have helped us come to this conclusion
o NGOs were equally uniformed about this comparison
Multiple treatment experiments can be informative
o Duflo, Kremer, Robinson (2010) reflects an iterative process
o succession of experiments on fertilizer use were run over a period of several years
o each set of results prompting the need to try out a series of new variation in order to better understand results of previous one
Theoretical Motivation
o Experiments designed to assess whether there is a demand for commitment products (Ashraf, Karlan, and Yin 2006) – came from theoretical motivation
o Karlan and others – experiments emerging as powerful too for testing theories
Biggest Advantage:
Experiments may be that they take us into terrain where observational approaches are not available
Objections raised by critics best viewed as warnings against over-interpreting experimental results
Also concerns about what experiments are doing to development economics as a field
Generalizability
Environmental Dependence - Core element of generalizability – would the same result occur in a different setting?
Effect is not constant across individuals – likely vary systematically with covariates?
Concern of implementer effects and compliance – smaller organization (NGO) – estimated treatment effect reflects unique characteristics of implementer
e.g. some NGOs refuse to randomize
Randomization Issues
Fact that there is an experiment going on might generate selection effects that would not arise in non-experimental setting (being part of an experiment and being monitored influences participants)
Villagers not used to private organization going around offering them things
Necessary that individuals are not aware that they are excluded from program (difficult when randomization is at individual level, easier if randomization is at village level)
Equilibrium Effects
Program effects from small study may not generalize when program is scaled up
e.g. :
Vouchers to go to private school
Students end up with better education and higher incomes
Scale up program to national level
Crowding in private schools (collapse of public schools)
Returns to education fall because of increased supply
Experimental evidence overstates returns to vouchers program
Notes from: “Instruments, Randomization, and Learning about Development” (Deaton 2010)
Effectiveness of development assistance is topic of great public interest
Much public debate among non-economists takes it for granted that, if the funds were made available, poverty would be eliminated -- Amongst economists, it is mixed.
Macro perspective: can foreign assistance raise growth and eliminate poverty?
Micro perspective: what sorts of projects are likely to be effective? Should aid focus on roads, electricity, schools, health clinics?
Answer – we don’t know – how should we go about finding out?
Frustration with Aid organizations
Particularly the World Bank
Allegedly failing to learn from its projects and to build up a systematic catalogue of what works and what does not
Movement toward randomized controlled experiments:
Esther Duflo:
“ randomized trials can revolutionize social policy during 21st century just as they revolutionized medicine during the 20th”
---- Lancet editorial headed “ The World Bank is finally embracing science”
Deaton argues:
under ideal circumstances randomized evaluations of projects are useful for obtaining convincing estimates of the average treatment effect of a program or project
This focus is too narrow and too local to tell us “what works” in development and to design policy or to advance scientific knowledge about development processes
Argues that work needs to be refocused – not answer which projects work but why
Randomization in the Tropics
Under ideal conditions an RCT can estimate certain quantities of interest with minimal assumptions
Originally laid out by Neyman in 1920s:
What we observe in the data:
E(Y1|T=1) – E(Y0|T=0) [*]
What we would like to know is effect of being treated on the treated:
E(Y1-Y0|T=1)
[*] rewritten as:
[E(Y1|T=1) – E(Y0|T=1)] + [E(Y0|T=1) – E(Y0|T=0)]
second term is = 0 by randomization
given expectation is a linear operator we have: E(Y1-Y0|T=1)
Difference in means between treatment and control groups is estimate of the treatment effect on the treated
Since the treatment and control groups are random, is an estimate of the treatment effect for all
Result depends on randomization and linearity of expectations
RCT is only informative of the mean of the treatment effects
Cannot identify anything else in the distribution --- e.g. median which could be of equal interest to policy makers
Also would like to know the fraction of population for which the treatment was positive (cannot know this either)
So if very few are receiving large benefits, everyone else is hurting – could not know this
In practice, researchers who run RCTs present results other than the mean:
Yi = b0 + b1Ti + b2Xi + b3Ti*Xi + ei
Xi are baseline controls
Interaction between Xi and Ti – can see if treatment affected some groups more than others
These estimates depend on more assumptions than trial itself
One immediate charge is data mining
Sufficiently determined examination of any trial will eventually reveal some subgroup for which treatment effect was significant
Such analysis does not share the special features of an RCT – must be assessed same way we assess any other non experimental or econometric study
Also no (econometric) guarantee that another RCT run just on that subgroup will yield same result
Also relevant are assumptions – regarding variances between treatment and control group
Bigger question:
RCTs allow investigator to induce variation that might not arise nonexperimentally – but are these the relevant ones?
RCTs of “what works”
even when done without error of contamination
unlikely to be helpful for policy or move beyond the local
unless they tell us something about why
RCTs are not targeted or suited to these questions
Actual policy will always be different than experiments:
General equilibrium effects that operate on large scale
Outcomes are different when everyone is covered by treatment rather than a few
Experimental subjects are not representative of population
Small development projects at village level do not attract attention of corrupt politicians
Scientists or experimentalists more careful than government implementers
Transporting successful experiments?
Mexico’s PROGRESA program
Conditional cash transfer program paid to parents if children attend schools and clinics
Now in 30 other countries
Is this a good thing?
Cannot simply be exported if countries have
Pre-existing anti-poverty programs with conditional transfers
No capacity to meet increased demands of education and health care
No political support
Combination of mechanism and context that makes for scientific progress
Much interest in RCTs, and instrumental variables, and other econometric techniques that mimic random allocation
comes from skepticism of economic theory
impatience with its ability to deliver structures that seem helpful in interpreting reality
Internal versus external validity:
Contrast between the rigor applied to establish internal validity and the looser analysis to render it policy relevant
To do this typically use some theory or some other information from observables – both go against simplicity of RCTs
Applied and theoretical economists have never been so far apart
Failure to reintegrate is not an option
Otherwise no chance of long term scientific progress extending from the RCTs.
RCTs that are not theoretically guided are unlikely to have more than local validity
11
The millennium development goal calls for a universal primary education by 2015 little consensus on how to achieve this goal or how much it
would cost
12
One view attracting additional children to school will be difficult since
most children not in school in developing countries are earning income their families need
Another view potential contribution of children of primary school age to family
income is very small hence modest incentives could significantly increase enrollment
13
Reducing the Cost of Education Some argue school fees prevent many students from attending school cite dramatic estimates from sub-Saharan Africa
free schooling introduced -- primary school enrollment
reportedly doubled Often data used for these estimates are unclear: free schooling is sometimes announced simultaneously with
other policy initiatives often accompanied by programs that replace school fees with per
pupil grants from the central government which create incentives for schools to over-report enrollment
14
Randomized experiments can isolate the impact of reducing costs on the quantity of schooling Several programs have gone beyond simply reducing school fees by actually paying students to attend school in the form of either cash grants or school meals School health programs can also increase quantity of schooling but this raises the question of how best to implement such programs One view is that the reliance on external financing of medicine is not sustainable and instead advocates health education, water and sanitation improvements and so forth
15
Quality of Education Notes from “Teacher Absence in India” (Kremer et. al.) Study entails a nationally representative survey on 3700 schools in India Three unannounced visits were made to each school
16
Absence data comes from direct physical verification of teacher’s presence not relying on logbooks, interviews, etc.
Teacher is recorded as absent if investigator could not find the teacher in the school during regular working hours
Journal of the European Economic Association (Resubmitted version, 11/27/04)
4
which absence calculations based on a similar methodology are available
(Table 1).3 Only 45 percent of teachers were actively engaged in teaching at
the time of the visit.4
Within India, the absence rate ranged from 15 percent in Maharashtra to 42
percent in Jharkand (Table 2).5 Absence rates are generally higher in low-
income states: doubling per capita income is associated with a 4.7 percentage
3 Most of these estimates come from other countries covered by the same research project on
provider absence in education and health, carried out by the authors of this study and using
standardized methodology (Chaudhury and others 2004).
4 Even with a generous allowance for the possibility that enumerators’ visits diverted some
teachers from teaching, it is unlikely that more than half of the teachers would have been teaching
at the time of the visit. See Kremer and others (2004).
5 Table 2 includes 19 of the 20 states surveyed. Fieldwork in the twentieth state, Delhi, was
delayed for bureaucratic reasons, and the data were received too late to be analyzed here.
Teacher absence (%)
Peru 11Ecuador 14Papua New Guinea 15Bangladesh 16Zambia 17Indonesia 19India 25Uganda 27
TABLE 1: Teacher absence rates by country
Source: Chaudhury, Hammer, Kremer, Muralidharan, and Rogers (2004) for most countries; Habyarimana and others (2004) for Zambia; World Bank (2004) for Papua New Guinea.
Journal of the European Economic Association (Resubmitted version, 11/27/04)
5
point lower predicted absence. The rates of teaching activity among the
teachers who are present are lower in higher-absence states and schools. In
some states, only 20 to 25 percent of teachers were engaged in teaching at the
time of the visit.
Absence rates are considerably higher than could be accounted for by
official non-teaching duties, such as staffing polling stations during elections or
conducting immunization campaigns, which are sometimes cited as important
causes of absence. Based on the responses of each school’s head teacher or
primary respondent, official non-teaching duties account for only about 4
percent of total absences. In other words, on any given day, only about 1
percent of primary teachers are absent because they are carrying out official
non-teaching-related duties.6 Preliminary calculations by the authors suggest
6 While stated reasons for absence should be taken with a grain of salt, there does not appear to
be any reason for head teachers to understate this cause of absence.
State Absence (%) State Absence (%)
Maharashtra 14.6 West Bengal 24.7Gujarat 17.0 Andhra Pradesh 25.3Madhya Pradesh 17.6 Uttar Pradesh 26.3Kerala 21.2 Chhatisgarh 30.6Himachal Pradesh 21.2 Uttaranchal 32.8Tamil Nadu 21.3 Assam 33.8Haryana 21.7 Punjab 34.4Karnataka 21.7 Bihar 37.8Orissa 23.4 Jharkhand 41.9Rajasthan 23.7 Weighted Average 24.8
TABLE 2: Teacher absence in public schools by state
19
One in four teachers are absent in a typical primary school in India Absence rates are generally higher in low-income states Higher teachers’ salaries do not seem to be associated with lower teacher absence Since nominal teachers’ salaries are very similar across states relative teachers’ salaries are higher in poorer states
yet poorer states have higher absence rates
20
Determinants of Teacher Absence
Journal of the European Economic Association (Resubmitted version, 11/27/04)
7
Higher teacher salaries do not seem to be associated with lower teacher
absence. We did not directly collect data on individual teacher salaries, but in
every Indian state, salaries increase with education, experience, and rank.
Teachers with a college degree are 2-2.5 percentage points more likely to be
absent. Being 10 years older increases the probability of absence by around
1.0-1.5 percentage points (though this effect becomes weaker after controlling
for state fixed effects). Head teachers are 4-5 percentage points more likely to
be absent than regular teachers (even after controlling for age and education).
Since absence increases with all three of these variables, it is likely that better-
paid teachers are more absent. Similarly, although regular teachers are typically
Category Yes No
Rich state (96-97 per capita income > $275)? 21.7 28.0
Female? 21.9 27.2
Older than 40 years? 27.1 21.4
Completed bachelors degree? 24.2 21.7
School has a toilet for teachers? 21.6 27.1
School has electricity connection? 19.2 28.2
Commute < 30 minutes? 21.4 25.2
Rural school? 25.2 22.9
Headteacher is absent? 22.2 18.5
School inspected in the past 3 months? 21.0 27.0
Belongs to this town/village? 21.6 23.3
PTA has met in the past 3 months? 21.0 26.5
TABLE 5. Absence rates (in percent) by various individual and school-level characteristics (public schools only)
Journal of the European Economic Association (Resubmitted version, 11/27/04)
8
[1] [2] [3]
No state or village fixed effects
With state fixed effects
With village/town fixed effects
1.34 2.09 2.06(0.54)** (0.53)*** (0.53)***
0.12 0.05 0.08(0.03)*** (0.03) (0.03)**
2.31 1.89 1.86(0.51)*** (0.51)*** (0.51)***
0.26 0.17 -0.58(0.59) (0.60) (0.65)4.19 4.54 4.61
(0.64)*** (0.62)*** (0.64)***-0.6 0.06 1.49
(1.32) (1.25) (1.28)-0.31 0.56 -0.07(0.64) (0.64) (0.73)0.56 1.12 0.94
(0.75) (0.74) (0.76)-1.08 -1.06 -0.84
(0.53)** (0.51)** (0.51)*0.5 0.18 1.86
(0.79) (0.78) (0.96)*-0.87 -0.9 -0.08(0.68) (0.65) (0.85)-1.5 -1.49 -0.93
(0.32)*** (0.32)*** (0.39)**0.77 -0.09
(0.84) (0.79)0.84 0.95 1.08
(0.37)** (0.37)** (0.64)*1.17 0.67 0.9
(0.70)* (0.67) (0.82)-0.05 -0.04 -0.08
(0.02)*** (0.02)** (0.02)***-2.28 -1.94 -2.45
(0.66)*** (0.64)*** (0.85)***-1.42 -1.15 -0.74
(0.36)*** (0.38)*** (0.46)-0.04 0 0.02(0.04) (0.04) (0.04)-0.37 -0.96 -0.86(0.54) (0.53)* (0.56)-0.79 0.44 -1.83(0.96) (0.92) (1.21)-1.7 -1.36 0.16
(0.72)** (0.68)** (0.87)-1.99 -1.36 -2.35
(0.70)*** (0.97) (1.33)*2.34 3.58 7.88
(1.23)* (1.21)*** (1.37)***0.01 1.65 7.54
(1.41) (1.41) (1.64)***-6.73
(2.49)***Constant 51.14 22.81 15.59
(9.80)*** (2.45)*** (2.82)***
Observations 41725 43130 43142R-squared 0.09 0.09 0.18
* significant at 10%; ** significant at 5%; *** significant at 1%
Regressions include a full set of controls for the day of week of the visit, for the round of the visit, and for the time of day of the visit relative to the school day (see full paper for details).
Mid-day meal exists
Government public school
Private aided school
Log of state per-capita income
The binary absence variable has been multiplied by 100 to allow the coefficients to be read as percentage changes.
Notes: Robust standard errors clustered at the school level are given in parentheses.
Duration of current posting (years)
Teacher's place of origin is village/town where school is located
PTA exists
PTA met at least once in last 3 months
Multigrade teaching
Pupil-teacher ratio
School has been inspected in past 3 months
Mean parental education of 4th-grade children (1-7 scale)
Recognition/award scheme exists in district
School infrastructure index (1 to 5)
Rural school
Distance to nearest paved road (1 to 5)
Belongs to a teachers union
Married
Has children of school age (5-14)
Paid regularly (1=Yes)
Has a college degree
Attended training in last 6 months
Head teacher
Contract teacher
TABLE 6: OLS estimates of teacher absence Dependent variable (visit-level observation):
Gender (1 = Male)
Age
Journal of the European Economic Association (Resubmitted version, 11/27/04)
8
[1] [2] [3]
No state or village fixed effects
With state fixed effects
With village/town fixed effects
1.34 2.09 2.06(0.54)** (0.53)*** (0.53)***
0.12 0.05 0.08(0.03)*** (0.03) (0.03)**
2.31 1.89 1.86(0.51)*** (0.51)*** (0.51)***
0.26 0.17 -0.58(0.59) (0.60) (0.65)4.19 4.54 4.61
(0.64)*** (0.62)*** (0.64)***-0.6 0.06 1.49
(1.32) (1.25) (1.28)-0.31 0.56 -0.07(0.64) (0.64) (0.73)0.56 1.12 0.94
(0.75) (0.74) (0.76)-1.08 -1.06 -0.84
(0.53)** (0.51)** (0.51)*0.5 0.18 1.86
(0.79) (0.78) (0.96)*-0.87 -0.9 -0.08(0.68) (0.65) (0.85)-1.5 -1.49 -0.93
(0.32)*** (0.32)*** (0.39)**0.77 -0.09
(0.84) (0.79)0.84 0.95 1.08
(0.37)** (0.37)** (0.64)*1.17 0.67 0.9
(0.70)* (0.67) (0.82)-0.05 -0.04 -0.08
(0.02)*** (0.02)** (0.02)***-2.28 -1.94 -2.45
(0.66)*** (0.64)*** (0.85)***-1.42 -1.15 -0.74
(0.36)*** (0.38)*** (0.46)-0.04 0 0.02(0.04) (0.04) (0.04)-0.37 -0.96 -0.86(0.54) (0.53)* (0.56)-0.79 0.44 -1.83(0.96) (0.92) (1.21)-1.7 -1.36 0.16
(0.72)** (0.68)** (0.87)-1.99 -1.36 -2.35
(0.70)*** (0.97) (1.33)*2.34 3.58 7.88
(1.23)* (1.21)*** (1.37)***0.01 1.65 7.54
(1.41) (1.41) (1.64)***-6.73
(2.49)***Constant 51.14 22.81 15.59
(9.80)*** (2.45)*** (2.82)***
Observations 41725 43130 43142R-squared 0.09 0.09 0.18
* significant at 10%; ** significant at 5%; *** significant at 1%
Regressions include a full set of controls for the day of week of the visit, for the round of the visit, and for the time of day of the visit relative to the school day (see full paper for details).
Mid-day meal exists
Government public school
Private aided school
Log of state per-capita income
The binary absence variable has been multiplied by 100 to allow the coefficients to be read as percentage changes.
Notes: Robust standard errors clustered at the school level are given in parentheses.
Duration of current posting (years)
Teacher's place of origin is village/town where school is located
PTA exists
PTA met at least once in last 3 months
Multigrade teaching
Pupil-teacher ratio
School has been inspected in past 3 months
Mean parental education of 4th-grade children (1-7 scale)
Recognition/award scheme exists in district
School infrastructure index (1 to 5)
Rural school
Distance to nearest paved road (1 to 5)
Belongs to a teachers union
Married
Has children of school age (5-14)
Paid regularly (1=Yes)
Has a college degree
Attended training in last 6 months
Head teacher
Contract teacher
TABLE 6: OLS estimates of teacher absence Dependent variable (visit-level observation):
Gender (1 = Male)
Age
23
Higher teacher salaries do not seem to be associated with lower absence No direct evidence but salaries increase with education, experience, rank Absence increases with: college degree older than 40 head teacher
Teacher absence is lower in schools with better infrastructure Toilets for teachers Electricity connection Library Covered classrooms Non-mud floors
24
Notes from “Addressing Absence” (Banerjee and Duflo) Obvious method to fight teacher absence is to monitor more intensively External control need not always be about monetary incentives Most common type control: someone in the institutional hierarchy (headmaster of a school) is
giventask of keeping an eye on teacher and penalizing absences Alternative method use some impersonal method, such as a camera, for recording absence An NGO in rural India experimented with a camera
25
In this area absence rate was 44% Most schools are one-teacher schools: when the teacher is absent children just go back home and lose entire day of schooling
120 schools were selected to participate in this study 60 randomly selected schools (treatment schools) NGO gave the teacher a camera with instructions to take a picture of himself /herself every day at opening time and at closing time
Figure 1
Figure 1
Figure 2: Impact of the CamerasNumber of Schools Found Open Times in Treatment and
Comparison schools(out of 13 visits)
0
2
4
6
8
10
12
0 1 2 3 4 5 6 7 8 9 10 11 12 13
Attendance Frequency (x)
Num
ber o
f Tea
cher
s pr
esen
t exa
ctly
x ti
mes
Treatment Control
27
Experimental Design
Teachers received a bonus as a function of the number of days they actually attended Teachers received a salary of 1,000 Rs. monthly if they were present at least 21 days in a month Each additional day carried a bonus of 50 Rs. up to a maximum of 1,300 per month. Each day missed carried a penalty of 50 Rs. Therefore the way the bonus was set up the average teacher’s salary remained 1,000 Rs. per month which was what teachers were paid in the remaining 60 schools (the comparison schools).
28
The program resulted in an immediate improvement in teacher attendance The absence rate of teachers was cut by one half Given the structure of the payment, the average salary in the treatment schools ended up matching almost exactly the average salary in the comparison schools The incentives were therefore effective without an increase in teachers’ net pay
Treatment Control Difference(1) (2) (3)
School Open 0.66 0.64 0.02(0.11)
41 39 80
Number of Students Present 17.71 15.92 1.78(2.31)
27 25 52
Teacher Test Scores 34.99 33.62 1.37(2.01)
53 56 109
Teacher Highest Grade Completed 10.21 9.80 0.41(0.46)
57 54 111
0.83 0.84 0.00(0.09)
27 25 52
0.78 0.72 0.06(0.12)
27 25 52
Blackboards Utilized 0.85 0.89 -0.04(0.11)
20 19 39
Infrastructure Index 3.39 3.20 0.19(0.30)
57 55 112
Fstat(1,110) 1.21p-value (0.27)
Table 1: Is School Quality Similar in Treatment and Control Groups Prior to Program?
E. School Infrastructure
Percent of Teachers Interacting with Students
Percentage of Children Sitting Within Classroom
Notes: (1) Teacher Performance Measures from Random Checks only includes schools that were open during the random check. (2) Infrastructure Index: 1-5 points, with one point given if the following school attribute is sufficient: Space for Children to Play, Physical Space for Children in Room, Lighting, Library, Floor Mats
A. Teacher Attendance
B. Student Participation (Random Check)
C. Teacher Qualifications
D. Teacher Performance Measures (Random Check)
Treatment Control Difference Treatment Control Difference(1) (2) (3) (4) (5) (6)
Took Written Exam 0.17 0.19 -0.02(0.04)
1136 1094 2230
Math Score on Oral Exam 7.82 8.12 -0.30 -0.10 0.00 -0.10(0.27) (0.09)
940 888 1828 940 888 1828
Language Score on Oral Exam 3.63 3.74 -0.10 -0.03 0.00 -0.03(0.30) (0.08)
940 888 1828 940 888 1828
Total Score on Oral Exam 11.44 11.95 -0.51 -0.08 0.00 -0.08(0.48) (0.07)
940 888 1828 940 888 1828
Math Score on Written Exam 8.62 7.98 0.64 0.23 0.00 0.23(0.51) (0.18)
196 206 402 196 206 402
Language Score on Written Exam 3.62 3.44 0.18 0.08 0.00 0.08(0.46) (0.20)
196 206 402 196 206 402
Total Score on Written Exam 12.17 11.41 0.76 0.16 0.00 0.16(0.90) (0.19)
196 206 402 196 206 402
Levels Normalized by ControlTable 2: Are Students Similar Prior To Program?
Notes: (1) Children who could write were given a written exam. Children who could not write were given an oral exam. (2) Standard errors are clustered by school.
A. Can the Child Write?
B. Oral Exam
C. Written Exam
Treatment Control Diff Until Mid-Test Mid to Post Test After Post Test(1) (2) (3) (4) (5) (6)
0.79 0.58 0.21 0.20 0.20 0.23(0.03) (0.04) (0.04) (0.04)
1575 1496 3071 882 660 1529
0.78 0.63 0.15 0.15 0.15 0.14(0.04) (0.05) (0.05) (0.06)
843 702 1545 423 327 795
0.78 0.53 0.24 0.21 0.14 0.32(0.04) (0.05) (0.06) (0.06)
625 757 1382 412 300 670
Figure 3: Impact of the Cameras(out of at least 25 visits)
Notes: (1) Child learning levels were assessed in a mid-test (April 2004) and a post-test (November 2004). After the post-test, the "official" evaluation period was ended. Random checks continued in both the treatment and control schools. (2) Standard errors are clustered by school. (3) Panels B and C only include the 109 schools where teacher tests were available.
Table 3: Teacher AttendanceSept 2003-Feb 2006 Difference Between Treatment and Control Schools
A. All Teachers
B. Teachers with Above Median Test Scores
C. Teachers with Below Median Test Scores
0
2
4
6
8
1 4 7 10 13 16 19 22 25Atte ndance Fre que ncy
Num
ber
of T
each
ers p
rese
nt e
xact
ly x
tim
es
Treatment
Control
30
In another experiment: in treatment schools, if the headmasters marked the preschool
teachers present a sufficient number of times for the teacher to receive a prize (a bicycle).
This experiment had no effect Absence rates were not reduced This outcome suggests that when human judgment is involved in a system where rules are often bent incentives may easily be perverted
31
Notes on “Remedying education: Evidence from two randomized experiments in India” (Banerjee, Cole, Duflo, and Linden)
Millennium Development Goals – primary education should be universal Dismal quality of educational services that developing countries offer the poor 2005 – India-wide survey: 44% children aged 7-12 cannot read 50% cannot subtract even though most are enrolled in school
32
In these conditions: policies that promote enrollment may not promote learning recent evidence suggests this to be the case
students do not learn anything in additional days they spend at
school We know very little about how to improve the quality of schools in a cost-effective way
33
Number of randomized experiments have demonstrated: spending more on resources such as textbooks, flip charts, or
additional teachers has had no impact on childrens’ test scores advocated reforms designed to change incentives faced by
teachers, parents and children We do not know enough to give up on inputs need to be more school-specific inputs targeted at weaker children may be effective
34
Consider two randomized experiments in urban India:
(1) remedial education program targeted weakest children
children taken out of regular classroom to work with young
women (from the community who has completed secondary school) for 2 hours per day (school is at least 4 hours per day)
35
(2) computer assisted learning program addressed to all children but adapted to each child’s current level
of achievement
children offered two hours of shared computer time per week play games that involve solving math problems at appropriate
difficulty level
36
Evaluation design Half schools given tutors for grade 3 (group A) other half for grade 4 (group B) in 2001-2002 (year 1). Program continued in 2002-2003 (year 2) – schools in group A were now assigned tutors in grade 4. Schools in group B in year 1 received assistance for grade 3 in year 2. In each year children in grade three in schools that received the program for
grade four = comparison group children that receive program for grade 3 = treatment group
Computer assisted learning: ½ schools given computers in first year (treatment), other half next year (comparison).
TABLE ISAMPLE DESIGN AND TIME LINE
Year 1 (2001–2002) Year 2 (2002–2003) Year 3 (2003–2004)
Grade 3 Grade 4 Grade 3 Grade 4 Grade 3 Grade 4
(1) (2) (3) (4) (5) (6)
Panel A: VadodaraBalsakhi
Group A (5,264 students in 49 schools in year 1;6,071 students in 61 schools in year 2) Balsakhi No balsakhi No balsakhi Balsakhi No balsakhi No balsakhi
Group B (4,934 students in 49 schools in year 1;6,344 students in 61 schools in year 2) No balsakhi Balsakhi Balsakhi No balsakhi No balsakhi No balsakhi
Computer-Assisted Learning (CAL)Group A1B1 (2,850 students in 55 schools in year
2; 2,814 students in 55 schools in year 3) No CAL No CAL No CAL CAL No CAL No CALGroup A2B2 (3,095 students in 56 schools in year
2; 3,131 students in 56 schools in year 3) No CAL No CAL No CAL No Cal No CAL CALPanel B: Mumbai
BalsakhiGroup C (2,592 students in 32 schools in year 1;
5,755 students in 38 schools in year 2) Balsakhi No balsakhi No balsakhi Balsakhi No balsakhi No balsakhiGroup D (2,182 students in 35 schools year 1;
4,990 students in 39 schools in year 2) No balsakhi No balsakhi Balsakhi No balsakhi No balsakhi No balsakhi
Notes: This table displays the assignment to schools in various treatment groups in the three years of the evaluation.Group A1B1 and A2B2 were constituted by randomly assigning half the schools in Group A and half the schools in Group B to the Group A1B1 and the remaining schools to
the Group A2B2.Schools assigned to Group A (resp. B) in 2001–2002 remained in Group A (resp. B) in 2002–2003. Twelve new schools were brought in the study and assigned randomly to Groups
A and B.Schools assigned to Group C (resp. D) in 2001–2002 remained in Group C (resp. D) in 2002–2003. Ten new schools were brought in the study and assigned randomly to Groups
C and D.
1242Q
UA
RT
ER
LY
JO
UR
NA
LO
FE
CO
NO
MIC
S
38
Key Outcomes: Learning measured using annual pre-tests given during first few weeks of school and post-tests given at end of term
39
Short-term effects Simple differences between post-test scores tutor program successful
TABLE IITEST SCORE SUMMARY STATISTICS FOR BALSAKHI AND CAL PROGRAMS
Pretest Posttest
Treatment Comparison Difference Treatment Comparison Difference
(1) (2) (3) (4) (5) (6)
A. Balsakhi: VadodaraYear 1 (grades 3 and 4)
Math �0.007 0.000 �0.007 0.348 0.171 0.177(0.059) (0.070)
Language 0.025 0.000 0.025 0.794 0.667 0.127(0.061) (0.076)
Year 2 (grades 3 and 4)Math 0.046 0.000 0.046 1.447 1.046 0.401
(0.053) (0.078)Language 0.055 0.000 0.055 1.081 0.797 0.285
(0.058) (0.071)B. Balsakhi: Mumbai
Year 1 (grade 3)Math 0.002 0.000 0.002 0.383 0.227 0.156
(0.108) (0.126)Language 0.100 0.000 0.100 0.359 0.210 0.149
(0.108) (0.102)Year 2 (grades 3 and 4)
Math �0.005 0.000 �0.005 1.237 1.034 0.203(0.058) (0.107)
Language 0.056 0.000 0.056 0.761 0.686 0.075(0.054) (0.061)
1246Q
UA
RT
ER
LY
JO
UR
NA
LO
FE
CO
NO
MIC
S
41
Math competencies: Vadodara: 19.5% third grade children pass grade one test in math
competencies Mumbai: 33.7%
Verbal competencies:
Vadodara: 20.9%
Mumbai: 83.7%
41
Because test scores have a strong persistent component precision of the estimated program effect can be increased
substantially by controlling for child’s pre-test score Table III – controls for pre-test score of child, grade and school –
find substantial treatment effect
42
Longer run impacts Must determine whether changes generated by interventions persist over time and last beyond period in which intervention is administered Compare effect of being exposed one or two years if effects are durable should be cumulative
Last two rows of Table III: present estimate of the impact of two years of exposure to the program These estimates are the difference between the year 1 (2001-2002) pre-test score and year 2 (2002-2003) post-test score for students
TABLE IIIESTIMATES OF THE IMPACT OF THE BALSAKHI PROGRAM, BY CITY AND SAMPLE
Number ofobservations
Dependent variable: test scoreimprovement
(posttest � pretest)
Math Language Total
(1) (2) (3) (4)
A: Pooling grades andlocations
Mumbai and Vadodaratogether year 1 12,855 0.182 0.076 0.138
(0.046) (0.056) (0.047)Mumbai and Vadodara
together year 2 21,936 0.353 0.187 0.284(0.069) (0.050) (0.060)
B: Pooling both gradesVadodara year 1 8,426 0.189 0.109 0.161
(0.057) (0.057) (0.057)Vadodara year 2 11,950 0.371 0.246 0.331
(0.073) (0.061) (0.070)Mumbai year 1
(grade 3 only) 4,429 0.161 0.086 0.127(0.075) (0.066) (0.067)
Mumbai year 2 9,986 0.324 0.069 0.188(0.145) (0.081) (0.112)
C: Grade 3Vadodara year 1 4,230 0.179 0.102 0.152
(0.086) (0.085) (0.085)Vadodara year 2 5,819 0.418 0.233 0.354
(0.107) (0.089) (0.100)D: Grade 4
Vadodara year 1 4,196 0.190 0.114 0.166(0.072) (0.076) (0.073)
Vadodara year 2 6,131 0.307 0.240 0.289(0.078) (0.068) (0.074)
E: Two year (2001–2003)Mumbai pretest year 1 to
posttest year 2 3,188 0.612 0.185 0.407(0.141) (0.094) (0.106)
Vadodara pretest year 1 toposttest year 2 3,425 0.282 0.181 0.250
(0.094) (0.079) (0.088)
Notes: This table reports the impact of the Balsakhi Program, for different groups and years. Each cellrepresents a separate regression of test score improvement on a dummy for treatment school, controlling for initialpretest score. Standard errors, clustered at the school-grade level, are given in parentheses. Estimates, whichinclude Mumbai year 2, use intention to treat as an instrument for treatment. Normalized test score gain is thedifference between posttest and pretest for Panels A–D and the difference between posttest in year 2 and pretestin year 1 for panel E. The total score is the sum of the normalized math and language scores.
1250 QUARTERLY JOURNAL OF ECONOMICS
45
Investigate whether the program had effects which lasted beyond the years during which the children were exposed Tested children in grade 4 and 5 at the end of year 3 (2003-2004) when the program had ended
TABLE ISAMPLE DESIGN AND TIME LINE
Year 1 (2001–2002) Year 2 (2002–2003) Year 3 (2003–2004)
Grade 3 Grade 4 Grade 3 Grade 4 Grade 3 Grade 4
(1) (2) (3) (4) (5) (6)
Panel A: VadodaraBalsakhi
Group A (5,264 students in 49 schools in year 1;6,071 students in 61 schools in year 2) Balsakhi No balsakhi No balsakhi Balsakhi No balsakhi No balsakhi
Group B (4,934 students in 49 schools in year 1;6,344 students in 61 schools in year 2) No balsakhi Balsakhi Balsakhi No balsakhi No balsakhi No balsakhi
Computer-Assisted Learning (CAL)Group A1B1 (2,850 students in 55 schools in year
2; 2,814 students in 55 schools in year 3) No CAL No CAL No CAL CAL No CAL No CALGroup A2B2 (3,095 students in 56 schools in year
2; 3,131 students in 56 schools in year 3) No CAL No CAL No CAL No Cal No CAL CALPanel B: Mumbai
BalsakhiGroup C (2,592 students in 32 schools in year 1;
5,755 students in 38 schools in year 2) Balsakhi No balsakhi No balsakhi Balsakhi No balsakhi No balsakhiGroup D (2,182 students in 35 schools year 1;
4,990 students in 39 schools in year 2) No balsakhi No balsakhi Balsakhi No balsakhi No balsakhi No balsakhi
Notes: This table displays the assignment to schools in various treatment groups in the three years of the evaluation.Group A1B1 and A2B2 were constituted by randomly assigning half the schools in Group A and half the schools in Group B to the Group A1B1 and the remaining schools to
the Group A2B2.Schools assigned to Group A (resp. B) in 2001–2002 remained in Group A (resp. B) in 2002–2003. Twelve new schools were brought in the study and assigned randomly to Groups
A and B.Schools assigned to Group C (resp. D) in 2001–2002 remained in Group C (resp. D) in 2002–2003. Ten new schools were brought in the study and assigned randomly to Groups
C and D.
1242Q
UA
RT
ER
LY
JO
UR
NA
LO
FE
CO
NO
MIC
S
already 0.13 higher in the treatment group in year 3 (as shown incolumn (3)).
Table IV corrects for this initial difference by estimating (1),where the treatment is the participation of the school in the CALprogram. The CAL program has a strong effect on math scores(0.35 standard deviations in the first year (year 2) and 0.47standard deviations in the second year (year 3)). It has no dis-cernible impact on language scores (the point estimates are al-ways very close to zero). This is not surprising, since the softwaretargeted exclusively math skills, although some spillover effectson language skills could have occurred (for example, because theprogram increased attendance, or because the children got prac-tice in reading instructions, or because the teachers had reallo-cated time away from math to reading). The effect on the sum oflanguage and math test scores is 0.21 standard deviations in year
TABLE IVIMPACT OF THE CAL PROGRAM, BY YEAR
Number ofobservations
Dependent variable:Test score improvement
(posttest � pretest)
Math Language Total
(1) (2) (3) (4)
A: Effect of the CAL programVadodara both years 11,255 0.394 �0.025 0.191
(0.074) (0.082) (0.083)Vadodara Year 2 5,732 0.347 0.013 0.208
(0.076) (0.069) (0.074)Vadodara Year 3 5,523 0.475 �0.005 0.225
(0.068) (0.042) (0.051)B: Balsakhi and CAL program: Main effects and interactions (Vadodara, Year 2)
CAL 5,732 0.408 0.017 0.242(0.087) (0.084) (0.087)
Balsakhi — 0.371 0.229 0.315(0.112) (0.104) (0.112)
CAL � balsakhi — �0.144 �0.020 �0.086(0.141) (0.134) (0.141)
This table reports the impact of the CAL program. In Panel A, each cell represents a separate regression,of test score gain on a dummy for treatment school, controlling for initial pretest score. In Panel B, eachcolumn represents a regression, of test score improvement on a dummy for the CAL program, a dummy forthe Balsakhi program, and an interaction term, as well as a control for initial pretest score. Standard errors,clustered at the school-grade level, are given in parentheses. Normalized test score improvement is thedifference between posttest and pretest. The total score is the sum of the normalized math and languagescores.
1252 QUARTERLY JOURNAL OF ECONOMICS
TABLE VSHORT- AND LONGER-RUN IMPACTS OF PROGRAMS, BY INITIAL PRETEST SCORE
Sample
Probability ofassignmentto balsakhi
Program effect in year 2 Persistence of program effect
Math Language TotalNumber of
observations Math Language TotalNumber of
observations
(1) (2) (3) (4) (5) (6) (7) (8) (9)
Panel A: Balsakhi, 2002–2003All children 0.313 0.371 0.246 0.331 11,950 0.053 0.033 0.040 9,925
(0.073) (0.061) (0.070) (0.047) (0.041) (0.041)Bottom third 0.446 0.469 0.317 0.425 4,053 0.096 0.097 0.103 3,356
(0.088) (0.074) (0.084) (0.045) (0.038) (0.040)Middle third 0.341 0.374 0.240 0.339 3,874 0.021 �0.024 0.001 3,226
(0.082) (0.069) (0.080) (0.056) (0.054) (0.052)Top third 0.162 0.229 0.174 0.216 4,023 0.015 0.006 0.009 3,343
(0.076) (0.076) (0.077) (0.069) (0.062) (0.061)Panel B: CAL, 2002–2003
All children — 0.347 0.013 0.208 5,732 0.092 �0.072 0.008 4,688(0.076) (0.069) (0.074) (0.045) (0.048) (0.045)
Bottom third — 0.425 0.086 0.278 1,962 0.107 0.004 0.046 1,586(0.106) (0.089) (0.102) (0.046) (0.047) (0.046)
Middle third — 0.316 0.005 0.183 1,844 0.085 �0.105 �0.015 1,511(0.081) (0.081) (0.082) (0.055) (0.069) (0.058)
Top third — 0.266 �0.033 0.146 1,926 0.073 �0.105 �0.013 1,591(0.073) (0.081) (0.078) (0.072) (0.064) (0.068)
This table reports the effects of the Balsakhi and CAL Programs over the short- and medium-term, according to the child’s position in the initial pretest score distribution. Column(1) reports the probability of actually being taught by the balsakhi, conditional on being in a treatment school. Each cell in columns (2)–(8) represents a separate regression of testscore gain on a dummy for treatment, controlling for initial pretest score. In Panel A, intention to treat is used as an instrument for treatment. Columns (2)–(4) give the one-yearprogram effect, estimated as the difference in normalized test score between the posttest and pretest in year 2 (2002–2003). Columns (6)–(8) give the cumulative effect of each programone year after both interventions had stopped. The dependent variable for these regressions is the difference between an end of year test in year 3 (2003–2004), and the pretest inyear 2 (2002–2003). Standard errors, clustered at the school-grade level, are given in parentheses.
1254Q
UA
RT
ER
LY
JO
UR
NA
LO
FE
CO
NO
MIC
S
48
Using difference between 2004 and 2002 Average effect becomes insignificant Not case for weaker students – effect remains significant
49
Find that both programs has a substantial positive effect on children’s academic achievement at least in the short run
Find remedial program impacts weaker students no average effects (consistent with previous literature)
50
Remedial program is very low cost and easily replicated. Relies on local personnel who are trained for a short period earn 10-15 dollars/month.
Curriculum and pedagogy are simple and standardized rapid turnover among tutors.
Use whatever space available (free classroom, playground, hallways)
51
Computer assisted learning Government delivered 4 computers to each government run primary school in the city. Idea of using computers to remedy shortage of qualified teachers is very popular in Indian policy circles. Evidence on impact of computers in developed countries is not encouraging
Recommended