Upload
august-flynn
View
223
Download
0
Tags:
Embed Size (px)
Citation preview
1
Ministry of Finance and Ministry of Samurdhi
Welfare Benefits Board
The Targeting Formula: Analysis Using Pilot Data
Welfare Workshop
Colombo, 16-17 November 2003
2
Developing criteria for identifying welfare beneficiaries
The idea of proxy means test formula (PMTF): to find a weighted combination of “proxy” variables/indicators that together identify or predict whether a household is poor or not
The PMTF assigns a “score” to every household, based on information collected from the household for all variables that are included in the formula
Choice of proxy variables must balance between 3 criteria: able to identify the poor with some accuracy; easily observable and measurable; cannot be manipulated easily by household
PMTF developed by regression analysis using household surveys Analysis is used to identify appropriate set of variables and weights Alternative models from Sri Lanka Integrated Survey (SLIS, 1999-2001),
and Consumer Finance Socio-Economic Survey (1996-97)
3
The Proxy Means Test Formula using SLIS
Notes:1) All scores are derived from regressions of (log of) per capita consumption expenditure on a set of variables2) The score for each variable is its coefficient in the regression, multiplied by 100, and rounded to the nearest integer3) Lower the score, poorer the household is considered to be3) The aggregate score for each household is calculated as constant +/- the weight on each variable. For each 0-1 variable, multiply the score by 1 if true for household, by 0 if not true. For each continuous variable (denoted by *), multiply the score by the value of the variable for the household)
Variables Weights
Community characteristics Public/Private bank in community 8 Divisional Secretariat in community 9
Household assets Car/van 40 Cooker (kerosene/gas/electric) 17
Bicycle/Tricycle 4 Fan 11 Refrigerator 12 Motorcycle/Scooter 8 Radio/CD/Cassette player 4 Sewing Machine 7 Tractor 15 TV/Video player 8
Land and livestock Cultivable land owned by household : 1<Acres<=2 7
2<Acres<=4 8 Acres>4 16
Livestock (any) 8 Household head
Not a female who is widowed/separated/divorced 5 Age: 70-79 -5 80 and above -13 Education: Passed OL or Grade 11 7
Passed AL/GAQ/GSQ 10 Has Degree/PG/Diploma 16
Work: Salaried employment or in business 5 Household demographics
Household size: 3-4 members -23 5-6 members -39 7-8 members -52 >8 members -59
All children age 5-16 attend school 6 Housing characteristics
Dwelling owned by hhold 4 Fuel for cooking: Gas/electricity 13 Toilet: Private and flush type 16 No. of Rooms (excl. kitchen/bath) per hhold member* 16 Walls: Not cabook/mud/plank/cadjan 6
Constant 707
4
Validating the formula through a pilot
Pilot targeting exercise conducted over the last few months
Information collected on variables in the 2 formulas through application forms in selected GN divisions
Analysis of data collected to test how the predictions from the formula measure up against real data from an application process
The Pilot Applicants
District Name No. of GN Divisions
No. of Households
COLOMBO 3 1,739 KALUTARA 5 845 KANDY 4 862 NUWARA - ELIYA 14 4,104 GALLE 23 2,652 MATARA 6 1,680 HAMBANTOTA 21 2,973 VAVUNIYA 6 1,813 TRINCOMALEE 17 5,063 BADULLA 4 695 MONARAGALA 5 2,282 RATNAPURA 6 1,995 Total 114 26,703
5
Key questions analyzed using pilot data
What was the rate of response to the pilot application, and how well has the application form worked?
How does the coverage (% of population identified as beneficiaries) of the formula for different cutoff scores measure up to what was simulated from survey data?
What are the implications of coverage figures for the choice of cutoff points for eligibility ?
What can we say about coverage of groups that are likely to be especially vulnerable ?
How do the results compare across different regions of the country?
What implications can be drawn for field work or data collection efforts ?
6
Response to pilot targeting exercise and implications
58% (in terms of population) response rate for all pilot areas: 55% in non-NE areas, 69% in the 2 NE districts
What we expect: people not interested in program would not fill up the form Concern: did everyone who would have been interested fill up the form ?
Special concern: those who have been “missed” by the program so far, did they fill the form?
What about those who filled, but information is incomplete? Because of non-response, all population coverage estimates will have to be calculated
assuming that all who did not apply would not have qualified for the program
* denotes estimates using average household size from census information (for non-Northeast districts), since population figures for Northeast districts are not available
Census No. of Applicants Completed applications
Population Households Population Households Population Households 199,151* 48,501 115,580 26,703 107,418 24,660 All
Districts (58.0%) (55.1%) (53.9%) (50.8%)
156,583 38,134 86,309 19,827 80,154 18,299 Non-NE Districts (55.1%) (52.0%) (51.2%) (48.0%)
42,568* 10,367 29,271 6,876 27,264 6,361 NE Districts (68.8%) (66.3%) (64.0%) (61.4%)
7
How serious is the problem of incomplete information for the purpose of analysis?
Age of household head is the most frequently missing variable No evidence that missing information differentially affects certain TYPES household,
which would have caused a bias in results due to missing variables
There are some variables that were imperfectly measured: cultivable land, type of toilet, homeless
Variables # households
missing Assets 178 Housing 100 Rooms per member 419 Single female household head 630 Head’s age 950 Head's education 336 Head’s main occupation 790 Cultivable land 320 Household size 7
# households Any missing variable 2166
No missing variable 24660 Homeless households 354
8
Cumulative distribution of pilot scores from SLIS model:Comparison with results from SLIS sample
score
Pilot SLIS SLIS_actual
600 650 700 750 800 850 900 950 1000
0
.2
.4
.6
.8
1
The pilot sample is poorer than the SLIS national sample
Expected since pilot sample contains only those who are interested in the program
SLIS_actual is log per capita expenditure times 100, SLIS refers to predicted score from SLIS data, and Pilot score is calculated using the SLIS model on all pilot sample cases for which the model score could be calculated
SLIS Actual
SLIS predictedPilot sample
9
Applying SLIS model to pilot: program coverage
% covered in pilot sample for every cutoff point is higher than that predicted from SLIS –since pilot applicants are poorer
Important: % covered among total population in pilot districts for every cutoff point is close to that predicted from SLIS The difference between predicted and pilot population coverage becomes higher as
cutoff point is raised This difference also suggests that all the poor households probably did not apply for
the pilot program
Coverage (%) : Non NE Coverage (%) : Entire
pilot sample Percentile of
actual per capita
consumption (SLIS)
Cutoff scores from
SLIS model
Coverage (%) predicted from SLIS Pilot
Sample Pilot
Population Pilot
Sample Pilot
Population
20th 695 12.0 20.4 11.2 20.3 11.8 25th 703 19.6 33.0 18.1 32.6 18.9
30th 709 26.3 42.8 23.6 42.5 24.6 35th 715 33.7 52.8 29.1 52.7 30.5 40th 721 41.9 62.6 34.5 62.4 36.2
10
Cutoff points and coverage
Coverage is higher for the two NE districts: a higher proportion of applicants in the total population brings the coverage closer to what is predicted
Choice of a cutoff point: recognize that the proportion of applicants in the population is expected to rise in the actual program, which will raise the % of population covered for a given cutoff E.g. with cutoff at a score of 709, pilot population coverage of 24.6% is
lower than predicted coverage of 26.3% But coverage will increase if strong efforts are made to have all the poor
apply to the program Thus 24.6% is lower bound of expected coverage; actual coverage should
be expected to be 2-3 percentage points higher
Last but not the least: recall the inherent tradeoff in choice of cutoff due to the formula being unable to identify the poor “perfectly” Higher the cutoff, higher the rate of “leakage” to non-poor Lower the cutoff, lower the rate of coverage among the poor
11
Coverage among specially “vulnerable” groups on applying the formula: high in pilot sample
0
0.2
0.4
0.6
20th 25th 30th 35th 40th
Households with DISABLED heads
0
0.2
0.4
0.6
20th 25th 30th 35th 40th
Households with SINGLE FEMALE heads
0
0.2
0.4
0.6
20th 25th 30th 35th 40th
Households with AGE 70+ heads
0
0.2
0.4
0.6
20th 25th 30th 35th 40th
Households with cultivable land<1 acre
SLIS actual SLIS predicted Pilot sample
Cutoff scores
Coverag
e
12
Coverage among other vulnerable groups also high in pilot sample…..
0
0.2
0.4
0.6
0.8
20th 25th 30th 35th 40th
Households with 7 or more members
0
0.2
0.4
0.6
0.8
20th 25th 30th 35th 40th
Number of CHILDREN (age 0-16)
As is coverage among large households & children…..
0
0.2
0.4
0.6
20th 25th 30th 35th 40th
Households who do not OWN house
0
0.2
0.4
0.6
20th 25th 30th 35th 40th
Households who do not OWN livestock
SLIS actual SLIS predicted Pilot sampleCutoff scores
Coverag
e
13
Coverage of the poor in specially vulnerable groups
Results from household survey (SLIS) show little reason for concern Undercoverage (proportion of poor “missed” by formula) is less for all
vulnerable groups than among the poor in general, for all cutoff points
Undercoverage Rates using SLIS formula Population of Poor Households with
Cutoff Overall
population of poor
disabled head
female single head
land owned<1 acre
not own house
age 70+ head
household size 7+
20th 0.64 0.62 0.56 0.62 0.52 0.54 0.40 25th 0.52 0.51 0.50 0.52 0.43 0.42 0.28 30th 0.43 0.40 0.41 0.41 0.35 0.33 0.20 35th 0.37 0.32 0.35 0.34 0.28 0.32 0.17
40th 0.28 0.26 0.27 0.26 0.21 0.25 0.12
Note: Calculations based on SLIS sample – representative for Sri Lanka, excluding the North & East
14
Coverage of vulnerable households who are in the poorest 30% of the population
When cutoff = 709 (30th percentile of actual consumption)(Results using SLIS data)
Disabled head of household Single female head of household
Cultivable land owned < 1 acre Household head's age > 70 years
Covered w hencutoff of 709 (30thpctile)
Not covered
15
Comparison between North & East districts and the rest
Significantly higher response rate to pilot application in N & E districts Leads to higher % of population covered (for any cutoff) in NE districts
than in non-NE districts
Most household characteristics in N&E pilot districts – defined by the variables in the formula – are not significantly different from those in the other pilot districts 3.1% of households in NE districts have a disabled head, compared to 4.8%
of households in non-NE; 14.0% of households in NE have a single female head, compared to 12.6% of those in non-NE
Coverage (%) in pilot sample
Coverage (%) in population in pilot
districts
Percentile of actual per capita
consumption (SLIS)
Cutoff score from SLIS
model
Coverage (%) predicted from SLIS
NE non NE NE non NE 20th 695 12.0 19.9 20.4 13.8 11.2 25th 703 19.6 31.2 33.0 22.1 18.1 30th 709 26.3 40.9 42.9 28.9 23.6 35th 715 33.7 51.3 53.0 36.4 29.1 40th 721 41.9 61.1 62.8 43.0 34.5
16
Comparison with groups of existing program beneficiaries
Distribution of Samurdhi beneficiaries in pilot somewhat correlated with SLIS formula scores in N&E; little correlation between the two in non-N&E districts
If cutoff was set at the 30th percentile, to benefit about 25% of pilot population
63% of beneficiaries are current Samurdhi recipients, 6% benefit from other welfare programs (but not Samurdhi), and 31% currently receive no benefits
Eligible households include: 47% of current Samurdhi recipients, 45% of those who receive other welfare benefits only, and 35% of those with no welfare benefits
North & East districts
0
10
20
30
40
50
60
70
1 2 3 4 5 6 7 8 9 10
Deciles by pilot scores using SLIS formula
O ther pilot districts
0
10
20
30
40
50
60
70
1 2 3 4 5 6 7 8 9 10
No welfare Samurdhi Other Welfare program
17
The homeless households
All homeless households should be in the program automatically The formula is not applicable for them; thus all results so far have been derived after
taking the homeless out of the sample
Proportion of homeless in the pilot sample Does not change between NE and non-NE districts Significantly higher in rural areas Q: Have all the homeless been able to apply?
Average household size much lower for homeless households 24% of homeless households comprise of a single member, compared to 1% of those
living in homes 5% of homeless households comprise of a 6 or more members, compared to 37% of
those living in homes
House-hold size
% of Non homeless
% of homeless
1 1.0 24.3 2 5.0 23.4 3 13.2 26.3 4 21.1 11.1 5 22.8 9.7
6+ 36.9 5.2
Total 100 100
% of homeless households
# homeless households All Rural Urban
ALL districts 354 1.33 1.42 0.41 Non – N&E 266 1.43 1.47 0.42
N&E 88 1.28 1.30 0.00
18
Lessons for field work: 1) formula is meant to replace subjective judgment
A single variable in most cases is not a good predictor for poverty With cutoff set at the 30th percentile (score=709), 26% of selected
households has TV/VCR, 41% has electric light, 50% has brick/cement wall Thus critical to collect all information, and not form pre-judgments about a
household’s poverty based on inspection – let the formula do its job !
0
10
20
30
40
50
60
70
20th 25th 30th 35th 40th
Cutoff percentiles
% V
alu
es
TV/VCR Fridge okay quality w all
okay quality f loor electricity for light gas/electricity for cooking
19
Lessons for field work: 2) all variables are necessary, but some are especially critical to measure accurately
Ten most “sensitive” variables for identifying the poor using SLIS formula Sensitivity depends on the the variable’s weight and its distribution Coverage is most affected by household size, rooms per member, and whether
household head is female and single
0
5
10
15
20
25
hhsiz
e5-6
room
per m
embe
r
hhsiz
e7-8
no fe
mal sin
gle wall
hhsiz
e3-4
dwell
own
tv/vid
eo
all ki
ds a
ttend
hhsiz
e9+
Cutoff at 30th percentile of consn (score=709) Cutoff at 25th percentile of consn (score=703)
Change in coverage (absolute value) when one variable is dropped from score
0
5
10
15
20
25
hhsiz
e5-6
room
per m
embe
r
hhsiz
e7-8
no fe
mal
single wall
hhsiz
e3-4
dwell
own
tv/vid
eo
all ki
ds a
ttend
hhsiz
e9+
20
Implications for a broader safety net system
There remain certain categories of poor and vulnerable households that are less likely to be picked up by the formula Poor households who are small in size ……who have to support disabled members (not necessarily the head of the
household) ……who have suffered a recent reversal of fortune (e.g. death/disability of a
primary earning member, loss of crops) ……affected by civil strife/violence/dislocation, unless the effect is directly in
terms of loss of assets or disability to household head ……affected by illness of head/other household members
Since the above are relatively rarely occurring events or shocks, the formula cannot incorporate these as variables that explain poverty for the general population
Such cases underscore the need for a broader safety net system, incorporating programs that address specific needs and vulnerabilities