The Mismeasure of Poverty, The Mismeasure of Poverty, Happiness & Bias, and Happiness & Bias, and

RegressionRegression

In “The Mismeasure of Poverty,” Nicholas Eberstadt notes that the percentage of Americans living under the poverty rate has been stagnant for 30 years, suggesting that

antipoverty efforts have failed, and that life has not improved for the nation's poor.

The problem, Eberstadt continues, is that there are anomalies in the official statistics.

Why does the poverty rate tend to go up as unemployment falls? Why hasn't it budged while the amounts of money the government spends on the poor

have more than doubled in constant dollars?

Eberstadt goes back and looks at how the poverty rate is calculated, and finds two realities that are masked by our current statistics, one heartening, one disheartening.

The first reality is that people living under the poverty line are materially much better off than they were three decades ago.

- They live in much bigger homes.

- Three-quarters own at least one motor vehicle.

- They spend roughly twice as much as they report as income, and not because they are going into debt. (Networths have not declined.)

In general, poor people today live at about the same standard of living as middle-class people did in the 1960s.

Eberstadt: The Mismeasure of PovertyEberstadt: The Mismeasure of Poverty

Eberstadt: The Mismeasure of PovertyOn the other hand, they live with greater insecurity. In fact, relatively

few people live permanently in poverty (power-law distributions).

But nearly a third of the U.S. population dips into poverty from time to time. Eberstadt paints a picture of greater volatility at the bottom end of the income scale -- a different image from the one portrayed by the immobile statistics, with radically different policy implications (such as?).

Daniel Gilbert: “I’m O.K., You’re Biased”Daniel Gilbert: “I’m O.K., You’re Biased”Research shows that decision-makers don’t realize how easily Research shows that decision-makers don’t realize how easily

and often their objectivity is compromised. Much of what and often their objectivity is compromised. Much of what happens in the brain is not evident to the brain itself.happens in the brain is not evident to the brain itself.

And yet, if decision-makers are more biased than they realize, they are less biased than the rest of us suspect:

bathroom scalesbathroom scales saliva test strips for dangerous enzyme deficiencysaliva test strips for dangerous enzyme deficiency evaluating students’ intelligence by examining info. one piece at a time; evaluating students’ intelligence by examining info. one piece at a time;

when subjects liked the student, they kept turning cards searching for one when subjects liked the student, they kept turning cards searching for one good piece of info; when they disliked the students, they turned over a few good piece of info; when they disliked the students, they turned over a few cards, shrugged and quitcards, shrugged and quit

The majority of people claim to be less biased than the majority of people:The majority of people claim to be less biased than the majority of people:

84% of medical resident claimed that their colleagues were influenced by 84% of medical resident claimed that their colleagues were influenced by gifts from Rx drug companies, but only 16% thought they were similarly gifts from Rx drug companies, but only 16% thought they were similarly influenced.influenced.

Daniel Gilbert: “I’m O.K., You’re Biased”Daniel Gilbert: “I’m O.K., You’re Biased”Because the brain cannot see itself fooling itself, the only

reliable method for avoiding bias is to avoid the situations that produce it.

Doctors should refuse to accept gifts from those who supply drugs to their patients, when justices refuse to hear cases involving those with whom they share familial ties and when chief executives refuse to let their compensation be determined by those beholden to them, then everyone sleeps well.

Professors should grade “blind” as often as possible.

What about college admissions?

Stock Market Volatility & Regression to the MeanStock Market Volatility & Regression to the Mean

Stock Market Volatility & Regression to the MeanStock Market Volatility & Regression to the Mean

RegressionAllows you to go beyond correlation and

to predict how much, say, a dependent variable Y (e.g., income) changes if independent variable X (e.g., years of education) changes by 1 unit.

RegressionTwo kinds of regression analysis:

- bivariate (income by years of education)

- multivariate (or multiple regression)

controls for other independent variables

(income by education, age, race, gender, etc)

Bivariate Regression

Y = a + bx

Y = dependent variable (what you basically want

to predict/determine)

x = independent variable

a = Y-intercept (the point at which the regression

line intercepts or crosses the Y-axis: anchor)

b = slope of the regression line (the regression

coefficient)

Bivariate RegressionExample: Amex Business Service does routine accounting as one of its basic job offerings. Its rate is $20/per hour plus a $25 floppy disk charge.

The total cost to a customer depends, of course, on the number of hours it takes to complete the job.

So the total cost, Y, of a job that takes x hours is . . .

Y = $25 + 20x

Time & Cost: 5 hours ($125), 7.5 hours ($175),

15 hours ($325), 20 hours ($425), 22.5 hours ($475)

Bivariate Regression

5.00 10.00 15.00 20.00

Time in Hours

200.00

300.00

400.00

Co

st (

$)

Cost ($) = 25.00 + 20.00 * hoursR-Square = 1.00

Bivariate RegressionY = a + bx + e (error term)

Real-life applications in the natural and social sciences are seldom this straight-forward.

Instead, what we usually have are sets of scores for various independent and dependent variables.

We display them on a scatterplot and use inferential statistics to determine both: the [1] slope (the regression coefficient) and, then, [2] predictions for other independent variables.

For example, consider the education levels and income of 32 employees ...

Bivariate Regression

5.00 10.00 15.00 20.00

Years

8000.00

12000.00

16000.00

20000.00In

com

e ($

)

Income ($) = 5077.51 + 732.40 * edction

R-Square = 0.56

Bivariate Regression & its Measure of Association Remember that Lambda (for nominal variables) and

Kendall’s tau-b/c (for ordinal variables) are PRE measures, which stands for Proportional Reduction in Error.

PRE: When predicting a dependent variable, how much error can you reduce by knowing the independent variable?

Regression has a similar statistic: R-Square or (R2) or “goodness of fit” always between 0 and 1 (0=no reduction in error, 1=error eliminated)

R2 = 0.49Vis

its

to C

ard

iolo

gist

s p

er e

nro

llee

Vis

its

to C

ard

iolo

gist

s p

er e

nro

llee

0.00.0

0.50.5

1.01.0

1.51.5

2.02.0

2.52.5

0.00.0 2.52.5 5.05.0 7.57.5 10.010.0 12.512.5 15.015.0

Number of Cardiologists per 100,000 residentsNumber of Cardiologists per 100,000 residents

Association between cardiologists and visits per person to Association between cardiologists and visits per person to cardiologists among Medicare enrollees: 306 HRRscardiologists among Medicare enrollees: 306 HRRs

SourceSource: John Wennberg (2005): John Wennberg (2005)

Discharges forHip Fracture

R2 = 0.06

Discharges forall MedicalConditionsR2 = 0.54

00

5050

100100

150150

200200

250250

300300

350350

400400

1.01.0 2.02.0 3.03.0 4.04.0 5.05.0# of Hospital Beds/per 1,000 Residents# of Hospital Beds/per 1,000 Residents

Dis

char

ge R

ate

Dis

char

ge R

ate

Association between # of hospital beds per 1,000 residents and Association between # of hospital beds per 1,000 residents and discharges per 1,000 Medicare enrollees in 306 HRRsdischarges per 1,000 Medicare enrollees in 306 HRRs

SourceSource: John Wennberg (2005): John Wennberg (2005)

Bivariate Regression & Measure of Association

5.00 10.00 15.00 20.00

Years

8000.00

12000.00

16000.00

20000.00In

com

e ($

)

Income ($) = 5077.51 + 732.40 * edctionR-Square = 0.56

13866.31

5.00 10.00 15.00 20.00

Years

8000.00

12000.00

16000.00

20000.00

Income ($) = 5077.51 + 732.40 * edctionR-Square = 0.56

13866.31

Bivariate Regression & Measure of AssociationX = 6 years of education & $8,000 income

Regression line estimated approx. $9,000 in income for 6 years of education.

Yet the mean score ($13,866) would have left you with $5,866 in estimated error.

With the regression line (coefficient), you reduced the amount of error by $4,866; you are left with $1,000 in estimated error.

In other words, by knowing the independent variable, you have $4,866 of explained variation and $1,000 of unexplained variation.

explained ($4,866)2

explained ($4,866)2 + unexplained ($1,000)2

R-Square = aggregating (summing) this for all of the data points in the scatterplot

Correlations

1 .751**

. .000

32 32

.751** 1

.000 .

32 32

Pearson Correlation

Sig. (2-tailed)

N

Pearson Correlation

Sig. (2-tailed)

N

Years

Income ($)

Years Income ($)

Correlation is significant at the 0.01 level (2-tailed).**.

5.00 10.00 15.00 20.00

Years

8000.00

12000.00

16000.00

20000.00

Inco

me

($)

Income ($) = 5077.51 + 732.40 * edctionR-Square = 0.56

13866.31

Bivariate Regression & Measure of Association

.751 x .751 = 0.56r2 = R2

Multivariate or Multiple RegressionY = a + b1x1 + b2x2 + b3x3 + e

Allows you to determine the marginal effect of an independent variable on one dependent variable by simultaneously controlling for several other independent variables.

The regression coefficient b1 estimates the average change in Y (dependent variable) for each unit change in independent variable x1, controlling for the other independent variables: x2 and x3, etc.

Same concepts as bivariate regression, but in a three-dimensional + scatterplot (which can’t be graphically illustrated)

The difference is that you obtain partial slopes and partial regression coefficients, which means the “marginal” effect that each independent variable has on the dependent variable.

0 to 1,600 Low (4.6%)1,600 to 3,150 Below Average (25.5%)3,150 to 5,150 Average (43.5%)5,150 to 6,750 Above Average (19.6%)6,750 to 8,350 High (4.9%)

grams/per 100,000 Individuals

8,350 to 11,000 Extremely High (1.8%)

Methylphenidate and Amphetamine Methylphenidate and Amphetamine Distribution (DEA data)Distribution (DEA data)

(average = 4,150 grams/100,000 individuals)(average = 4,150 grams/100,000 individuals)

Analysis of the Demand for Psychostimulants:Multivariate Regression

Data Sources:

– Dependent variable I: DEA data provides the distribution of methylphenidate and amphetamine in grams down to the 5 digit zip code-level (1998-2001)

– Dependent variable II: Source® Territory Manager by NDC HEALTH provides Rx Sales and Rx Quantity sold by product down to zip/county or HSA level from 1996 onwards; ** Average price to be derived from Total Sales/Total Quantity **

– Independent variables: Area Resource Files (ARF), InterStudy, Census Estimates, County Business Patterns, Dept. of Education’s Common Core of Data

Market Definitions:

– 3,030 U.S. counties

Selected Regression Coefficients for Market Model * indicates that the coefficient is significant at the 10% level, ** at the 5% level, *** at the 1% level

Variable RITALIN ADDERALL

Coefficient Std. Error Coefficient Std. Error

Intercept -5.34*** 1.64 8.05*** 1.56

Price of Ritalin -3.76*** .654 .828*** .220

Price of Adderall .436 .477 -3.86*** .410

Price of Dexedrine 1.13*** .284 1.28*** .273

Price of Dextrostat .653* .337 1.53*** .388

Per Capita Income 1.54*** .189 .228 .183

Population (in 100s) .115 .287 1.91*** .280

Selected Regression Coefficients for Market Model * indicates that the coefficient is significant at the 10% level, ** at the 5% level, *** at the 1% level

Variable RITALIN ADDERALL

Coefficient Std. Error Coefficient Std. Error

# of kids (<19) .768*** .254 -.860*** .261

HMO penetration -.062** .026 -.083*** .026

Students-to-Teacher -.392** .175 -.359** .180

# of Private School Students .064*** .018 .024 .020

Accountability Index^ .055*** .018 .041** .108

# of MDs (Total Patient Care)

.260*** .059 .085 .062

Young-to-Old MDs .094** .039 .110*** .038

^ Accountability Index: Ranges from 0 to 5 0 = no teacher accountability and 5 = most stringent policies

R2 (Ritalin) = 0.715 R2 (Adderall) = 0.672