Feb 21, 2006Lecture 6Slide #1 Adjusted R 2, Residuals, and Review Adjusted R 2 Residual Analysis Stata Regression Output revisited –The Overall Model –Analyzing

Feb 21, 2006 Lecture 6 Slide #1

Adjusted R2, Residuals, and Review

• Adjusted R2

• Residual Analysis• Stata Regression Output

revisited– The Overall Model– Analyzing Residuals

• Review for Exam 2


Homework Review

• Use variables 87-89 to make an “egalitarian” index for your dependent variable (Y)

• Use p98_ideo (ideology) as the independent variable (X) to predict egalitarianism. Fully interpret the results


Adjusted R2: An Alternative “Goodness of Fit” Measure

• Recall that R2 is calculated as:

• Hypothetically, as K approaches n, R2 approaches one (why?) – “degrees of freedom”

• Adjusted R2 compensates for that tendency

∑ ∑ −=−=

=

22

2

)( and )ˆ(

: where,

YYTSSYYESS

TSS

ESSR

ii

“explained sum of squares” “total sum of squares”


Calculating Adjusted R2

Ra2 =R2 −

K −1n−K

(1−R2)

• The bigger the sample size (n), the smaller the adjustment• The more complex the model (the bigger K is), the larger the adjustment• The bigger R2 is, the smaller the adjustment


Residual Analysis: Trouble Shooting• Conceptual use of residuals

– e, or what the model can’t explain

• Visual Diagnostics – Ideal: a “Sneeze plot”– Diagnostics using Residual Plots:

• Checking for heteroscedasticity• Checking for non-linearity• Checking for outliers

• Saving and Analyzing Residuals in Stata


Review: Assumptions Necessary for Estimating Linear Models

1.Errors have identical distributions

Zero mean, same variance, across the range of X

2.Errors are independent of X and other i

3.Errors are normally distributed

E[ i ] ≠ f(X) and E[i ] ≠ f( j , j ≠i)

i=0

X


The Ideal: Sneeze Splatter

e

Predicted Y

Problems: It is possible to “over-interpret” residual plots; it is also possible to miss patterns when there are large numbers of observations


Heteroscedasticity

e

Predicted Y

Problem: Standard errors are not constant; hypothesis tests invalid


Non-Linearity

e

Predicted Y

Problem: Biased estimated coefficients, inefficient model


Checking for Outliers

e

Predicted Y

Problem: Under-specified model; measurement error

Residuals formodel usingall data

Possible Outliers

Residuals for modelwith outliers deleted


Stata Regression Model:Regressing “Militant” onto “Egal”

Y is index of 81, 82 (rev’d), 84-86:1=low militant; 7=high militant

X is “Egal” -- an index of 87-891=reject egal; 7=accept egal

0

.05

.1

.15

.2

.25

Density

0 2 4 6 8egal


Regression Output. r egress mil itan t egal

S ource | SS d f MS Number of obs = 2 510-- - ---- - ---- - +--- - ---- - ---- - --- - ---- - ---- - -- F( 1, 2508) = 3 05.52 Model | 3 55.806666 1 3 55.806666 Pr ob > F = 0 . 0000 Res i dual | 2 920.83777 2508 1 . 16460836 R- squared = 0 . 1086-- - ---- - ---- - +--- - ---- - ---- - --- - ---- - ---- - -- Adj R- squared = 0 . 1082 Tot al | 3 276. 64444 2 509 1 . 30595633 Root MSE = 1 . 0792

-- - ---- - ---- - ---- - ---- - ---- - --- - ---- - ---- - ---- - ---- - ---- - ---- - ---- - ---- - ---- - - mil i tant | C oef. S t d. Er r . t P >|t| [ 95% Conf. In t erva l ]-- - ---- - ---- - +--- - ---- - ---- - --- - ---- - ---- - ---- - ---- - ---- - ---- - ---- - ---- - ---- - - e gal | - . 2323246 . 0132916 -1 7.48 0 . 000 -. 2583882 -. 2062609 _cons | 4. 837749 . 0611609 7 9.10 0 . 000 4 . 717818 4 . 957681-- - ---- - ---- - ---- - ---- - ---- - --- - ---- - ---- - ---- - ---- - ---- - ---- - ---- - ---- - ---- - -


Regression Plot

0

2

4

6

8

0 2 4 6 8egal

95% CI Fitted values

militant


Predicted Case Confidence Bands

0

2

4

6

8

0 2 4 6 8egal

95% CI Fitted values

militant

Confidence Bands for Predicted Cases


Residual Plot

0

.1

.2

.3

.4

.5

Density

-4 -2 0 2 4Residual

ProbableOutliers


Examination of Residualsgsort e (or you can use “-e”)list resp_id egal militant yhat e in 1/5

Use the respondent ID number to find the relevant observation in the data set

. l ist re sp_id egal mil itan t yhat e in 1/ 5

+-- ---- - ---- - ---- - ---- - ---- - --- - ---- - ---- - ---- - ---- - ---+ | re sp_id egal mi lita nt yhat e | |- - ---- - ---- - ---- - ---- - ---- - --- - ---- - ---- - ---- - ---- - ---| 1. | p1270 1 1 4. 605425 - 3.605425 | 2. | p 462 2. 333333 1 4. 295659 - 3.295659 | 3. | a1339 2 1. 2 4. 3731 - 3.1731 | 4. | a1091 2. 333333 1. 2 4. 295659 - 3.095659 | 5. | p 873 1 1. 6 4. 605425 - 3.005425 | +-- ---- - ---- - ---- - ---- - ---- - --- - ---- - ---- - ---- - ---- - ---+


Residuals v. Predicted Values

-4

-2

0

2

4

Residuals

3 3.5 4 4.5Fitted values

Using an “ocular test,” neither non-linearity nor heteroscedasticity are obvious here. But should we trust our eyeballs?


Formal Test for Non-linearity:Omitted Variables

Tests whether adding 2nd, 3rd and 4th powers of X will improve the fit of the model:

Y=b0+b1X+b2X2+b3X3+b4X4+e. ovtes t

Ramsey RESET te st usi ng powers of the fi t ted val ues of mil itan t Ho: model has no omi tted var iabl es F ( 3, 2505) = 5 . 29 P r ob > F = 0 . 0012


Formal Tests for Heteroscedasticity

Tests to see whether the squared standardized residuals are linearly related to the predicted value of Y:

std(e2)=b0+b1(Predicted Y). hette st

Breusch- Pagan / Cook-Wei sber g te st fo r het eros kedast icit y Ho: Constan t var ianc e Var iabl es: fi t ted val ues of mil itan t

chi 2(1) = 11. 61 Pr ob > chi 2 = 0 . 0007


Case-wise Influence AnalysisThe Leverage versus Squared Residual Plot

p477p649

a711

a909

a85

a665

p950

p657a873a878

p1329a1144

a438

a1423

p1347

a1218

p493

a209a1318p131

a853

a1010p109

p82

p359

a330

a767a664

a1377

p1096p1065a528

p1443

p1408a16

a1253

a1288a1068

p506

p1437p354

a776

a442

a113

a481a1241a437

a1354

p574

p296

a1509

a477

a1366a1117

a50a549

p1292

a394p1181p1367p1045

a757a756

p888a1371a1084a405p1269

a1092p855

p267p1461a1260p31p826

p444

p1262

p484a1302

a902p288

a247p567

a329a1502

a1378a319p1097p230a831

a9

p1283a868a1512p1129p374

a306

p750p1182a950a587

a743

p1247p967

a1127a1017p258a748a252p111

p954a539a957p85p1005a1265

a1455a239p930

a1164a204a847a1174

a741p1195a978a357a1478a208

a648a577p625a62a1086p1072

a927a1026p187p866a962a1201a871p510a924

p546a1120p1079

a286a650a1209a688a33a333

a183a328a482p384a126p1043

a817a1a583

p1048p1471

a321a1404a391

p983a445a421

a1505a931a1430p1107

a138

a419a1508a194

a763p680p862p408a257

a12p853p60p592a1097p326p989a686p1375p770a1406p518a17a132p400a1193a266p463p419p1138p158p445p91p1321p797p540a22a557a406a1005a976p1031

a1251p665p535a1520a1123p1444

a1237a487a171

p1101p653a1290

a1178a1170a507p1060

a443a851p720a285a538p76a999

a533p1202

a1385p1084

a889a633

a760p1467a1310p173a479a324a200p779a23a625p422a265

p686

a370p216a1526p1448a1227a47p846p1288a547p1106

a1345p270a1038a311

a1531a1244

p1280a573a90a799p1317

a865a165p372a503a468a240a1055p612a864

p1083a1166a1151

a1287p871p1418p308a398

p593a1311a107a758a1320a1393a118

a1074a367a234a38p943a1043a745a801

a596a84a1011a410

p440p961a1050a732p426a814p1249a602a1394a340a685p157a904a474

p1343a697a1152p1273p523

a517p118p1030a876

a55a1359a793a26p370a797a236

a659p684p504a1255a1252p246a344a315p90p1123a1106p152a1149a1171p10a683a11p1041a20a1162a390a305p990p178

p606p640

a754p1126a472p1255a521a1333p748p399

p1071a260p1439a1527a1087a1477a282p1113

p1285p695p948p857p932a1334p236p946a385p1p434

a1462p937p736a680a1217a1202a1203a775p627

a710a1364a485p1139a1250

p1440a1079p1032p1037a778p1086p342a856a1292a94

p1219a551a811a536

a537p947p1442a1342p740a100a125p960p897p1406p453p787p872a788

a470p636a276a1064a88a632a908a1073

a353p821p661p1414a590p696a1175p929p550a451a747

a622a371p743a800a1248a961p667a857a212

a1095a133a1397p496a75a966

p569a203a222p1196a527a187a603p669a1154a317p1302a579a925p543a1133p127a1013a560a480p1073p648a524a486a951a1024p1276

a668p777a718a854p22a462

a129p1104a170a302

p529p175a798a489p827a1278a500

a1280a550a644a1410a1530a173a163a1012p1165p327p224p762a456

a1090

p979a359a310a189a409a691

p1171a293a1326p480a284a610a469p443p72a1309p447p539a910a80a1400a1500

a32a944p563a952p751p687p1438

a130a832a116p431a229

a1454a323a1452a1346a1085a1497p580a65a818p1246a709p893

a117a501p289p727p642a545a1418p611a651a186a752a177

a460a396p1272a418

a985p147p1303p1146a867a726p887a215

a71p1159p1299p69a755p403p416a1363a764p1429a221

a934a992p1020p544a1450a233p851a1521a256a1372p1278p598a1332a1504a1419p1328a532

a762p351p710a1328p1301p782p1161a720a1211a1204

a708

a156p1370p169a696a211p488p502a837a859p191a1033p104a906a1083a402a82a473

p309p1091p575p46a1206a612

p852a40a529p454a127p1339a504a140a1513a7

a196a1380a448a972p958a1351a1323a1007a572p74p1290p479a1165a1122p1125a68

a510a102a575p1050p1417a147a314p1234p208a1503a1270a1111p928a701p1209p360

p34a1470a1483a1299a363a609p1231

a1056a382a628a513a1132a1102a1533p735a154

p901a31a1199a564a1487a1389p936a727a1082p242a122p507a56

p1289p994a991a1046a841a1025

a1369a1501a850a86a238a1519

a308p775p576a214a813a508a272a1514p73a670

a1049a1475a1070a395a852a586a615a1224a505a120p957a848p783

a490a493a956a1047a531a1445a159a322a1160p883a14p617

a195a1001p70p1203p505a885

p1078a1435a49a887p376

p251a1447p61a241p941a1312p589a639p974p211p1122a903

a1329a1192p238a1275a735a326p1386p1221a1161a552a1134a870a698a459a27a43a509a518a652a1021a1163

a59a506p3

a1243a780p47a378p281p205a1361a1239a1315a87p801a1008a373a191a149a766a630a19a1130p192a146p934a431a444a519a1231a1066p931p879a1236a1424a578a436a567p987a226p629a872a1002a988a1262a54

a275p884

a258a300

a880p25a1362p245a1336a143a821p869a746a729a1398a783a1440p455a819

p314p898p37p683a148p1000a618p39p7p278

a476p982a380p1325p1466a810

a6p756a465p1351a1534a877a1420a562p108p1433a128a169p524p956p1462a728a936a327p927p1248a463p28

a250p366p1469a245a307p345a1425a175a591a901a467a769p726a1139a1271a427a1535

a1103a1301a1456a464a392

a365p940a1061a955a1279p798a892

a1382a492a30a1431p1175p997

a1532p1208a1182a895a111a953a666a1000a565a556p1332p708a1492p1011a1037p56

a73a1223a948p40p1360p1399

p1211p13p465a219

a526a942a926p1423a1298

a70a828a389a1235p652p103p467a360p1136a733a1388p343p1241a1125a643p534p1258p367a255a965a1076

a881a646a658p536a1516p1434

a968p1094a1476a890p908p241a1267a1465

p558a1460a1496a1489a730a29a933a1113a593p265p452p791a563a488p347p1153a1343a457a580a335a546a782p115p913

a1030p643

a555a278p1021p981p282a425a3

a1268p584a1350p721p257a849a1273a1434a496p333p1304p578a484a42a1365p1427

a1020a1016p1436p747a428a243p247p116a452p1022p1425a1448p1015p630p213p95a181p1028a232p181

a1374p1229a415a1247

a681a199a105p490a145a682p1468a607a178a1109p52a566

a937a888a1254a1466p850a1006p1220a57a893p237a225a404a827a1327a512a1229p685p825a809p837a1184a1023

a1042a1058a1187p608

a1148a823a1356a997

a1198a1080a899a1426a287p1238a136p763a917a1081p1192a1449

p209p57a804a1291p538a723p1457a597

a1499p1340a15

p1025p568p922a1089a281a768p889p986a690p861a198p1410p1327a949p1034p1127p613a1474

a1409p1286a188a705a655

a725a724a1135a1071p885a106p906a1057a542a898p1036a858p1133a1035p473a497

a954a274a1383p66a1119a862p1134a1304p271p1222a1325

p820p1183a838a816p1300p766p1204

a969p1320a1220p401p864a182a92p519p429a626a423p290a929a994a1069a1283a1396a845p810p1064a1432a844a369p588a417a494a530a1341a938a288

p1403a1044p844p1432

p300a923a900p577a269a13a695p1419p425a1473a829p839p965

a638a348a377a34a1104a152a1368a582a412a1219a1443a96p1218a692a1099p481p1212a1387a635a1213a1370a1124a1375a217a1207a271a1141a874a237a1276a1469a155p379a350a1436a1427a280a135a433a839a97a599a1468a1524a1191a153a619a375a39a220a172a744a736a157a1490a1522a1177a1422p275a986a407

a592p556a998p1080p1214p1130p1356p723

a1464p1149

a355a627p847p154p1059a1529p276a623p313p1074a1003p520a8a1480a916

a1411a167a440a1322a1518p240a833a28p1191a1179p618p350p975p1268a461p220

a641p373p26p1397p595a1510p951p1311

a569a270p845a1094a61a420p1082a1305p145

p1190a455a101p1062p521p1100a1317a83a511a907p335a791a1190a1472

a620a634p1184a193a21p122p1346

p168p83p55a491a759a1353p475p176p1067p198p1293a309p336p464

a706a1045

a663a53p570p554

a606p620p59a1098p219a1032a1446a231a1442a466a765a1157p1381a1121p460a1264a1482

p953p924p1336p977a1131a1233p806p691

a1158p1157p450p1405a679

a807a1416p880a64p699a441p1284p786p337a296a1439p1092a588a806a722a179p882a523p944a290a971p1069p1357a784a672p903

a166a77a717a1101

p53a1525a354p244

a825p1355a861p97a347p497a458a739a112a413a824a1036a742p485a772p223a740a1168

p1380a774a703a939a1225p417p161

a1214

p363a554a568a960a1195a1059a958a1407p195a41a571a1232a675a1451

p815a790a1399p79a1052a1348a1222a1376a820a160p210p448a292p1257a206

p1392p411a450p963p902

p622a1286

a1176a139a894p754a1014p1251a604a1401a1297p526p153p939a932p402a283a1180a1330a1349

p377a1027a1319a123a1523a794p1279a362p679a716p894

p295p1163p793

p291

p71p121a514p604p255p626

p1008p915a361p722p358a334p43p1049p1103a940a1221a1395a981

p1105a383a787p171a761a168

p1052a246p663p1205p226p1447

a1022a1284p814p17a1511a1186p681p1277p522a478a715p19a721a141a1438a259a1386p926a453p332a1028p1465a1381p1151

a381a1188

a1303p203a1347p647a1289p935a349a802p4

a228a1118a1507a674a1210a261a812a1018a1463a218p508p268a1078p1331

p406p637p356a1392a1340a1428p301a400

a366p112a671p811a678

a918p129a1413a559p27p632a1173a836p1164p781a1421p738p1259p1145

p881a1096p1459a543a5a1189

a301

a617

a291a483a99p692a387a1067p602

p294p1390a10a1355a1402a1528p517

a1156p555p1018

p749p75a185p644a230

a110a1515a973p1051a51p1058a882

a753a162p9a1062a343a1461a700a822a251p435a1269a815p1053a996a1249a1259

p405a262a210p1449a1453a332a1493

a150a346p1063

p572a318a834p1076p969p280p134p250p1314a399a1147a1488p1124

p1109a297

a174p785a570a137a980p895

p364a180a1481p303a687p671a115p1451p8a989a104a995

a403p892a36p221a714

a515a915p165

p45p703p591a103p1256a1240a1172a1277a982

p285p605p757a750a987p283a1112a522a553p907

a544p1167a647p386a338p1150a843

a795a79a299

a1100a1307a977p409a1263p461p98a1040p725p135

a921p393a631a1216a796p860p466p586p100p1193p737

p1087p1198a388

p905a471p817p1066a1065

a846p1057a963p306p362p1162

a352a447a351a336a548a640a1258a95a35a777a1137

a624a930

p707p407p334p875

p1395p590a1034p832a1331p768p765a541a320a1459a525a781a295a368p1261a605a1412a1110a946p993p1114

a786p1295

p1075a1129a1295a1009p698p1333

a661p137a277p633p2a1060a98a1338a273a990

p1070p456p357a920p378p1441p1294

p254a1138a702

p148a376a1479p1330a439p1387a1181p392p382a561p509

p1042p1016a1256a345p437a1215p1242a1257

a576p183

a1031a213

a1444a411p29a1169p1239

a60a1029a922p949p99a202p1093

p471p767p274p1411a662

a869a1150a279a430a967

p745p88a1517a1367p197a207p1313p126p830

a298a608p742a855p858

p1344a1053

p1364p999p773p368p1128a25a1458p1158a205

a589a331a912

p184p133a316

p758a805p299a595p582p387a535

a1261a1077p730a1335

a227a303

a142p614p715p365p886

p541p512a1316a516a1405

a905p1089a614a1429p1201p41a1196a121a1321

a1143

p1307a1063p1004a244p557p503a1417a785

p1010a1185a67a93p102a707a1384a4a1226a959p1027

p760a1457

p1254p701p398a268p1169p678a1467

p302a642p316a520

p971p1394p329p1120

p690a1495a190a1491a1433

a48a534a601p809a91a1197a1200a731

p49a684a713a414a676p1377a1212a397a584a24

p163p1174

a637a422

a974p607a645p1115a254a339a364p709p138a401

p610a1281

a749

p93a789p988a1105p896p962a863

a235p427

p412p391p1135p585p772a374p207

p341p1006

a884p600p110a224p804

p973a264a1296a1107

a76a356a1266p978a594a1075

p225a1136p1154a689p1054p62

p50p1173p162

a803

p662a424a770p597a201p1197p261p1237p1187p621

p532

p141p1213a883a1140a69p731a935a993

a636p1398

a983a826a429a808p32a192

p1309a161p286

p800a1048

a1114p1102

a897a896a44p717

a124p272a242

a941a660

a1205a975p105p527

a1155

p319

p1378p248

a2a1116a737p439p733

a1308p217a18p146

p513

a1153p1450a585p321a842p353p317p672p231

p1323

p863p136p1002

a1093a1294p81p676a249

p1287a677p170p430a694

p487p256

a341a1245

p964p831a1485p120a1390a372p1233

p189p269p914

a1403a45a63a792a654p1141

p38a712

a495

a1357a114a78p476a1337p553a1272

p1117a1228

a432p125p1039p764

a1293a454p421

a393p1315a358a673p273a498a434

a223p1353p1077p706

p619a74p1376p755

a1344a840a1486

p199p48a1004a1115p1240p1413a151p682a1391

a253p1470

a1051

a108p1009a1379a751

p1372p537

a581a89a928a879p1224p1345

a657a197p655a1484p959a1167a408

p375p185a109p442p1046a598p1215

p396p734

p498

p474a1146a499

a1324

a914a738

a52p1244a1054a58a304a1282p1431

a1506a386a248a574

p182a835

p635

a1274

a426p459a860p674

p843

p128a911a1360p413

a1494a131a947a704a1019p1044

a1441a653p1361a158p51a289

a616a970

p840a611a1373a164

p1460p324p1026

a1208p1172a379

a1159p673p728p89p628p1236a621

p1029p596p996

a1108p995

p912p259

a1039

p404p1297p101a1183a656a1088a416a1498a1194p1369p483a771p998p790a46p714a779p1445a1072p1245a144

a313a72

p1137a384a1471p525

a891a312a1242a449p349p664

a1306p1389

p646p1274p824

p904p1366p528

p390

a446p234a216

a943

p1228a337a1246

p214a294a600p200p1252p599

p945p202

a1408a558

a1358a1126a886p92a649a37

a830a1238p206p545

p1186

a1145

p1178a719p739a1015a1128

a435

p96a773

a66a699

p1166

p322

p1422a984

p601

p410p388

a81

p1170a979p1416p1227p331

p624

p500

a475

p873

a1091

a1339

p462

p1270

.0005

.001

.0015

.002

Leverage

0 .001 .002 .003 .004 .005Normalized residual squared


What to Do?• Nonlinearity

– Polynomial regression: try X and X2

– Variable transformation: logged variables– Use non-OLS regression

• Heteroscedasticity– Re-specify model

• Omitted variables?• Use non-OLS regression (WLS)

• Influential and Deviant Cases– Evaluate the cases– Run with controls (multivariate model)– Omit cases (last option)


Review

• Basic statistics and functions

• Calculus– Derivatives and critical values

• Deriving b1 and b0

– Intuitive meaning of formulas

• Interpreting OLS bivariate regression output

• Basic regression diagnostics

Documents

Feb 21, 2006Lecture 6Slide #1 Adjusted R 2, Residuals, and Review Adjusted R 2 Residual Analysis Stata Regression Output revisited –The Overall Model –Analyzing