Success and Attrition Factors for High Achieving Underrepresented/ Underserved Students

Barry Nagle, GMS/UNCF Senior Research AssociateJin Liu, Research Analyst

National Scholarship Providers Association Annual ConferencePittsburgh, PAOctober 2014

2

Background: GMS Program

The Gates Millennium Scholars (GMS) program, established in 1999, is a 1.6 billion dollar initiative funded by grant from the Bill & Melinda Gates Foundation.

The goal of GMS is to promote academic excellence and to provide an opportunity for 20,000 outstanding students with significant financial need to reach their full potential.

3

Background: Program Partners

• UNCF- the United Negro College Fund is the administrator of the GMS initiative and has partnered with the following organizations:

• Asian & Pacific Islander American Scholarship Fund

• American Indian Graduate Center Scholars

• Hispanic Scholarship Fund

4

Topics

• Success/Attrition Factors Demographic Characteristics Institution Characteristics Student Education Characteristics

High School GPA Nomination Composite Scores STEM Major Status Deferment

• In Development: Engagement• Applying the Knowledge• Discussion

5

Study Purpose

• Identify students potentially at-risk of not attaining graduation

6

Success Factors:Demographics

7

Demographics: Cohort

• Overall graduation rates are approximately 90% (Scholars in Cohorts 1-8)

91.5% 89.6% 91.3% 89.8% 90.7% 89.9% 90.4% 91.0% 90.7%

0%

25%

50%

75%

100%

Cohort 1 Cohort 2 Cohort 3 Cohort 4 Cohort 5 Cohort 6 Cohort 7 Cohort 8 Overall

Per

cen

tag

e

Cohort

8

Demographics: Gender

• Females earn an undergraduate degree at higher rates than males

• Females have a 91% graduation rate. Males have a 89% graduation rate. X2= 18.8366, df=1, p<.000

90.7%

89.3%

91.4%

0% 25% 50% 75% 100%

Total

Males

Females

Percentage

Gen

der

9

Demographics: Primary Ethnicity

• Three Primary Ethnicity groups have graduation rates of 90% or more (X2= 576.0241, df=3, p<.000)• African American: 93.4%• American Indian: 72.0%• Asian & Pacific Islander: 95.8%• Hispanic American: 92.5%

90.7%

92.5%

95.8%

72.0%

93.4%

0% 25% 50% 75% 100%

Total

Hispanic American

Asian & Pacific Islander

American Indian

African-American

Percentage

Pri

mar

y E

thn

icit

y

10

Demographics: Gender and Primary Ethnicity

• For every ethnicity, females have higher graduation rates than males (X2= 595.1700, df=7, p<.000). Female-Male differences:• African American: 3.3%• American Indian: 2.9%• Asian & Pacific Islander: 0.4%• Hispanic American: 2.1%

94.4%

72.9%

96.4%92.6% 91.4%91.1%

70.0%

94.4% 92.2% 89.3%

0%

25%

50%

75%

100%

African-American American Indian Asian & PacificIslander

HispanicAmerican

Total

Females Males

11

Demographics: Other

Graduation Rates for Students with Known Status

• First-generation college students• First-Generation: 90.2%• Non First-Generation: 89.5%

• Dependency Status• Dependent: 90.6%• Independent: 87.1%

12

Success Factors:Institution Characteristics

13

Institution Characteristics: Top Graduation Rates

Top Ten by Graduation Rate/50 Students or More

Earned Degree

TotalGraduation

Rate

Spelman College 66 67 98.5%

Duke University 63 64 98.4%

Stanford University 173 176 98.3%

University of Southern California 99 101 98.0%

Georgetown University 49 50 98.0%

Harvard University 136 139 97.8%

University of Pennsylvania 70 72 97.2%

Columbia University in the City of New York 60 62 96.8%

Brown University 75 78 96.2%

Cornell University 97 101 96.0%

14

Institution Characteristics: Private vs. Public

• Scholars that attend private institutions graduate at higher rates than Scholars that attend public institutions

• Private school attendees have a 93.9% graduation rate compared to the public school attendee graduation rate of 89.3% (X2= 81.2958, df=1, p<.000)

90.7%

93.9%

89.3%

0% 25% 50% 75% 100%

Total

Private

Public

Percentage

Inst

itu

tio

n T

ype

15

Institution Characteristics: Private/Public and Gender• Females and males that attend private institutions graduate at

higher rates than Scholars that attend public institutions (X2= 105.8424, df=3, p<.000)

94.6%

92.7%

90.1%

87.5%

0% 25% 50% 75% 100%

Private: Females

Private: Males

Public: Females

Public: Males

Percentage

Inst

itu

tio

n T

ype

16

Institution Characteristics: Private/Public and Primary Ethnicity• Scholars that attend private institutions in all PE groups have

greater graduation rates than public school attendees (X2= 618.804, df=7, p<.000). Private-Public differences:• African American: 1.8%• American Indian: 6.1%• Asian & Pacific Islander: 0.9%• Hispanic American: 4.8%

92.8%

72.3%

95.5%90.8% 89.3%

94.6%

78.4%

96.4% 95.6% 93.9%

0%

25%

50%

75%

100%

African American American Indian Asian & PacificIslander

HispanicAmerican

Overall

Public Private

17

Institution Characteristics: Private/Public, Gender and Primary Ethnicity• For all primary ethnicities, females and males that attend private

institutions have higher graduation rates than Scholars that attend public institutions (X2= 642.6317, df=15, p<.000).

• AA, AP, and HA Scholars that attend private institutions have graduation rates of 90% or higher

• AI Scholars that attend private institutions have graduation rates of 70% or higher

18

Success Factors:Student Education Characteristics

19

Student Characteristics: High School GPA

• Scholars that earn a degree have higher High School GPAs than Scholars that do not earn a degree

• Degree earners have a 3.77 mean HS GPA compared to 3.70 for non-degree earners (t= 11.3729, df=924.425, p<.000)

3.68

3.77

0 1 2 3 4

No Degree Group

Earned Degree Group

High School GPA

20

Student Characteristics: High School GPA by Gender• Female and male Scholars that earn a degree have higher High

School GPAs than female and male Scholars that do not earn a degree, respectively (F=48.55, df=3, p<.000)

• Significant TukeyHSD contrasts:• Female Earned Degree HS GPA (3.78) vs. Female Non-Earned

Degree HS GPA (3.67)• Female Earned Degree HS GPA (3.78) vs. Male Non-Earned

Degree HS GPA (3.70)• Male Earned Degree HS GPA (3.77) vs. Male Non-Earned

Degree HS GPA (3.70)• Male Earned Degree HS GPA (3.77) vs. Female Non-Earned

Degree HS GPA (3.67)

21

Student Characteristics: High School GPA by Primary Ethnicity• For each Primary Ethnicity group, the HSGPA of the earned degree

group was higher than the non-degree earning group (F=68.95, df=7, p<.000)

• Sixteen TukeyHSD contrasts were statistically significant

3.74 3.73 3.86 3.773.70 3.65 3.77 3.71

0

1

2

3

4

AA AI AP HA

Earned Degree No Degree

22

Student Characteristics: High School GPA by Gender and Primary Ethnicity• For each Primary Ethnicity and Gender group, the HSGPA of the

earned degree group was higher than the non-degree earning group (F=68.95, df=7, p<.000)

• 54 (of 120) TukeyHSD contrasts were statistically significant3.72 3.75 3.74 3.73 3.86 3.86 3.79 3.773.71 3.65 3.67 3.65 3.84 3.70 3.68 3.73

0

1

2

3

4

AA-Male AA-Female AI-Male AI-Female AP-Male AP-Female HA-Male HA-Female

Earned Degree No Degree

23

Student Characteristics: Nomination Scores

• Scholars that earn a degree have higher nomination composite scores than Scholars that do not earn a degree

• Degree earners have a 77.49 mean composite score compared to 76.10 for non-degree earners (t= 5.3582, df=799.943, p<.000)

76.10

77.49

0 25 50 75 100

No Degree

Earned Degree

24

Student Characteristics: Nomination Scores by Gender• Female and male Scholars that earn a degree have higher

nomination composite scores than female and male Scholars that do not earn a degree, respectively (F=15.61, df=3, p<.000)

• Significant TukeyHSD contrasts:• Female Earned Degree score (77.49) vs. Female Non-Earned

Degree HS GPA (76.26)• Female Earned Degree HS GPA (77.49) vs. Male Non-Earned

Degree HS GPA (75.86)• Male Earned Degree HS GPA (77.50) vs. Male Non-Earned

Degree HS GPA (75.86)• Male Earned Degree HS GPA (77.50) vs. Female Non-Earned

Degree HS GPA (76.26)

25

Student Characteristics: Nomination Scores by Primary Ethnicity• AP and HA Scholar degree earners have nomination scores higher

than non-degree earners. AA and AI degree earner nomination scores are slightly lower than non-degree earners (F=86.6, df=7, p<.000)

• Fifteen TukeyHSD contrasts were statistically significant

78.4 73.7 78.5 77.378.4 73.8 78.1 76.9

0

25

50

75

100

AA AI AP HA

Earned Degree No Degree

26

Student Characteristics: Nomination Scores by Gender and Primary Ethnicity• The following Gender/PE degree earner groups have nomination

scores higher than non-degree earners: AA males, AI males, AP females, HA males, HA females. (F=41.40, df=15, p<.000)

• 53 (of 120) TukeyHSD contrasts were statistically significant

78.5 78.373.1 74.0

78.3 78.5 77.5 77.278.3 78.472.7 74.4

78.5 77.9 76.9 76.9

0

25

50

75

100

AA Males AA Females AI Males AI Females AP Males AP Females HA Males HA Females

Earned Degree No Degree

27

Student Characteristics: Nomination Score AreasCognitive

• Curriculum rigor • Overall academic achievement • Structure of/use of language in essays

Non-Cognitive

• Positive self-concept/self-esteem • Realistic self-appraisal • Understanding and navigation of social and organizational

systems • Preference for long-term over immediate need • Successful leadership experience • Community service • Non-traditional, Self-directed acquisition of knowledge or skill • Evidence of strong support person

28

Student Characteristics: Nomination Score Correlation ResultsSignificant Cognitive Correlations

• Curriculum rigor (r=0.1445, p <.000)• Overall academic achievement (r=0.1369, p<.000)• Structure of/use of language in essays (r=0.769, p<.000)• Cognitive composite (r=.1634,p<.000)

Significant Non-Cognitive Correlations

• Positive self-concept/self-esteem (r=0.0349, p=.007)• Understanding and navigation of social and organizational

systems (r=0.0312, p=0.16)• Preference for long-term over immediate need (r=0.0441,

p=.001)• Non-Cognitive composite (0.0260, p=.045)

29

Student Characteristics: Nomination Score Differences by Area

0.37 0.36

0.080.05

0.10 0.11

0.03-0.02 -0.01 0.01

0.22

0.12

-0.1

0.0

0.1

0.2

0.3

0.4

0.5C

urri

culu

m R

igor

Aca

dem

ic A

chie

vem

ent

Sel

f-C

onc

ept

Rea

listic

Sel

f-A

ppra

isal

Nav

igat

ing

So

cial

Sys

tem

s

Lon

g-T

erm

Goa

ls

Lead

ersh

ip

Co

mm

unity

Ser

vice

Kn

ow

ledg

e A

cqui

red

Sup

po

rt P

erso

n

Ess

ays

Ave

rag

e D

iffer

ence

30

Student Characteristics: Nomination Score t-test ResultsSignificant Cognitive Mean Differences

• Curriculum rigor (d= 0.37, t=9.439, p <.000)• Overall academic achievement (d= 0.36, t=9.438, p <.000)• Structure of/use of language in essays (d= 0.22, t=5.152, p

<.000)• Cognitive composite (d= 0.94, t=10.47, p <.000)

Significant Non-Cognitive Mean Differences

• Positive self-concept/self-esteem (d= 0.08, t=2.227, p =.026)• Understanding and navigation of social and organizational

systems (d=0.10, t=2.114, p =.035)• Preference for long-term over immediate need (d=0.11,

t=2.950, p =.003)• Non-Cognitive composite (d=0.35, t=1.555, p =.120)

31

Student Characteristics: Nomination Scores

Area Significant For All Scholars

Significant for Degree Attainment in Five Years

Curriculum Rigor Yes Yes

Academic Achievement Yes Yes

Essays Yes Yes

Cognitive Index Yes Yes

Yes means the correlation between the area and degree attainment was statistically significant at the .05 level

Cognitive Areas

32

Area Significant For All Scholars

Significant for Degree Attainment in Five Years

Self-Concept Yes Yes

Realistic Self-Appraisal Yes

Navigating Social Systems Yes Yes

Long-Term Goals Yes Yes

Leadership Yes

Community Service Yes

Knowledge Acquired Yes

Support Person Yes

Non-Cognitive Index Yes

Yes means the correlation between the area and degree attainment was statistically significant at the .05 level

Non-Cognitive Areas

Student Characteristics: Nomination Scores

33

Student Characteristics: STEM Major

• Scholars that are STEM majors graduate at higher rates than non-STEM majors

• STEM majors have 92.9% graduation rate compared to 89.9% for non-STEM majors (X2= 11.3737, df=1, p<.000)

90.7%

89.9%

92.9%

0% 25% 50% 75% 100%

Overall

Non-STEM

STEM

Graduation Rate

34

Student Characteristics: STEM Major by Gender

• STEM and Non-STEM females have higher graduation rates than STEM and NON-STEM males (X2= 260.1778, df=3, p<.000)

94.5% 90.7%90.3% 88.6%

0%

25%

50%

75%

100%

Female Male

STEM Non-STEM

35

Student Characteristics: STEM Major by Primary Ethnicity• Graduation rates for all PE groups are higher for STEM majors than

non-STEM majors except for HA Scholars (X2= 219.9267, df=7, p<.000)

93.9%

78.5%

97.0% 92.5%93.3%

70.7%

94.7% 92.7%

0%

25%

50%

75%

100%

AA AI AP HA

STEM Non-STEM

36

Student Characteristics: Deferment Types

• Deferment Types

• Academic

• Personal Hardship

• Medical

• Service

• Emergency

• Personal (Administrative): Includes only Scholars that were given this deferment prior to Senior year

• Scholars Considered

• Year Confirm: 2000-2007

• Freshman Start Point

37

Student Characteristics: Graduation Rates by Deferment Type

Type Graduates Population Graduation Rate

Academic 451 519 86.9%

Emergency 21 23 91.3%

Medical 114 136 83.8%

Personal (Administrative)^ 91 230 39.6%

Personal Hardship 456 615 74.1%

Service 24 28 85.7%

Two or More Types 96 210 45.7%

No Deferments 6321 6631 95.3%

^May be low due to lack of information on graduation status

38

Student Characteristics: Graduation Rates by Deferment Type and PE Group

Type AA AI AP HA Overall

Academic 86.5% 74.2% 93.3% 91.0% 86.9%

Emergency 94.4%* -- 50.0%* 100%* 91.3%*

Medical 82.0% 71.8% 100%* 93.3% 83.8%

Personal (Administrative)^

43.6% 22.8% 31.3%* 62.2% 39.6%

Personal Hardship 77.8% 46.3% 86.0% 81.0% 74.1%

Service 60.0%* 71.4%* 100%* 100%* 85.7%*

Two or More Types 56.9% 21.7% 61.1%* 56.9% 45.7%

No Deferments 96.4% 86.9% 97.6% 95.4% 95.3%

*Less than 30 Scholars

^May be low due to lack of information on graduation status

39

Student Characteristics: Graduation Rates by Deferment Time

Type Graduates Population Graduation Rate

0.5 AY 369 428 86.2%1 AY 738 1085 68.0%1.5 AY 74 126 58.7%2 AY 50 85 58.8%>2 AY 22 37 59.5%No Deferments 6321 6631 95.3%

86.2%

68.0%

58.7% 58.8% 59.5%

95.3%

0.5 AY 1 AY 1.5 AY 2 AY >2 AY No Deferments

Graduation Rate by Academic Year Deferment Time

40

Factor Being Developed:Student Engagement

41

Engagement Index

• Annual GMS Engagement Survey

• Five Engagement Areas• Academic Engagement (Degree goal, Study habits, Class

Preparation) • Campus Engagement (Activity participation, Campus service

usage)• Community Engagement (Volunteer/Public Service)• GMS Program Engagement (Knowledge/use of program

resources, Program activity participation, Scholar engagement)• Non-Engagement (Non-academic related employment/time)

• Individual index areas are combined for an overall index

42

Engagement Index

• Engagement level is defined as high, moderate, low, or no engagement in each area.

• Overall Engagement Formula:• Academic + Campus + Community + Program – Non-

engagement

43

Engagement Index: Preliminary Results All Institutions

Engagement Area High Moderate Low No

Academic 42.9% 55.1% 2.0% 0.0%

Campus 77.6% 17.3% 4.9% 0.2%

Community 26.9% 37.7% 29.4% 6.0%

Program 11.8% 41.8% 44.4% 2.1%

Non-Engagement Area High Moderate Low No

Non-engagement level 8.6% 34.8% 7.2% 49.5%

Overall Engagement High Moderate Low No

Combined 15.7% 65.0% 19.2% 0.1%

44

Engagement Index: Preliminary Results Campus Engagement Manager Institutions

Engagement Area High Moderate Low No

Academic 40.8% 57.4% 1.8% 0.0%

Campus 75.8% 18.1% 5.8% 0.3%

Community 25.1% 38.8% 36.1% 0.0%

Program 19.0% 47.3% 32.1% 1.6%

Non-Engagement Area High Moderate Low No

Non-engagement level 8.1% 34.5% 6.0% 51.5%

Overall Engagement High Moderate Low No

Combined 17.8% 65.0% 17.1% 0.1%

45

Engagement Index: Comparison of High Engagement Levels

Academic Campus Community Program Non-Engagement Combined

40.8%

75.8%

25.1%19.0%

8.1%

17.8%

44.6%

79.0%

28.3%

6.0%9.0%

14.1%

High Engagement Levels

CEM Institutions Non-CEM Institutions

46

Engagement Index: Questions for Next Steps

• Engagement Scores by Level: Do we adjust the cut-scores for each level?

• Weighting: Do we weight engagement levels differently when developing the overall engagement score?

• Outcomes: What outcomes are appropriate to link to engagement?

47

In Development: Applying this Knowledge

48

Applying this Knowledge

• Goal is to use this information to develop a dashboard in the following areas

• Graduation: Due to high graduation rates, valid prediction model can not be developed. Instead will look at success/risk level in each area

• Graduate school in program funded area: Completing a logistic regression model for this outcome

49

Applying this Knowledge: Graduation

• Graduation: To inform on graduation likelihood, index will be created for each Scholar

Scholar Name

Nomination Score

GPA First-Gen

Institution Type

Deferment

Engagement

Overall

Scholar One Sample H H H H H H H

Scholar Two Sample L L L L L L L

• Decisions to be made: Areas to include, risk level for each area, how risk areas will be combined.

50

Applying this Knowledge: Graduate School

• Program funds students for graduate school in these areas:• Computer Science• Education• Engineering• Library Science• Mathematics• Public Health• Science

• Independent Variables being evaluated: Gender, PE, Major, GPA, Institution, First-generation status, Engagement.

• Decisions: Other variables to be considered

51

Discussion