The Excellence Achievement Gap:State-Level Data

Jonathan A. Plucker, Ph.D.Terry Spradlin, M.P.A.

Amber EspingIndiana University

American Psychological AssociationAugust 13, 2006

2

Current Policy Landscape

• Tremendous focus on minimum competency.

• Strongly codified in NCLB, although many pre-NCLB state accountability systems focus on minimum competency.

3

Minimum Competency

• AYP = Adequate Yearly Progress• AYP determined by % students passing

state-defined content standards– Complex calculation …

• Most states have minimum competency (MC) standards– Although some claim to have “high standards,”

they’re generally MC standards

4

MC Systems Are Not a Bad Thing

• Low passing rates within MC systems are deplorable– Especially the large achievement gaps

• Eliminating these gaps and getting every student to MC is an ethically justifiable goal.

5

But is it Enough?

• What does 100% MC by 2014 get us?• NAEP description of “Basic” achievement level

– Partial mastery of prerequisite knowledge and skills that are fundamental for proficient work at each grade.

• Minimum proficiency will have many societal and humanitarian benefits, but it will probably not be the economic engine for which many hope.

• A better goal is to get as many students as possible into the highest levels of achievement.

6

What About Excellence?

• There is growing concern that the tight focus on MC neglects the needs and development of academic excellence.

• This was predictable at the time NCLB passed.

• Counterargument: Rising tides carry all ships.– But do they?

7

The Excellence Gap

• We are interested in whether MC achievement gaps are mirrored in achievement disparities at high levels of academic performance.

• Research Question: Do states with small and shrinking MC gaps also have small and shrinking excellence gaps?

8

History of AddressingExcellence Gaps

• Javits Act has focused on eliminating excellence gaps, but evaluations of funded programs provide very limited evidence of effectiveness.– Javits Act remains very, very small and faces annual

calls for its elimination.

• Some state efforts, almost always short-lived and generally not well-evaluated.

• Many local efforts, but small scale and also rarely evaluated and disseminated.

Research Question:

Do states with small and shrinking MC gaps also have small and shrinking excellence gaps?

10

Method I

• Identified states who are credited with small/shrinking and large/growing MC gaps– Based on Education Trust and NAEP reports

• Examined student performance on state criterion-referenced tests– Disaggregated by race– Examined pass rates for top scoring category

• E.g., Pass+ or Advanced

• Examined student performance on NAEP reading tests– Disaggregated by race– Examined pass rates for Advanced Proficiency category

11

Method II

• Looked at trends over three year periods

• Focused on reading tests

12

Sample for State Tests

• States with shrinking MC gaps

• Biggest decreases in Black-White elementary reading gaps:– Colorado, Delaware, Florida (all 5%)

• Biggest decreases in Hispanic-White gaps:– Delaware (18%), Virginia (7%), Florida (5%)

• Source: Ed. Trust, Measured Progress

13

State Data on Black-White Excellence Gaps: Colorado Grade 4 Reading

0

2

4

6

8

10

12

2002 2003 2004

% P

as

sin

g A

dv

an

ce

d L

ev

el

African American White

14

State Data on Black-White Excellence Gaps: Delaware Grade 5 Reading

2002 2003 2004

Black <5% <5% <5%

White 13% 12% 12%

15

State Data on Black-White Excellence Gaps:Florida Grade 4 Reading

0

2

4

6

8

10

12

2002 2003 2004

% P

assi

ng

Ad

van

ced

Lev

el

African American White

16

State Data on Latino-White Excellence Gaps: Delaware Grade 5 Reading

2002 2003 2004

Latino <5% <5% <5%

White 13% 12% 12%

17

State Data on Latino-White Excellence Gaps: Virginia Grade 5 Reading

05

1015202530354045

2002 2003 2004

% P

assi

ng

Ad

van

ced

Lev

el

Latino White

18

State Data on Latino-White Excellence Gaps: Florida Grade 4 Reading

0

2

4

6

8

10

12

2002 2003 2004

% P

assi

ng

Ad

van

ced

Lev

el

Latino White

19

Summary: State Data

• Little evidence that state’s lauded for shrinking MC gaps have also been able to shrink excellence gaps– … unless the rate of White students achieving

advanced scores decreases.

20

Problems with State Data

• Wide range in …– implementation of NCLB– quality of standards and assessment systems– definition of advanced performance

• Which make state comparisons problematic

• Potential solution: Use state NAEP data

21

Method

• Selected states with the 10 highest and lowest MC achievement gaps

• Examined % of students achieving Advanced level– Disaggregated by race, specifically Black and

White students– Focused on Grade 4 Reading test

22

NAEP AchievementLevel Descriptions

• Basic – Partial mastery of

prerequisite knowledge and skills that are fundamental for proficient work at each grade.

• Proficient– …

• Advanced – Superior performance.

• Basic Grade 4 Reading– [Students] demonstrate an understanding

of the overall meaning of what they read. … They should be able to make relatively obvious connections between the text and their own experiences and extend the ideas in the text by making simple inferences.

• Advanced Grade 4 Reading– [Students can] generalize about topics in

the reading selection and demonstrate an awareness of how authors compose and use literary devices. When reading text appropriate to fourth grade, they should be able to judge text critically and, in general, to give thorough answers that indicate careful thought.

23

2003 NAEP Grade 4 Reading

0

5

10

15

20

25

30

35

40

State (Minimum Competency Gap)

% S

co

rin

g A

dvan

ced

Black White

24

2003 NAEP Grade 4 ReadingBlack-White Excellence Gaps

Note. DC data not included due to size of gap.

0

2

4

6

8

10

12

14

16

25

2005 NAEP Grade 4 Reading

0

5

10

15

20

25

30

% S

co

rin

g A

dv

an

ce

d

White Black

26

2003 NAEP Grade 4 ReadingBlack-White Excellence Gaps

Note. DC data not included due to size of gap.

0

2

4

6

8

10

12

14

State (Minimum Competency Gap)

27

2003 and 2005 NAEP Grade 4 ReadingExcellence Gap Ratios

Note. DC data not included due to size of ratio. WV in 2003 and OH in 2005 had 0% Black students score at the Advanced level, hence the missing data.

0

2

4

6

8

10

12

14

16

18

AK (13)

WV (1

3)

WA (1

6)

HI (19

)

KY (19)

NM (1

9)

OK (22)

DE (23)

IA (2

3)

OR (23)

AL (3

2)

CT (33)

NJ (3

3)

OH (33)

WI (

33)

NE (34)

IL (3

6)

MI (

36)

MN (3

9)

2003 2005

28

Summary: NAEP Data

• States with smaller MC gaps generally have smaller excellence gaps

• But the gaps are still very, very large• Difference between states studied appears

to be the Advanced rate for White students– i.e., Advanced rate for Black students is

uniformly and consistently low, ranging from 0 to 4 compared to 5 to 29 for White students in 2005

29

Conclusions

• This study provides evidence that …– States with smaller MC achievement gaps

generally have smaller excellence gaps.– Excellence gaps in all of the states studied are

quite large and improving gradually, if at all.– Much of the improvement may be due to

declines in performance by White students rather than increases by students of other racial backgrounds.

30

Future Research

• Expand to …– other content areas and assessments– additional years, additional racial and

demographic groups, such as SES, gender, ESL– other outcomes

• Graduation rates, attendance, college attendance and performance, student engagement

• Refine tools for estimating excellence gaps– e.g., use 3-year moving averages for ratios

31

References and Data Sources

• Education Trust, Measured Progress Report– http://www2edtrust.org/edtrust/images/MeasuredProgress.doc.pdf

• Colorado data– http://www.cde.state.co.us/cdeassess/index_assess.html

• Delaware data– http://dstp.doe.state.de.us/DSTPMart/default.asp#Disagg

• Florida data– http://www.fcatresults.com/demog/GetReport.aspx

• Virginia data– http://pen2.vak12ed.edu/cgi-bin/broker?_service=doe_prod&_program

=prodcode.doerp101rcdp001.sas#gr5

• Florida data– http://www.fcatresults.com/demog/GetReport.aspx

32

More References and Data Sources

• Generic NAEP Achievement Level Descriptions– http://nces.ed.gov/nationsreportcard/reading/achieve.asp

• Test- and Grade-Specific NAEP Achievement Level Descriptions

– http://nces.ed.gov/nationsreportcard/reading/achieveall.asp • 2003 and 2005 NAEP Data

– http://nces.ed.gov/nationsreportcard/pubs/stt2003/20044564.asp and http://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=2006452

33

CEEP Contact Information:

Jonathan A. Plucker, Ph.D.Director

509 East Third Street

Bloomington, Indiana 47401-3654

812-855-4438

Fax: 812-856-5890

http://ceep.indiana.edu