Data for Higher Education Policymaking: Issues of Access, Analysis, and Presentation ASHE Graduate Student Public Policy Seminar Hans L’Orange and David

Data for Higher Education Policymaking:

Issues of Access, Analysis, and Presentation

ASHE Graduate Student Public Policy Seminar

Hans L’Orange and David Wright

November 17, 2005

Introductions

National association of state higher education coordinating and governing boards

Broadly, the mission of SHEEO is to assist its members and the states in developing and sustaining excellent systems of higher education

Who/What is SHEEO?

Introductions

Emphasize the importance of planning and coordination for higher education by promoting strategic planning and statewide coordination.

Promote cooperative relationships in the collection and exchange of data and information, development of standard definitions and practices, and conduct of studies.

Formulate and recommend desirable guidelines for state and federal relationships to institutions of higher education.

Encourage studies and other action to advance statewide planning and coordination.

SHEEO’s mission objectives:

Introductions

SHEEO pursues its mission by:

Organizing regular professional development meetings for its members and their senior staff;

Maintaining regular systems of communication among the professional staffs of member agencies;

Serving as a liaison between the states and the federal government;

Studying higher education policy issues and state activities and publishing reports to inform the field; and

Implementing projects to enhance the capacity of the states and SHEEO agencies to improve higher education.

Background

Competitive

For higher ed, systems, institutions, and faculty

Distributed

Technological advancements have “freed” data

• Greater access, expectations, and abilities

Political

Data important as the environment has become increasingly…

Background

Set the context, boundaries, and conditions for decision-making

Ideally, you need data before the conversation begins (not always the case)

Agree on “the facts” before discussing goals, objectives, strategy

Understand “where we are”

Support tough decisions about where you’re going and how to get there

Functions of data within the policy setting

Background

Enhance student enrollment Support student success Maintain financial viability Operate strategically

Plan realistically Allocate resources appropriately Support decision making Assess management outcomes

Renew accreditation Demonstrate accountability

Higher ed challenges that require good data

Background

Outside some context, data are just meaningless points in space and time, without reference to either space or time.

Information relates to description, definition, or perspective (what, who, when, where).

Knowledge comprises strategy, practice, method, or approach (how).

Wisdom embodies principle, insight, moral, or archetype (why).

Gene Bellinger (2004).

http://www.systems-thinking.org/kmgmt/kmgmt.htm

Data, information, and knowledge

Background

Informed Stakeholders(Knowledge)

PolicyPreconceptions

Bias

Experience

Dogma

Beliefs

Anecdotes

“Story”Facts

AnecdotesGraphs/Tables

Data

Analysis

Information

StakeholderNeeds

Needs Analysis

Data Requirements

Assertions of TruthData to

policy

Background

Good data

USEFULAVAILABLE

REVEALING

TIMELY

INTUITIVE

RELEVANT

RELIABLE AND VALID

APPROPRIATE COMPARISON GROUP

USED

REPRESENTATIVE JUST ENOUGH

COMPREHENSIVE

Source: Merrill Schwartz, AGB

Background

The right amount of the right data

“There are two equally effective ways of keeping a board in the dark. One is to provide them with too little information. The other, ironically, is to provide them with too much.”

- From “Building Better Boards,” by David A. Nadler, Harvard Business Review, May

2004, p. 109

Background

Data overload

Inappropriate level of detail

Lack of governance perspective

Lack of strategic relevance

Insufficient distribution

Inattention to time constraints

Reliance on anecdote

Lack of context

Common mistakes in the use of data for policymaking at the institution board level

Data access

General references Measuring Up Higheredinfo.org NCES products

• IPEDS

• Tools: Executive Peer Tool (ExPT), Peer Analysis System (PAS), and Dataset Cutting Tool (DCT)

Census products (CPS, American Community Survey

Regional sites (SREB, WICHE) Mortenson’s Postsecondary

OPPORTUNITY

Data, data everywhere

Data access

Students and learning

NCES: IPEDS (GRS), sample surveys

ACT and College Board

NSSE and CCSSE

Faculty and staff

NCES

• IPEDS (Fall Staff, Faculty Salaries)

• NSOPF

AAUP

CUPA


Data access

Finance and facilities

NCES: IPEDS, NPSAS

SHEEO SHEF (annual), T&F policies survey (triennial)

Illinois State “Grapevine” survey

APPA (core indicators on facilities)

NASBO (fiscal survey)

NACUBO (endowments, comparative stats)

Tuition surveys - Washington HECB, College Board, NASULGC

NASSGAP (state financial aid)

ACE Pell Grant report


Data access

Adult/workforce

NCES: Nat’l Assessment of Adult Literacy

www.higheredinfo.org (under “Special Analyses)

ACE GED report

Still other data sources address:

Governance

Policy (WICHE SPIDO)

Emerging policy issues

Technology, distance education

K-12


Data access

HEA Reauthorization

Major issues are affordability, net price, the impact of student mobility and graduation rates, and “better consumer information”

IPEDS unit record (UR) proposal

Alternatives to a federal UR system

“Huge IPEDS”

Linking state UR systems

• Building on Following the Mobile Student

Recent initiatives that could affect data availability and quality

Data access

39 states have unit record systems

Data access

Expand NCES sample studies

National Student Clearinghouse

Academy One

Data Quality Campaign (K-12)

Any solution must adequately address issues of…

• Burden

• Privacy (FERPA)

• Relevance

Other alternatives to a federal UR system

• Currency/timeliness

• Security

• Trust(translates as “accountability”)

Data analysis and presentation

Rhythm and calendar

Language and writing style

R2 vs. ROI

Academic freedom

Nature and timing of “iterativeness”

Where the conversation is carried out

Academic: journals, conferences, listservs

Political: policy transfer, websites, hallways

The two cultures differ in terms of…


Know the policy!

Your data often contain artifacts of policy

Know the basic analytical conventions

FTE; inflation adjustment

Produce an “elevator ride” document

In addition to report and executive summary

“Write backward”

Recommendations first, methodology last

To maximize your effectiveness in the policy world…


Tell stories For illustration; not the same as basing

policy on anecdote

Paint pictures

To maximize your effectiveness in the policy world…

Data analysis and presentation“Telling stories”

“Despite increases, tuition is below national average”

by Jennifer Peltz

Fort Lauderdale Sun-Sentinel

June 27, 2005

Florida makes a point of keeping tuition low at its public universities and community colleges. But tell that to Sean Chapman.

He's a full-time student with a full-time job, working his way through a Florida Atlantic University finance degree with a position at a brokerage. Tuition and fees have risen 19 percent since he started four years ago, and they're likely to jump another 5 percent in the fall. Chapman's got about $7,000 in student loans already and at least a year to go.

"I just try not to think about it," groans Chapman, 21, a finance major who grew up near Pembroke Pines.

Data analysis and presentation“Telling stories”

“Despite increases, tuition is below national average” (cont’d)

Together, public universities and community colleges educate eight of every 10 college students in Florida, according to federal statistics. The schools are founded on the premise that the state and its students are sharing costs, with the state picking up the lion's share.

But that share is shrinking. Students' portion of the tab grew from about 23 percent to about 28 percent between 1995-96 and 2003-04, the last year for which figures are available from State Higher Education Executive Officers. The nonprofit group's annual studies reflect what's actually collected from students and their families, not what's paid on their behalf by federal, state or college-sponsored grants.

If that doesn't sound like much of a difference, try this: An FAU or FIU student's tuition and fees grew more than five times as fast as the state's per-student spending on higher education, as measured by the officers group. A local community college student's costs grew more than three times as fast. The student costs are calculated for students like Chapman: full-time undergraduates paying a discounted Florida-resident rate…


Pictures worth a thousand words


Pictures worth a thousand wordsU.S. Counties by Educational Needs Index Quartile

Source: www.educationalneedsindex.com

Analysis and presentation

U.S. Counties by Educational Needs Index Quartile

38

31

38

40

48

27

44

26

44

36

34

41

45

23

49

41

45

33

25

29

16

22

31

30

27

41

31

25

30

41

27

29

39

26

32

21

26

26

22

16

23

16

14

16

23

31

26

25

17

25

35

41

29

25

21

37

23

35

20

26

27

25

21

42

17

30

23

33

36

39

52

36

36

31

33

20

28

36

33

21

34

29

24

35

27

43

24

27

33

34

35

26

31

30

29

24

30

29

29

23

21

17

22

24

20

24

21

27

23

26

26

21

21

21

20

15

17

20

24

16

16

26

16

22

22

20

22

21

19

19

20

24

17

19

20

14

29

25

22

27

18

35

31

28

22

20

17

19

25

23

6

11

11

11

12

12

12

12

13

13

13

13

14

14

14

14

14

15

15

16

16

17

17

17

18

18

18

18

18

18

18

19

19

20

21

21

22

22

22

22

23

24

24

25

26

26

27

27

28

28

0 20 40 60 80 100

Alaska

Nevada

TexasNew Mexico

Georgia

Oklahoma

Louisiana

ArkansasMississippi

Haw aii

Kentucky

Alabama

TennesseeIdaho

South Carolina

Arizona

Florida

OregonWest Virginia

Washington

Utah

Montana

CaliforniaOhio

Maryland

North Carolina

Michigan

WyomingColorado

New York

Missouri

Illinois

Delaw areVirginia

Indiana

Vermont

Kansas

South DakotaWisconsin

Nebraska

Maine

North Dakota

New JerseyMinnesota

Connecticut

Rhode Island

New Hampshire

PennsylvaniaIow a

Massachusetts

9th to 12th Grade HS Graduate to College College Entrance to Graduation College Graduates

Source: www.higheredinfo.org, from NCES, ACT Institutional Survey, and IPEDS Graduation Rate Survey


Percent loss at each stage of transition


Pictures worth a thousand wordsHighest Attainment Level of HS Dropouts 10 Years Later

Note: Excludes any credentials earned out of state.

Source: Florida Education & Training Placement Information Program

13,742 High School Dropouts from 1990-1991

Attainment Status unchanged 68.2%*

GEDs 20.8%

Adult High School Diplomas 5%

Vocational Certificates 4.4%

AA ~1%

Bachelors ~1%

College Credit Vocational <1%

Masters <1%

• Dropouts who would have graduated with the class of 1991 attained education credentials, including high school diplomas or equivalencies, at much lower rates than their HS grad counterparts.

U.S. Public Higher Education Enrollments and Appropriations per FTE, Fiscal 1980-2004(Constant 2004 dollars adjusted by SHEEO Higher Education Cost Adjustment)

$6,094

$5,721$5,702

$6,874

Pu

bli

c F

TE

En

roll

men

t

$5,000

$5,500

$6,000

$6,500

$7,000

Ed

uca

tio

nal

Ap

pro

pri

atio

ns

per

FT

E

5M

6M

7M

8M

9M

10M

Note: State and local government support, excluding research, agricultural, and medical.Source: SHEEO SHEF

= Recession




Pictures worth a thousand words (variation on a theme)


A few real-world examples: web site demonstrations

Bringing it all together

Parting shots

You’re entering a grand conversation

Being a good conversationalist means…

Having something to say

Listening

Allowing that the other person just might be right

Acknowledge biases

Disciplinary

Institutional

Personal

Parting shots

Pareidolia - a type of illusion or misperception involving a vague or obscure stimulus being perceived as something clear and distinct.

On the nature of biases in the policy world

Parting shots

A decade-old toasted cheese sandwich said to bear an image of the Virgin Mary sold on eBay for $28,000.

Parting shots

Candles, flowers and a painting of the Virgin Mary embracing John Paul line the section of the Kennedy Expressway underpass believed to hold an image of the Virgin.

Parting shots

In 1996, a cinnamon bun thought to bear the likeness of Mother Teresa was sold in a Nashville bakery.

Parting shots

Contact Info:

David WrightSenior Research Analyst

[email protected]

Hans L’Orange Director, SHEEO/NCES NetworkDirector, Data and Information Management

[email protected]

3035 Center Green Drive, Suite 100Boulder, CO 80301

www.sheeo.org

Documents

Data for Higher Education Policymaking: Issues of Access, Analysis, and Presentation ASHE Graduate Student Public Policy Seminar Hans L’Orange and David