The Board’s Role in Overseeing Educational Quality

Peter Eckel, AGBVicki Golich, Metropolitan State University

DenverJeremy Haefner, Rochester Institute of

Technology

Overseeing Educational Quality

The Role of the Board

The Board? (Isn’t ed quality a faculty responsibility?)

• Fiduciary responsibility (parallel to financial audit)

• Responsibility ensure that decision makers have best tools and data available

• Ultimate responsibility for soundness and integrity of the institution’s programs

The Curriculum is the Faculty’s Responsibility…

1. Ensure that the institution has an appropriate set of learning outcomes statements

2. Ensure that efforts to determine the effectiveness of teaching and learning are in place and ongoing

3. Ensure that institutions use the data they collect for improvement

The Board’s Role is to Remind Them of That Responsibility

Peter Ewell. Making the Grade. AGB

Boards and Student LearningIncreasingly concerned;

but continually perplexed

6

What Boards Hear: The Public Story

What Boards See: How Students (and Families) Feel

Discussion: Your Board and Student Learning Outcomes• Does it spend too much; too little; or

just enough time on student learning outcomes?

• What is the relative balance of time discussing learning vs money matters?

• Does the board spend more time on student learning now vs. 5 years ago?

Most Boards Don’t Do Enough

AGB-Teagle Project: • Drake University (IA)• Metropolitan State University of Denver• Morgan State University (MD)• Salem State University (MA)• St. Olaf College (MN)• Rhodes College (TN)• Rochester Institute of Technology (NY)• Valparaiso University (IN)

A quiz: Does your board……

• Know the institution’s learning outcomes goals? – (Know that you have goals???)

• Have a dashboard learning outcomes (direct/indirect outcomes) ?

• Discuss ed quality with faculty?• See and discuss academic program reviews?• Set aside time for educational quality?

Don’t Be Surprised:Common Board Difficulties • Fish (accountants?) out of water

• Impatience/Lack of time: Academic issues take time

• Unfamiliar language of academic assessment (I thought NSSE was a sea monster…. And NIOLA was a town in Hawaii…..)

• Evidence of academic quality is often ambiguous and hard to interpret and use

Two Approaches

•Rochester Institute of Technology

•Metropolitan State University of Denver

14

Academic quality and the Board of Trustees from the

RIT point of view

April 2014

15

AGENDA

Context for a trilogy discussion on academic quality

The Academic Quality Dashboard

The Teagle AGB project

Education Core Committee By-laws

Plan of Work for AY 2013-2014

CONTEXT

The context for the conversation• 2008 – Provost hired with

mandate to engage the board

• Historically BOT very engaged with financial fiduciary role

• 2009 – Great recession begins to manifest

• Concern for academic quality in tough financial time

Partnership with Education committee• Charlie Brown, Chair,

Education Core Committee

• Devised a ‘trilogy’ of conversations on academic quality with the Education Core Committee– 2011: April, November, July

19

Objectives

To ensure academic quality is central to our roles and responsibilities• Be well informed• Understand fiduciary

responsibility is linked to academic quality

To advance academic quality @ RIT• Ensure strategy and

policies are appropriate• Ensure processes are

appropriate and in place• Advocate

To provide input to Middle States• Periodic Review Report

due June 2012• Next campus visit

20

Process

April

• Setting the stage• Understanding

quality through indicators

• The reading assignment

July

• Data and processes

• Discussion• Initial take-aways

and ideas

November

• Principles, processes, and practices

• An Education Committee dashboard

21

Important tool for discussion

Contents organized around indicators• Academic quality• Student-learning

outcomes• Retention and

graduation• Stakeholder input• Program review• Accreditation

22

Some questions you will see• Do we say what and how

much students should learn?

• What kinds of evidence do we collect about student learning?

• Are we benchmarking performance against external standards?

• What progress have we made in addressing recommendations from the last Middle States review?

• Who is responsible for assessment and how it is accomplished?

• How do we use assessment results?

• How does our performance measure up?

• What do student responses tell us about the quality of their academic experiences?

• Are we considering other stakeholder views?

ACADEMIC QUALITY DASHBOARD

Input Indicators

24

Legend: CR = Critical Reading, M = Math, W = Writing

Student Indicators

Faculty Indicators

Mean HS GPA Top 10%

Mean ACT Composite

Mean SAT (CR+M+W)

2010 32.4%

2011 36.8% 2012

35.3%

2010 1757

2011 1785

2012 1800

2010 27

2011 27

2012 28

76.3 76.4

78.9

75.0

76.0

77.0

78.0

79.0

80.0

2010 2011 2012

% of full-time T/TT Instructional Faculty with terminal degrees

9.5 9.6

10.310.5 11.0

10.0

6.0

8.0

10.0

12.0

14.0

2010 2011 2012

% AALANA T/TT Faculty

% A ALANA T/ TT FacultyKRA Goal

31.0

31.6

32.2

31.5

32.0

32.0

30.0

30.5

31.0

31.5

32.0

32.5

2010 2011 2012

% Female T/TT Faculty

% Female T/ TT FacultyKRA Goal

2010 90.0

2011 90.1

2012 90.2

25

Environmental Indicators Student Indicators

1st Yr Persistence

2nd Yr Persistence

Graduation Rate

60 65 70 75 80 85 90 95 100

87.786.3

8988

77.880.7

78.7

66.469.8

67.770

Undergraduate

1st Yr Persistence 2010-112011-122012-13

Graduation Rate 2010-112011-122012-13

60 65 70 75 80 85 90 95 100

76.377.4

76.6

68.973.9

71.5

Graduate

26

Environmental Indicators, cont’d

Student Indicators

NSSE Summary Questions

2006-07 2008-09 2010-111.00

1.75

2.50

3.25

4.00

3.27 3.23 3.27

4-Point Scale (1=Poor and 4 = Excellent)

How would you evaluate your entire educa-tional experience at this institution?

2006-07 2008-09 2010-111.00

1.75

2.50

3.25

4.00

3.03 3.06 3.18

4-Point Scale (1=Definitely No and 4=Probably Yes)

If you could start over, would you go to same institution you are attending now?

27

Environmental Indicators, cont’d Faculty Indicators

*Data source: IPEDS

2010-11 2011-12 2012-130%

20%

40%

60%

80%

100%

49.1% 49.1% 47.5%

66.4%77.7% 74.0%

Section Sizes < 20

Undergraduate Graduate

2010-11 2011-12 2012-130%

10%

20%

30%

40%

50%

60%

21.6% 22.9% 23.7%19.1% 19.5% 20.6%

57.2% 56.6%53.5%

% Sections Taught*

FT Non-TT faculty PT Non-TT faculty T/TT faculty 2010-11 2011-12 2012-130%

1%

2%

3%

4%

5%

3.6%4.1% 4.4%

0.3%

1.2%

0.4%

Section Sizes > 50

Undergraduate Graduate

2010-11 2011-12 2012-130.0%

25.0%

50.0%

75.0%

100.0%95.9% 96.0% 95.9%

Satisfactory = meeting, exceeding, or outstanding

% All FT Faculty Rated 'SATISFACTORY' in Teaching Effectiveness *

*2012-13 Preliminary (does not include NTID). Final available in Nov 2013

2010-11 2011-12 2012-1312.012.513.013.514.014.515.0

13.0 13.0

14.1

Undergraduate + GraduateStudents per Faculty Ratio

*Includes undergraduate + graduate sections

Output Indicators

28

Placement

2009-10 2010-11 2011-12 0%

10%20%30%40%50%60%70%80%90%

100%

20.8%13.8% 11.0%

72.2%77.2% 81.1%

93.0% 91.0% 94.6%

Undergraduate

% Grad School % Employed Total Employed + Grad School

2009-10 2010-11 2011-12 0%

10%20%30%40%50%60%70%80%90%

100%

13.3%6.1% 5.7%

84.4% 85.4% 84.6%

97.7%91.5% 95.1%

Graduate

% Grad School % Employed Total Employed + Grad School

Noteworthy: in 2011-12 HIGH response rate of 85.6%

Output Indicators, cont’d

29

Employer Satisfaction

Learning Outcomes

* 5 point Likert scale, 5 = Excellent and 1 = Poor

92.992.7

91.4

88.0

90.0

92.0

94.0

96.0

2009-10 2010-11 2011-12

Employer Co-op Satisfaction (% would hire permanently)

4.39 4.38 4.39

4.00

4.20

4.40

4.60

4.80

5.00

2009-10 2010-11 2011-12

Overall Quality of Student's Performance*

2009-10

2010-11

2011-12

KRA Goal

0% 20% 40% 60% 80% 100%

40%

56%

75%

65%

*Benchmark Levels = meeting or exceeding

% of All Programs that Achieved Bench-mark Levels*

2009-10

2010-11

2011-12

KRA Goal

0% 20% 40% 60% 80% 100%

90%

80%

84%

85%

% of Programs that Use Assessment Results and Pro-cesses to Guide Planning and Improvement

30

Q05. How well did the highest education from RIT prepare you for each of the following?

Current work status

Getting desired job or into adv degree program right after grad

Further graduate education

Responding to new career opportunities

Deepening my understanding and comm to personal dev

Contributing to my community

PCUAD RIT 2011 RIT 2009

Poor preparation

Fairpreparation

Goodpreparation

Excellentpreparation

*PCUAD = Private College and University Alumni DirectorsAlumni Relations leaders from 40 mid-size to larger-size private colleges and universities (50k-225k alumni) get together twice a year to share the benchmarking data and compare resources and results.

31

Q07. How important was each of the following to your experience as a student, and how well did RIT do at providing them?

Academics and classes

Skills and training for career

Relationship with the faculty

Traditions or values learned on campus

Opportunity to interact with alumni

Attending athletic events

Importance Performance

Not Important

Poor

SomewhatImportant

Fair

Very Important

Good

CriticallyImportantExcellent

RIT 2011

ECC BY-LAWS, 2ND READING

Section 1

Section 2

ECC PLAN OF WORK: 2013-2014

ECC Plan of Work: 2013-2014

Introducing the Board toEducational Quality Oversight: I

• Years of providing information from External Program or Accreditation Review of Academic Programs (most on a 7-year cycle)– Results of most recent year + – 1-year follow up to answer the question: What have we

done to address uncovered weaknesses• No real understanding of what such reviews

entailed– Provided extensive information about how these

reviews are conducted and the role of faculty/staff in each

Introducing the Board toEducational Quality Oversight: II

• Provided extensive detail regarding annual peer review processes related to assessment of student learning outcomes

• Introduced Board to some of the 21st teaching and learning/course redesign efforts in place

• Reported on status of online/hybrid/in-class course delivery

• In-progress efforts to identify University-level graduate learning outcomes

Introducing the Board toEducational Quality Oversight: III• Fall Board Retreat

– Prepared Report – Executive “Summary” (19 pages long) + 22 pages of detailed appendices

– Addressed Peter Ewell’s observations in Making the Grade (2006)

– How MSU Denver• Is embedding High Impact Practices• Adds value to students (modified open enrollment

institution)

– Devoted half-day to discuss

Trustee Jeopardy

Our Students

Graduation/ Pass Rates Our Faculty Our

Programs Metrics

10 10 10 10 10

20 20 20 20 20

30 30 30 30 30

40 40 40 40 40

50 50 50 50 50

10 Point Questions & Answers

• Students– What percentage of MSU Denver students are

transfers?• 60%

• Graduation Pass Rates– True or False: 50% of MSU Denver transfer

students graduate within 7 years?• False: MSU Denver transfer students graduate within 9

years.

10 Point Questions & Answers

• Faculty– How many tenured faculty does MSU Denver

employ? 310 281 223?• 281; MSU Denver also has 179 tenure track

(probationary) faculty members

• Our Programs– What does “SAI” stand for?

• Supplemental Academic Instruction

10 Point Questions & Answers

• Metrics– What % of Latino/Hispanic Enrollment does MSU

Denver need to achieve in order to achieve HIS designation?

• 25% Full Time Equivalent Students (FTES)

What Now?

• Deep focus on retention and how to determine which things we do matter as we seek to enroll, retain, and graduate students

• Academic and Student Affairs Leadership Team (also includes ITS) in constant conversation about what we are doing

• Improving metrics

Board Oversight of Student Learninghttp://agb.org/improving-board-oversight-student-learning

Lessons for Progress1.Ensure sufficient institutional

assessment capability

2.Start with what you already have

3.Make academic quality a priority of the board and institutional leaders

4.Attach the effort to other activities

Peter Eckel, “Lessons Learned about Student Learning: 8 Test Cases,” Trusteeship, Jan-Feb 2014

Lessons for Progress5. Educate the board on

education

6. Find the right focus

7. Allow for targeted deeper dives

8. Develop new board processes and use time differently

9. Deepen the board’s engagement with faculty

Peter Eckel, “Lessons Learned about Student Learning: 8 Test Cases,” Trusteeship, Jan-Feb 2014

Thank you Peter Eckel (Peter@agb.org)Vicki Golich (vgolich@msudenver.edu)Jeremy Haefner (jahpro@rit.edu)