MPI

MPIMission Perception Inventory

Institutional Characteristics and Student Perception of Mission:What Makes a Difference?

Ellen Boylan, Ph.D. ● Marywood University NEAIR 36th Annual Conference ● Baltimore, Maryland ● Nov. 7 – 10 , 2009

2

What’s in a mission?

3

AcademicExcellence Leadership

Community/Service

Professional Development

Spirituality/Ethical

Develop. OtherDevelop

intellectuallyPrepare for leadership

Sense of community

Develop professionally Develop ethically

Respect & understanding

Develop Acedemically Critical Thinking

Sense of Community

Professional Growth Personal Growth

Research & Expertise

ScholarshipIndependent

Learning Public ServiceSpecialized Expertise

Personal Advisement Diversity

Rigorous & Engaging

Compassionate Service

Truth & Social Justice

Care and Respect

Develop Intellectually

Prepare for Leadership Lives of Service

Prepare for Prof. Careers

Develop Spirit & Heart

Values-Based Education

ScholarshipAnalysis &

ComunicationSense of

CommunityDiciplines & Professions

Morally Responsible

Freedom & Diversity

Educational Excellence

Responsibility as Leaders

Care & Dignity for Community

Career Development Ethical Presence

Historical Remembrance

Comitment to Teach & Learn Service to Society

Professional Competence

Make Connactions

AcademicExcellence Leadership Skills

Involvment in Community

Develop Whole Person

Cultural & Social Growth


Lifelong Learning CommitmentCompassion &

EmpathyDevelop

ProfessionallyPromote Ethics &

Spirituality Human DignitySense of

Community Truth & JusticeDiversity &

Dignity

Intellectual InquiryResponsibility for

DecisionsCommitment to

ServiceCareer

Preparation Personal Growth Diversity


Community/Service





Sense of community




Sense of Community






Rigorous & Engaging



Care and Respect









Morally Responsible

Freedom & Diversity








Make Connactions







EmpathyDevelop




Dignity



ServiceCareer



Community/Service





Sense of community




Sense of Community






Rigorous & Engaging



Care and Respect









Morally Responsible

Freedom & Diversity








Make Connactions







EmpathyDevelop




Dignity



ServiceCareer



Community/Service





Sense of community




Sense of Community






Rigorous & Engaging



Care and Respect









Morally Responsible

Freedom & Diversity








Make Connactions







EmpathyDevelop




Dignity



ServiceCareer


scope of research

PurposeDevelop an instrument to measure

student perception of institutional

mission.

Test instrument reliability.

Uncover constructs (factors).

Observe constructs longitudinally.

GRANTS IN HIGHER EDUCATION FOR IMPROVED ASSESSMENT METHODS

4

scope of research

PurposeDevelop an instrument

Test instrumentreliability.

Uncover constructs.

Observe constructslongitudinally.

Develop a predictionequation.

Research Questions

1. Is the Mission Perception Inventory (MPI) a valid and reliable reliable measure of student perception of institutional mission? 2. What are the factors in the MPI? 3. Do the factors recur in repeated administrations of the revised MPI?

4. Are the factors equally reliable over time?

5. Can a school’s performance be predicted?

5

leaders of public and private institutions alike are thinking about spirituality these days, as the data suggest that's what their students are thinking about, too (Inside Higher Ed, 2009).

There is strong connection between institutional programs and student learning environment (Pascarella, 2001).

…institutions influence levels of engagement on campus as a result of structural features, programs, policies, and organizational culture (Kuh et al., 2005).

background

6

Select the National Survey of Student Engagement (NSSE) as the vehicle for inserting research questions.

Assemble a NSSE consortium to jointly engage in research to explore student perception of mission.

Develop question items; administer them to the consortium as a NSSE attachment.

Test the questions: reliability, factor analysis, and correlation analysis. Repeat annually.

design


7

MissionPerceptionInventory

(MPI)(~19 items)

SenseOf

Mission

Respect for

Diversity

Individual

ValuesSpiritualPractice

Administer

Consortium

mission

questions

(20 items)

derivation of the MPI and subscales

8

sense of mission (10 items = .90) • The mission of this institution is widely understood by students.

• Social and personal development is an important part of the mission.

• Ethical and spiritual development of students is important.

• This institution offers opportunities for volunteering and community service.

• This institution offers opportunities for developing leadership skills.

• There are opportunities for students to strengthen their religious commitment.

• This institution’s religious heritage is evident.

• Professors here discuss the ethical implications of what is being studied.

• As a result of my experience here, I am more aware of my own personal values.

• The mission of this institution is reflected in course offerings.

9

respect for diversity (5 items = .878)

The faculty, staff, and students here…• respect different religions

• respect different races and cultures ………………………………………………………………………………

• Students feel free to express individual spirituality.

• Different sexual orientations are accepted.

• The environment encourages appreciation of diversity.

10

The NSSE survey with attached Mission Perception Inventory (MPI) questions has been administered 182 times to close to 50,000 first-year and senior students at 112 unique institutions across the United States

every year since 2004.

administration


11

Consortia institutions and respondents by year

RespondentsInstitutions First Year Senior

2004 15 2,000 1,8272005 16 1,279 1,3322006 24 2,684 2,8542007 36 4,533 4,331

Catholic 2008 35 4520 5063 Indepen’t 2008 19 6805 6970

145* 21,821 22,377

*(112 unique institutions)

12

Do the factors recur in repeated administrations of the revised Mission Perception Inventory (MPI)

2004 2005 2006 2007 2008

Sense of Mission 8 10 10 10 10

Respect for Diversity 5 5 5 5 5

Individual Actions 2 0 0 0 0

Religious Practice/Spirituality 2 3 3 2 2

Mission Perception Inventory 17 18 18 19 19

Number of items in factors

13

Are the factors equally reliable over time?

Subscales 2004 2005 2006 2007 2008Catholic

2008Indept

cronbach cronbach cronbach cronbach cronbach cronbach

Sense of Mission .87 .88 .88 .90 .91 .88

Respect for Diversity .85 .84 .87 .86 .88 .87

Individual Actions .67 n/a n/a n/a n/a n/a

Religious Practice/Spirituality .62 .55 .54 .62 .62 .64

MPI scale .88 .89 .90 .91 .91 .90

14

(MPI) Mission PerceptionInventory

Report

2008 Mission Perception Inventory Report My University - Sample

Sense of Mission (Cronbach’s alpha=__)

First Year Mean 0.00 Median 0.00 25%ile 0.00 75%ile 0.00 Senior Year Mean 0.00 Median 0.00 25%ile 0.00 75%ile 0.00 Rank (out of x) First Year: Senior Year:

3.8

4.0

4.2

4.4

4.6

Sens

e of

Mis

sion

Sco

re

My University 4.69 4.56

Consortium 4.02 3.97

First Year Senior

Social and personal development of students is an important part of the mission at this institution.

Ethical and spiritual development of students is an important part of the mission at this institution.

This institution offers opportunities for volunteering and community service. This institution offers opportunities for developing leadership skills. As a result of your experience here, you are more aware of your own personal values. The heritage of the founding religious community of this institution is evident here. The mission of this institution is widely understood by students. The mission of this institution is reflected in its course offerings At this institution, there are opportunities for students to strengthen their religious commitment. The faculty at this institution discusses the ethical implications of what is being studied. As a result of my experience here, I am more aware of my own personal values.

Respect for Diversity

(Cronbach’s alpha=__) First Year Mean 0.00 Median 0.00 25%ile 0.00 75%ile 0.00 Senior Year Mean 0.00 Median 0.00 25%ile 0.00 75%ile 0.00 Rank (out of x) First Year: Senior Year:

3.5

3.6

3.7

3.8

3.9

4.0

4.1

Res

pect

for D

iver

sity

Sco

re



First Year Senior

The faculty, staff, and students here are respectful of people of different religions. The faculty, staff, and students here are respectful of people of different races and cultures. Students feel free to express their individual spirituality here. People of different sexual orientations are accepted socially here. The environment here encourages students to develop an appreciation of diversity.

NSSE 2008 Mission Perception Inventory Report Mission Engagement Consortium for Independent Colleges My University- Sample

In spring 2008, My University participated in the National Survey of Student Engagement (NSSE) Mission Engagement Consortium for Independent Colleges, and administered additional questions on mission to the survey’s first year and senior student respondents. Of ___ total survey respondents at My University, there were ___ first year students and ___ seniors.

Reliability and factor analysis of the instrument based on results from administrations of consortium questions have consistently produced a highly reliable scale measure called the Mission Perception Inventory (MPI)*. Further analysis reveals three highly correlated and reliable subscales within the MPI:

Sense of Mission ( 11 items) Respect for Diversity (5 items) Spirituality (2 items)

Also presented are mean comparisons on another scale, NSSE Beliefs and Values (NSSE/BV). The

Consortium administrator created the NSSE/BV scale for this study by a process of selecting, a priori, the question items from the NSSE instrument that seemed most related to mission and then grouping them.

Scale mean comparisons between My University and Consortium schools are presented in the bar graphs. The graphs compare My University first year and senior means compared to Consortium group means on the study’s five scales or benchmarks. The mean scores are shown on the vertical or “y” axis according to the Likert scale range used for items in that scale. Sense of Mission and Respect for Diversity items were rated on a scale from 1 – 5 (1=Strongly disagree, 5=Strongly agree).

In the column to the left of each graph, the rank (high to low) and semi-interquartile range are given for both first year and senior scale means. Note carefully that the y-axis interval is 0.5 for some scales, and 0.1 for others. In the columns under and/or to the right of the graph, the items contained in that scale are listed.

When comparing first year and senior mean scores, recall that the results of the NSSE 2008 produce a snapshot in time of student responses. Results do not indicate growth or increase in scale means from first to senior years, but simply a comparison of those cohort classes at one moment.

Spirituality (Cronbach’s alpha=__)

First Year Mean 0.00 Median 0.00 25%ile 0.00 75%ile 0.00 Senior Year Mean 0.00 Median 0.00 25%ile 0.00 75%ile 0.00 Rank (out of “x” schools) First Year: ___ Senior Year:

0.0

0.5

1.0

1.5

2.0

Rel

igio

us P

ract

ice

Scor

e



First Year Senior

Within the past week, have you participated in a religious service? Within the past week, have you spent time in private prayer or meditation?

2008 Mission Perception Inventory Report


15

2009…

Teagle Foundation Research Continues

• Develop the Mission Engagement Index.• Obtain NSSE 2009 consortia

data • Test using reliability analysis to produce factors• Distribute MPI reports • Compare consortia results• Conduct regression analysis to

create the new index• Use the Mission

Engagement Index (MEI) to assess performance.GRANTS IN HIGHER EDUCATION FOR IMPROVED ASSESSMENT METHODS

NSSE 2008 Benchmark Statistics by Consortia

16

17

Does the Mission Engagement Index describe causal relationships among

variables that affect mission perception? • Dependent variable

(Institution score)• Mission Perception Inventory• Sense of Mission scale • Respect for Diversity scale

• Independent variables (2008 data; need 15 cases per variable*)• Selectivity 15 institutions• Enrollment 30 institutions• Urbanicity 45 institutions• Resident % 60

institutions• Female % 75 institutions• Another? 90 institutions*…a recommended ratio of subjects to IVs of at least 15 to 1 will provide a reliable regression equation

(Stevens, 1992).

18

Selecting IVs for regression analysis

Institution regionCarnegie

Class%

resident%

female

%ethnic/

cauUG

enroll

%Part-time Setting

%Accept

a North U Master's 39 61 95 2119 25 Suburb 74

b Midwest U Master's 33 67 70 928 40 Suburb 57

c South National U 32 68 39 4130 42 Suburb 54

d South Bacc.. Coll 43 57 76 1011 21 Rural 66

e Midwest U Master's 43 57 70 1692 23 Suburb 85

f Midwest Bacc. Coll 42 58 89 978 83 Urban 70

g Midwest U Master's 31 69 78 2985 37 Suburb 93

h West U Master's 32 68 23 1096 60 Urban 94

i North Bacc. Coll 92 58 62 8568 16 Suburb 42

j Midwest U Master's 28 72 91 1502 37 Urban 79

l North U Master's 24 76 79 1320 815 Suburb 60

19

participating institutions by region

Midwest31%

North43%

South17%

West9%

2008 = 2 consortia, 54 institutions

Is there sufficient variability

?

20

“urbanicity” of participating institutions 2008

Rural15%

Suburban54%

Urban31%

Is there sufficient variabilit

y?

21

MPI 2008 Scales:Comparison of means by institution

type

Mean Score Scale Independent Catholic t(df) 2-tailed sig Sense of Mission 3.77 4.04 -2.637(14.356) .019* Respect for Diversity 4.03 4.00 0.330(40.000) NS Spiritual Practice 1.37 1.46 -1.310(14.856) NS MPI 3.46 3.63 -2.296(14.756) .037* NSSE Beliefs & Values 2.57 2.62 -0.882(15.876) NS

*p≤.05

22

Correlation of regression variables with MPI mean

Correlations

usn_setting pct_residence mpi_mean

Pearson Correlation -.203 -.258 .429**

Sig. (2-tailed) NS NS .005

religious_affliation

N 42 42 42

Pearson Correlation .590** -.484**

Sig. (2-tailed) .000 .001

usn_setting

N 42 42

Pearson Correlation -.456**

Sig. (2-tailed) .002

pct_residence

N 42

**. Correlation is significant at the 0.01 level (2-tailed).

23

Deriving coefficientsfor the Mission Engagement Index

(MEI)Model Summary

Model R R Square Adjusted R

Square Std. Error of the Estimate

1 .484a .234 .215 .163 2 .591b .349 .315 .153 a. Predictors: usn_setting b. Predictors: usn_setting, religious_affliation

24

Correlation of regression variables with Mission scale mean

Correlations

usn_setting pct_residence mission_mean

Pearson Correlation -.203 -.258 .486**

Sig. (2-tailed) NS NS .001

religious_affliation

N 42 42 42

Pearson Correlation .590** -.429**

Sig. (2-tailed) .000 .005

usn_setting

N 42 42

Pearson Correlation -.268

Sig. (2-tailed) .086

pct_residence

N 42

**. Correlation is significant at the 0.01 level (2-tailed).

25

Predictive Equation*Institution Predicted MPI Score =(Beta1)*(Value of “setting”)(-0.414)*(1, 2, or 3)(Beta2)*(Value of “institution type”)+ (0.345)*(1 or 2)Constant (3.687)

*Mortenson, T. (1997). Actual Vs Predicted institutional graduation rates for 1100 Colleges and universities. Opportunity, 58.

26

ProposedMission Engagement Index

BenchmarkActual Predicted Residual

StandardizedResidual

CatholicCollege A - Urban

4.26 4.10 0.16 0.60

College B - Suburban 4.17 3.91 0.26 1.00

College C - Rural 4.16 3.57 0.60 2.27

IndependentCollege A - Urban

3.67 3.84 -0.17 -0.64

College B - Suburban 3.47 3.49 -0.03 -0.10

College C - Rural 3.38 3.15 0.23 0.86

CatholicCollege A - Urban

3.66 3.96 0.30 1.64


College C - Rural 3.69 3.14 0.56 3.03

IndependentCollege A - Urban

3.47 3.62 -0.15 -0.80


College C - Rural 3.26 2.79 0.47 2.57

predictors: institution type, setting

MISSION PERCEPTION INVENTORY (MPI)

SENSE OF MISSION

The MEI will compare actual versus predicted scores on mission constructs.

Progress on mission effectiveness can be assessed by comparing MEI outcomes to institutional goals.

27

Sample Mission Engagement Index (MEI)

by Institution Type and Setting

No rest for the weary…

28

For statistical regression, cross validation with a second sample is highly recommended

(Tabachnick, p. 153).

29

ReferencesInside Higher Education (2009). Spiritual accountability. Retrieved from

http://www.insidehighered.com/layout/set/pri...assessment/01/02/2007/News

Kuh, D. G., Kinzie, J., Schuh, J. H., and Whitt, E. J. (2005). Never let it rest: lessons about student success from high-performing colleges and universities. Change, 37(4), 44-51.

Mortenson, T. (1997). Actual Vs predicted institutional graduation rates for 1100 colleges and universities. Opportunity, 58.

Pacarella, E. T. (2001). Identifying excellence in undergraduate education. Change, 33(3), 18-27.

Stevens, J. P. (1992). Applied multivariate statistics for the social sciences (2nd edition). Hillsdale, New Jersey: Erlbaum.

Tabachnick, B. G. & Fidel, L. S. (1996). Using multivariate statics, third edition. New York, NY: HarperCollins.

http://www.insidehighered.com/layout/set/pri...assessment/01/02/2007/News

30

Carlow University Pittsburgh PA Carroll College Helena MT Catholic Theological Union Chicago IL Chaminade University of Honolulu Honolulu HI Chatfield College Saint Martin OH Chestnut Hill College Philadelphia PA Christ the King Seminary East Aurora NY Christendom College Front Royal VA Christian Brothers University Memphis TN Clarke College Dubuque IA College of Mount Saint Joseph Cincinnati OH College of Mount Saint Vincent Riverdale NY College of New Rochelle New Rochelle NY College of Notre Dame of Maryland Baltimore MD College of Saint Benedict St. Joseph MN College of Saint Catherine St. Paul MN College of Saint Elizabeth Morristown NJ College of Saint Joseph in Vermont Rutland VT College of Saint Mary Omaha NE College of Saint Rose Albany NY College of Saint Scholastica Duluth MN College of Saint Thomas More Fort Worth TX College of Santa Fe Santa Fe NM College of the Holy Cross Worcester MA Creighton University Omaha NE De Paul University Chicago IL DeSales University Center Valley PA Divine Word College Epworth IA Dominican College Orangeburg NY Dominican House of Studies Washington DC Dominican School of Philosophy and Theology Berkeley CA Dominican University River Forest IL Dominican University of California San Rafael CA

Donnelly College Kansas City KS Duquesne University of the Holy Spirit Pittsburgh PA D'Youville College Buffalo NY Edgewood College Madison WI Elms College Chicopee MA Emmanuel College Boston MA Fairfield University Fairfield CT Felician College Lodi NJ Fontbonne University St. Louis MO Fordham University Bronx NY Franciscan School of Theology Berkeley CA Franciscan University of Steubenville Steubenville OH Gannon University Erie PA Georgetown University Washington DC Georgian Court University Lakewood NJ Gonzaga University Spokane WA Gwynedd-Mercy College Gwynedd Valley PA Hilbert College Hamburg NY Holy Apostles College & Seminary Cromwell CT Holy Cross College Notre Dame IN Holy Family University Philadelphia PA Holy Names University Oakland CA Immaculata University Immaculata PA Iona College New Rochelle NY Jesuit School of Theology Berkeley CA John Carroll University Cleveland OH John Paul the Great Catholic University San Diego CA King's College Wilkes-Barre PA La Roche College Pittsburgh PA

The United College of San Antonio San Antonio TX Thomas Aquinas College Santa Paula CA Thomas More College Crestview Hills KY Thomas More College of Liberal Arts Merrimack NH Trinity University Washington DC Trocaire College Buffalo NY University of Dallas Irving TX University of Dayton Dayton OH University of Detroit Mercy Detroit MI University of Great Falls Great Falls MT University of Mary Bismarck ND University of Notre Dame du Lac Notre Dame IN University of Portland Portland OR University of Sacramento Sacramento CA University of Saint Francis Joliet IL University of Saint Francis Fort Wayne IN University of Saint Mary Leavenworth KS University of Saint Mary of the Lake/ Mundelein Seminary Mundelein IL University of Saint Thomas St. Paul MN University of Saint Thomas Houston TX University of San Diego San Diego CA University of San Francisco San Francisco CA University of Scranton Scranton PA University of the Incarnate Word San Antonio TX Ursuline College Cleveland OH Villa Maria College of Buffalo Buffalo NY Villanova University Villanova PA Viterbo University La Crosse WI Walsh University North Canton OH Washington Theological Union Washington DC Weston Jesuit School of Theology Cambridge MA Wheeling Jesuit University Wheeling WV Wyoming Catholic College Lander WY Xavier University Cincinnati OH Xavier University of Louisiana New Orleans LA

Our Lady of Corpus Christi Our Lady of Holy Cross College Our Lady of the Lake College Our Lady of the Lake University Presentation College Providence College Queen of the Holy Rosary College Quincy University Regis College Regis University Rivier College Rockhurst University Rosemont College of the Holy Child Jesus Sacred Heart School of Theology Sacred Heart University Saint Ambrose University Saint Anselm College Saint Anthony College of Nursing Saint Bernard's School of Theology and Ministry Saint Bonaventure University Saint Catharine College Saint Charles Borromeo Seminary Saint Edward's University Saint Elizabeth College of Nursing Saint Elizabeth School of Nursing Saint Francis College Saint Francis Medical Center College of Nursing Saint Francis University Saint Gregory's University



Walsh UniversityBennington CollegeBerry CollegeBirmingham-Southern CollegeClark UniversityClarkson UniversityDaemen CollegeFaulkner UniversityFlorida Hosp College of Health SciManchester CollegeManhattanville CollegeMarlboro CollegeMonmouth UniversityMountain State UniversityNaropa UniversityPaul Smiths CollegeSpringfield CollegeStetson UniversityWebster University WorldwideYeshiva University

Bennington College NorthBerry College SouthBirmingham-Southern College SouthClark University NorthClarkson University NorthDaemen College NorthFaulkner University SouthFlorida Hosp College of Health Sci SouthManchester College MidwestManhattanville College NorthMarlboro College NorthMonmouth University NorthMountain State University MidwestNaropa University WestPaul Smiths College NorthSpringfield College NorthStetson University SouthWebster University Worldwide SouthYeshiva University North

Discussion:

31

Ellen Boylan, Ph.D.Director of Institutional Research and Assessment

[email protected]

Office of Planning and Institutional Researchhttp://cwis.marywood.edu/instresearch/activity.stm

Supported by a grant from

http://www.teaglefoundation.org/grantmaking/grantees/assessmentmethods.aspx

Marywood University

Institutional Characteristics & Student Perception of Mission:What Makes a Difference?

mailto:[email protected]

http://cwis.marywood.edu/instresearch/activity.stm

32

MPIMission Perception Inventory

Institutional Characteristics and Student Perception of Mission:What Makes a Difference?

Ellen Boylan, Ph.D. ● Marywood University NEAIR 36th Annual Conference ● Baltimore, Maryland ● Nov. 7 – 10 , 2009

Documents

MPI