Learning Connections 24/7: Assessing Learning across Curricular, Co-Curricular

and Student Development Programs

International Assessment and Retention Conference, St. Louis, MO, June 8, 2007

Margaret Bloom, Vice ProvostChristine Krueger, Core of Common StudiesMark McCarthy, Student Affairs Susan Mountin, Manresa Project

Marquette University

Research Extensive Private, faith-based Mission driven

“Marquette University is a Catholic, Jesuit university. Our mission is the search for truth, the discovery and sharing of knowledge, the fostering of personal and professional excellence, the promotion of a life of faith, and the development of leadership expressed in service to others.”

Learning Assessment at Marquette

The Marquette assessment system is focused on two essential learning outcomes:

Institutional learning outcomes which address the integrated learning expected of every student upon graduation.

Program (undergraduate major and graduate degree) learning outcomes expected of every student at program completion.

Framework for Undergraduate Learning Assessment

Institutional

Learning Outcomes

Major/Program Learning Outcomes

Common Core Learning Outcomes

Co-CurricularLearning Outcomes

9 Knowledge AreasCompetencies

9 Knowledge AreasCompetencies

College CompetenciesDiscipline Competencies

Values, Commitments &Leadership

Required CoursesLearning Objectives Core Courses

Learning Objectives

Services/Programs

Common Definitions and Assessment Process

Each program and unit uses the common MU assessment system vocabulary and has an assessment plan that contains the specified MU common assessment system components.

Program and Unit Flexibility and Decision-Making

Each program faculty/unit staff define learning outcomes, performance indicators, identify and use measures, and

evaluate their data.

Governing Principles of theMarquette Assessment System

MARQUETTE MISSION

MU Assessment System Components

CONSTITUENTS

EVALUATION INTERPRETATION

OF EVIDENCE August 2007

ASSESSMENT COLLECTION,

ANALYSIS OF EVIDENCE Late Spring 2007

LEARNING OUTCOMESFALL 2005

MEASURABLE PERFORMANCE

INDICATORS

DIRECT & INDIRECT

MEASURES Spring 2007

ACTIONS TO IMPROVE LEARNING

Fall 2007

FEEDBACK FOR CONTINUOUS

IMPROVEMENT

Modified from Rogers, G. Rose-Hulman Institute of Technology, 2005

Undergraduate Major

Business Administration

Student

Student Affairs

Institutional

Mission Related ProgramsCore of Common

Studies

Business Core

Major Coursework

MU Assessment Systems: Integrated Levels of Assessment

Learning Outcomes

Performance Indicators

Collect & Assess

Evaluate Measures

Actions to Improve Learning

Broad Participation a Key FactorFaculty and Staff Directly Involved in Development of the

Marquette Assessment System

Institutional LevelUniversity Assessment CommitteeBoard of Undergraduate Studies

Assessment Steering Team36 Faculty/Staff/Administrators

ProgramAssessment Leaders

108 Faculty

Core CurriculumReview Committee

15 Faculty

Student Affairs andCo-Curricular

Program Directors 16 Directors

Marquette (Institutional) Undergraduate Learning Outcomes

Upon completion of an undergraduate degree at Marquette, students are able to…

1. Utilize critical thinking and reflect to effect positive change in themselves, others, and their communities.

2. Exercise just, responsible, and competent leadership in professional, intellectual, and societal contexts.

3. Apply the knowledge and skills of an academic discipline, program, or profession to a career or graduate study.

4. Communicate in modes appropriate to various subjects and diverse audiences.

5. Pursue an integration of knowledge into a comprehensive, transcendent vision of life.

6. Act for social justice within the diverse, global human family.

Majors and Degree Programs

58 Majors

35 Masters Degree Programs

17 Doctoral Degree Programs

3 Professional Degree Programs

Majors and Graduate Programs

Each Academic Program’s Assessment Plan indicates direct and indirect measures.

Located at www.marquette.edu/assessment/plans/index

Direct measures often include senior-level papers, case studies, or projects.

Indirect measure: University Senior Survey contains up to 5 items for students to self-rate on their specific program learning outcomes.

1st Annual Program Assessment Reports due Sept. 31, 2007

Core of Common Studies

Assessment and Integration

University Core of Common Studies Integrated Learning Outcomes

Upon completion of the Core of Common Studies, students are able to…

1. Apply the perspectives, concepts and traditions of multiple disciplines to personal, professional, intellectual and societal challenges.

2. Communicate in modes appropriate to various subjects and diverse audiences.

3. Pursue an integration of knowledge into a comprehensive, transcendent vision of life.

4. Act as responsible members of the global human family, with knowledge of, and respect for, individuals and cultures in their diversity.

Integrated foundational competencies for responsible leadership

Reading comprehension of brief text offering multidisciplinary analysis of global problem

Interpretation of quantitative and qualitative evidence via multiple-choice questions seeking best answer

Communication of diverse content to diverse audiences via 3 randomly assigned tasks:

Summary for senator

Letter to editor of non-US student newspaper

Graph illustrating data extrapolated from reading

Pilot Core Assessment On-line Integrated Assessment Instrument

Pilot Core Assessment Results

25 student volunteers (17 completed)

Preliminary Results

Quantitative/qualitative evidence questions (12%-71% outstanding)

Prose communication (80% > proficient)

Graphic communication (20% proficient)

Understanding the Results

Student focus group

Actions

Refinement of instrument and rubrics

Implementation for incoming seniors

Math across the curriculum??

Student Affairs Assessment

Counseling CenterOffice of Residence LifeRecreational SportsOffice of Student DevelopmentStudent Health Services

Student Affairs DivisionShared Learning Outcomes

As a result of their experiences with the Student Affairs services and programs, students are able to…

1. More clearly understand who they are in regards to mind, body and spirit.

2. Utilize their skills to create positive change in themselves, others and their communities.

3. Collaborate with others to accomplish goals.

4. Engage with others who are different from themselves.

Office of Student DevelopmentLearning Outcomes

After participating in student development programs students are able to…

Use their talents to benefit others.

Interact effectively with people from diverse backgrounds and lifestyles.

Population Based Assessment

Multi-Institutional Study of Leadership (Spring 2006)

Based on the Social Change Model of Leadership (HERI, 1996)

Two groups participated in the study

Random sample of MU undergraduate students

Select sample of students involved in Student Affairs sponsored leadership and service experiences.

MSL Initial Findings Related to Student Development Outcomes

No mean difference reported between the random and student affairs involved groups on pre-test scales for:

Cognitive Development Leadership Efficacy Appreciation of Diversity

Significant mean differences found between groups on the scales for:

Citizenship, Leadership Efficacy, Diversity Discussions Appreciation of Diversity

Program Based Assessment

Leadership Summit One-day urban experience followed by keynote address on integrating faith and learning in action and reflection activities

Assessment Results90% stated that as a result of participation I…

• Am more likely to take action to address needs and problems in the community.

• Have a greater appreciation for different perspectives.• Am more likely to educate others on issues affecting the Milwaukee

community.

Key findings related to outcomes

Leadership Summit assessment results80% stated that as a result of participation I…

• Better understand the importance of reflection.• Learned about methods that leaders use in creating change.

Learning Themes Identified in statements of reflectionDeeper understanding of the broader Milwaukee community.Increased commitment to serve others.Greater understanding of themselves and incorporation of values such as faith, justice, compassion, empathy and hope into work and life.

Assessment Challenges

Whom to assess

Controlling for pre-college experiences and perceptions

Alignment of program goals with assessment strategies

Impact of singular experiences on student development

Complexity of developing and using direct measures

Integrative ProgramsMission Related Programs

Manresa Project

Mission Related ProgramsThe Manresa Project

1. Understand the difference between discernment and decision-making.

2. Practice Ignatian Spirituality as a component of their faith and life choices.

3. Understand Jesuit Global Mission ideals.

4. Explore individual meaning and purpose in life (vocation).

After participation in Manresa Project activities students are able to…

Assessment Tools Used

Direct measures Spiritual Autobiography

Site-supervisor observations

Course assignments

Weekly group reflection

Indirect measures

Surveys: Multi-Institutional Study of Leadership (MSL)

Interviews

Focus groups

The Manresa Project Indirect Assessment Results 2006

Used the MSL data comparing Marquette students with students at other Catholic universities

Marquette students significantly higher on items (outcomes)

Developing understanding of social justice

Strengthening spiritual life

Defining values

Becoming more independent

Knowing how “God has called me” to use my gifts and talents

Integrating service into life

The Manresa ProjectAssessment Challenges

Program has many diffuse opportunities for engagement

First year reading Manresa supported courses Sponsored speakers Seminars, workshops and conferences Retreats

Data collection difficult Relies on external partners (faculty, site supervisors,etc.) Direct contact with small number of students, but a broad

impact across the Marquette community

Faculty working across disciplines on integrating global understandings:

Shared Futures Project with AAC&U

Assessment has been an integrating influence across campus

Inclusion of Mission based learning in the common understanding of Marquette student learning

Assessing 24/7 Learning: Unanticipated Cultural Changes