INDIANA UNIVERSITY Outcomes Assessment: Learning from Capstone Portfolios Frank DiSilvestro, Janet Johnson, Henry Merrill, Marjorie Treff American Association

I N D I A N A U N I V E R S I T Y

Outcomes Assessment: Learning from Capstone Portfolios

Frank DiSilvestro, Janet Johnson,

Henry Merrill, Marjorie Treff

American Association for Adult and Continuing Education

Oklahoma City, OK; November 19, 2015


This session presents content analysis research on 100 Capstone Portfolios completed by graduates of Indiana University online M. S. in Adult Education during 11 years (Fall 2007 – Spring 2015).

Purpose of this research is to determine the impact of the Capstone Portfolio on student learning and the program curriculum.


Introduced to curriculum in 2004, the goals of the Capstone Portfolio course were to:

• Provide the opportunity for students to create a portfolio documenting the quality of their work in courses and the quality of their understanding and application of adult education principles, theory and best practices;

• Provide the opportunity for students to reflect on their growth and development, and describe ways in which their learning was instrumental and/or transformative


Outcomes assessment has become a major focus in higher education during the last two decades.

Palomba and Banta define assessment as “… the systematic collection, review, and use of information about educational programs for the purpose of improving student learning and development” (1999, p. 4).


The focus on assessment stimulated focus on teaching and learning, called the scholarship of teaching and learning (SoTL).

Focus on the learning-teaching/facilitating transaction invites posing questions about and reflecting on these processes to improve student learning and, by sharing in the larger teaching commons, advancing the profession overall (Huber & Hutchings, 2005).


This summative evaluation research process provided a lens for insight into both the students’ documentation of what they’ve learned and, to some extent, on the program level curriculum.

It is a summative process created by content analysis of 100 capstone portfolios completed from 2007-2015.


Capstone Portfolios include:

– A theme or metaphor for the Portfolio– Introductory and Closing Framing

Statements– Minimum of 8 artifacts selected from

completed coursework with reflective caption statements


The caption statements reflect the meaning students make of their learning experience. This study focuses on student narratives: it analyzes their reflective statements about their learning, and how they made meaning of the experience of completing the degree program.

It is this “making meaning of the experience” that provides an opportunity to investigate the types of learning that occur.


Definitions of reflective learning typically emphasize purposeful critical analysis of knowledge and experience in order to achieve deeper meaning and understanding.

Their common premise is that of returning to an experience to examine it, deliberately intending that what is learned may be a guide in future situations, and incorporating it into one’s existing knowledge.


Methods -1:– Researchers used a basic interpretative

qualitative study approach using content analysis

– Researchers analyzed 100 selected student portfolios completed from Fall 2007 through Spring 2015.

– We identified the presence of specific words, phrases or concepts from the student Capstone Portfolios.

– We analyzed student reflective statements about their learning and drew inferences based on student narratives and artifacts.


Methods -2:– We also analyzed student descriptions

about how they made meaning of the Capstone experience.

– As authors, we are aware that we are constructing a larger story of the outcomes and meanings of completing the degree based on the portfolios included for analysis.


SUMMARY OF THE FINDINGS FROM ACE PORTFOLIOS

Gender: Women (77%); Men (23%). This reflects general enrollments in the program.


Type of bachelor’s degree preparation:21 Arts & humanities degrees16 Social sciences degrees 9 Natural science & math19 Business 7 Education21 General Studies (BGS) 19 Professional & other*

* Includes nursing, dietetics, radiography, telecommunication/journalism, organizational leadership, labor studies, civil engineering, computer science and ministry


Other degrees earned prior to enrollment:

• Associates degrees 10• Masters degrees 7• Doctoral 1


Employment sectors represented at time of completing portfolio:

• K – 12 education 5

• Post-secondary education 55 (13-teaching; 9-academic advising; 33-administration)

• Health/clinical education 8

• Non-profit organizations 5 (2-administration; 3 workforce learning/training)

• Business 19 (6-training; 4-HR administration; 9-general administration)

• Military/government 5 (3- workforce learning/training; 2 general administration)

• Self-employed/consulting 2

• Paralegal 1


Distribution of artifacts from courses:

590 artifacts – Selected from required core courses (not including Capstone Portfolio course) 262 artifacts – Selected from ACE elective courses

179 from adult education elective courses 83 from outside/transfer elective courses

852 artifacts total in 100 capstone portfolios


Plans for lifelong learning indicated:

• Formal/informal professional

development: 94

• Leisure activities: 17

• Additional Masters degree: 1

• Consider doctoral degree: 24


Civic engagement examples

• Habitat for Humanity• Career research volunteer at university• Literacy & GED tutor• Juvenile Correction Facility Volunteer• Upward Soccer K3-K5 Coach, Cook and server local ministry,

Volunteer Life Coach at Prison Ministry.• President of the Zug International Women’s Club• Vice-President, County Tourism Board• Volunteer Worship Ministries Coordinator• College mentor for The Extra M.I.L.E. mentoring program; also

volunteer at local jail to mentor female inmates• The Right Brain Initiative - Photography and Imagination

Interviews Volunteer, Adelante Empresas- Outreach Volunteer


Findings: The reflective statements discussed are representative of the 100 portfolios studied. The reflective statements indicate that the goals of the Capstone course were met in that students (1) documented the quality of their work in

courses and the quality of their understanding and application of adult education principles, theory and best practices; and

(2) reflected on their growth and development, and described ways in which their learning was instrumental and/or transformative.


SoTL Implications:

• We have begun to increase opportunities for students

to engage in reflective learning by incorporating

learning journals in several required courses

• Adapted Brookfield’s (2015) Critical Incident

Questionnaire (CIQ) for use at the end of each day of

a 3-day intensive face-to-face seminar (D521

Participation Training). After reviewing journals and

CIQs, we share critical questions and insights with

the entire class, and facilitate processing and

analysis.

Documents

INDIANA UNIVERSITY Outcomes Assessment: Learning from Capstone Portfolios Frank DiSilvestro, Janet Johnson, Henry Merrill, Marjorie Treff American Association