Assessing for Learning: Focusing an Interdisciplinary General Education Revision on Student OutcomesKate Marley, Associate Vice President for Academic AffairsJohn M. Burney, Vice President for Academic AffairsDoane College

Ida Asner, Director of Consulting ServicesLiveText

Presentation Overview

Participants will• Understand the value of sequencing learning skills and

outcomes across four years.• Review a curriculum design that reduces broad distribution

requirements and provides three levels of liberal arts seminars• Learn about a focus on outcomes both for faculty

development and for assessment of student learning• Understand issues of implementing and sustaining an

interdisciplinary curriculum that builds authentic assessment within the course structure

Your issues

• In a one minute paper jot down the biggest obstacles to designing programs based on LEAP principles and using authentic assessment at your own institutions. Then share those obstacles with a colleague.

Timeline for developing a new general education program

2011 -2012: Mission reviewed and revised 2012: Initial writing of new mission-driven general learning outcomes2012-2013: Slowly built new Gen Ed curriculum• Focus on scaffolding student development toward the outcomes• Process of draft and comment in faculty forums• Ongoing faculty education on active learning through workshops and

shareshopsMay 2013: Revised curriculum passed by faculty 2013-2014: Planning and development• Faculty development • Design of new courses• Revision of existing courses and new approval processes• Development of assessment plan, rubric building and testing• Review of assessment management software systems2014-2015: Implementation of the first year of the new curriculum and assessment

Doane’s Mission

Doane College’s mission is to provide an exceptional liberal arts education in a creative, inclusive, and collaborative community where faculty and staff work closely with undergraduate and graduate students preparing them for lives rooted in intellectual inquiry, ethical values, and a commitment to engage as leaders and responsible citizens in the world.

Doane Core Connections (DCC)• Foundational Areas of Knowledge – 1 course in each of 7

interdisciplinary areas• Community and Identity• Mathematical Reasoning• Rhetorical Communication• Global and Cultural Contexts• Scientific Perspectives• Human Creativity• In Search of Meaning and Values

• Liberal Arts Seminars (LAR)• LAR 101 Inquiry Seminar: Learning the Art of Inquiry (freshmen)• LAR 202 Integrative Seminar: Democracy and Diversity (sophomores)• LAR 303 Impact Seminar: Connecting Knowledge to Choices and Actions

(juniors and seniors)

• Experiential Studies 33-34 credits

Traditional outcomes

Through the liberal arts, students will:• Understand foundational areas of knowledge.

• Community and Identity• Mathematical Reasoning• Rhetorical Communication• Global and Cultural Contexts• Scientific Perspectives• Human Creativity• In Search of Meaning and Values

• Develop crucial intellectual skills• Communicate effectively• Use information wisely• Pursue a healthy lifestyle

Advanced outcomes• Build connections of knowledge across various disciplines.

Students will learn to:• synthesize knowledge across foundational areas and specialized

studies;• develop creative and imaginative insights and expressions; and• apply and integrate knowledge collaboratively to solve complex

problems.• Adapt their liberal education to serve and to lead at all levels

of citizenship. Students will learn to:• pursue a refined, empathetic understanding of a multifaceted

world;• orient their own ethical compasses to act accordingly; and • engage with people of varying perspectives to build just societies.

Hallmark: Progressively building to advanced skills across four years through the Liberal Arts Seminars

• Traditional skills introduced in LAR 101• Traditional skills practiced, advanced outcomes

introduced in LAR 202• Traditional and advanced skills applied in LAR 303

LAR 101 LAR 202 LAR 303

First Year Second Year Third Year Fourth YearFall Spring Fall Spring Fall Spring Fall Spring

LAR 101 Inquiry Seminar: Learning the Art of Inquiry – First Year

• A course designed to introduce students to college-level writing, discussion, critical thinking, and critical reading. Faculty will choose a topic for each section in order to help students learn information research skills, to work collaboratively, and to gain an appreciation for interdisciplinary study and multiple perspectives.

LAR 202 Integrative Seminar: Democracy and Diversity – Sophomore Year

• In a collaborative setting, students will apply and integrate knowledge and experiences to examine complex questions related to democracy and diversity from multiple perspectives.

• This course can address far-reaching issues that are enduring or contemporary in areas such as culture and values, science and society, global interdependence, citizenship, or human dignity and freedom.

LAR 303 Impact Seminar: Connecting Knowledge to Choices and Actions – Junior or Senior Year

• This course will develop students’ teamwork and leadership skills to prepare for citizenship or work as they connect theory, practice and experience.

• Students, drawn from multiple fields of specialized study, will collaboratively analyze a complex real-world problem, develop an empathetic understanding of multiple perspectives needed to comprehend the issue, and propose possible solutions.

• Students will be engaged through experiential pedagogies such as service learning.

Key to Effective Scaffolding: Pedagogy is as important as content• Active Learning• Collaborative Learning• Peer-to-peer education• Role Playing Simulations – Reacting to the

Past• Inquiry-Based and Problem-Based Learning• Service Learning• Reflective Writing

Faculty DevelopmentWorkshops:Feb 2012: Technology and Liberal Education – Bryan AlexanderAug 2012 Meeting Students Where They Are – Jennifer McCrickerdAug 2012: Trends in General Education – John BurneyAug 2013: Developing Key Skills and Courses for the Core Curriculum –

Engebretson, Kalbach, Marley and BurneyJan 2014: Student Reflection and Student Portfolios – John ZubizarretaMay 2014: Rubric norming with LAR faculty – Marley and VertinAug 2014: Cooperative Learning as an Effective Pedagogy – Karl SmithJan 2014: Active learning strategies, exploring initial assessment data – Marley

and Vertin

Faculty teams sent to conferences:• AAC&U General Education• Collaboration for the Advancement of College Learning• HLC Assessment• Reacting to the Past Annual Institute

Need for New Direct Assessments

• Had no direct evidence of student achievement in LAR 101• Now adding two more seminars• Faculty buy-in for implementation would be essential• Assessment data could provide evidence of seminar

effectiveness

• New Assessment Team of faculty from across college –• Charged to develop direct assessments for DCC – 9 faculty and dean• Team attended a fall HLC assessment workshop, met weekly

• Scaffold of Liberal Arts Seminars• Tremendous opportunity for DEVELOPMENTAL

assessment of outcomes in seminar series

Assessment Team Identified Common Outcome Threads in Seminar Scaffold

• Critical Thinking• Communication• Reflection, Synthesis, Integration• Collaboration

Intellectual Skills

IntellectualEngagement

Threads for New Direct Assessment:• Intellectual Skills

• Critical Thinking• Communication

• Intellectual Engagement• Learner Self-Awareness• Intellectual Curiosity• Engagement with Diverse Cultures and Communities• Connections Across Disciplines• Connections Among Academic and Co-Curricular Experiences

Rubric building• Two teams of 6 faculty were tasked with building and testing the

rubrics• Rubric criteria and descriptions of development written and

revised• Two rounds of using rubrics with student papers and revision• Intellectual Engagement team also developed reflective writing

prompts for each level of seminar

• Rubrics and norming process shared periodically with faculty• January faculty workshop on reflective writing with John

Zubizarreta• May faculty workshop to scale the norming to all LAR and

interested faculty• Teams provided select sample papers for replication of scoring and

discussion in small groups

Doane College Intellectual Skills Rubric

Criteria Exceeding (4) Achieving (3) Progressing (2) Beginning (1)

Does not meet min.

req. (0)Engage in discovery: Student explores a main topic and articulates a position.

Problem or issue

The central problem or issue is defined clearly along with subsidary

questions that explore the complexity of the problem or issue.

The central problem or issue is clearly articulated along with

indication of the complexity of the issue.

The central problem or issue is stated although complexity of the

issue may not be addressed.

The problem or issue is identified in vague terms and/or distracting sub

problems are identified.

Problem or issue is not identified.

Position - argument, claim, hypothesis,

thesis

Specific position is stated and sustained throughout. It is

thoughtful, creative or unique, and takes into account the complexities

of the issue and limitations.

Position is stated and sustained throughout.

Position is stated but may not be sustained consistently.

Position is stated, but may be simplistic, vague and/or confusing.

No position is stated.

Gather and evaluate facts and assumptions: Student provides context and evidence in support of the position.

Context

Student provides an exceptional treatment of the context for the

problem or issue. Relevant assumptions are presented along

with potential biases.

Student provides a thorough treatment of the context for

problem or issue and is able to articulate relevant assumptions

and potential biases.

Student provides adequate context for the problem or issue, though

may be somewhat lacking in information and/or include

irrelevant information. Some assumptions are identified.

Student provides some context for the problem or issue, though may lack

critical information or irrelevant information may be included.

Assumptions are not likely to be identified.

No context is provided.

Evidence

Relevant, credible evidence is gathered, reported, and interpreted

clearly, fairly, and accurately. In addition student utilizes evidence in a novel and/or esp. effective way.

Relevant, credible evidence is gathered, reported, and

interpreted clearly, fairly, and accurately.

Relevant evidence is gathered and reported accurately. There may be limited interpretation of evidence or

some evidence may not be from credible sources.

A minimal amount of relevant evidence is gathered and reported but

interpretation is probably not provided. Some irrelevant or distracting evidence may also be included, or evidence may

not be from credible sources.

No evidence is provided.

Develops conclusion with relevant evidence: Student arrives at overall conclusion and articulates implications and consequences.

Conclusion and Implications

Student clearly and articulately develops overall conclusion from position and evidence. Provides

nuanced exploration of implications and consequences.

Student develops overall conclusion from position and

evidence. Explores implications and consequences of

conclusion.

Overall conclusion stated. Some statement of implications and

consequences may be included, but may be poorly explored.

Overall conclusion stated. May not flow clearly from position and evidence, implications and consequences not

explored.

No conclusion is offered.

Practices Effective Communication: Student considers purpose, structure, organization, and appropriate conventions to effectively convey meaning to an intended audience.

Purpose and Audience

Demonstrates an exceptional understanding of audience and

purpose that focuses all elements of the work to effectively convey

meaning.

Demonstrates understanding of audience and purpose that

conveys meaning.

Demonstrates an occasionally inconsistent understanding of

audience and purpose that occasionally obscures meaning.

Demonstrates an inconsistent understanding of audience and purpose

that frequently obscures meaning.

Lacks understanding

of audience and purpose.

Structure and organization

appropriate to genre, discipline, medium

Demonstrates a sophisticated organization to achieve maximum

coherence and momentum.

Demonstrates coherent organization with consistent flow

and continuity.

Demonstrates organization but may lack overall coherence, flow,

and/or continuity.

Demonstrates inconsistent organization, lacking overall coherence, flow, and/or

continuity.

Lacks adequate organiz. or structure to

convey meaning.

Conventions approp. to genre, discipline,

medium - such as in writing -

grammar, sentence fluency, spelling

Skillfully communicates meaning with particular clarity, fluency, and

possibly creativity. It is virtually error free.

Employs skillful communication that conveys meaning effectively. It may have errors, but they do

not interfere with understanding.

Uses straightforward communication that conveys

meaning. It may include errors that occasionally obscure meaning.

Uses communication that may include significant number of convention errors

which frequently obscure meaning.

Use of appropriate

communication conventions is so poor as to

obscure meaning

Doane College Intellectual Engagement Rubric

Criteria Exceeding (4) Achieving (3) Progressing (2) Beginning (1)

Does not meet minimum requirements

(0)

Learner self-awareness

Demonstrates the skills of a mature learner, able to reflect in

depth, yielding a nuanced or complex awareness of their learning process intrinsically

motivating them to guide their own development as a learner.

Demonstrates the ability to candidly reflect on own

learning process yielding awareness and takes some responsibility to guide own development as a learner.

Demonstrates the ability to reflect on own learning process but superficial awareness and action

regarding own development as a learner.

Demonstrates a willingness to reflect, may inadequately

identify strengths and weaknesses, may not

understand own role in own learning.

No willingness to reflect on own learning.

Intellectual curiosity

Independently motivated to learn for the sake of increasing own knowledge and understanding, often tenacious, innovative or creative and tend to extend

curiosity beyond the classroom and apply learning in other

settings.

Motivated to acquire knowledge beyond practical

ends, appreciates understanding the

significance and application of knowledge.

Demonstrates interest in acquiring knowledge for practical ends such as

improving own performance, shows interest in applying

knowledge or understanding its significance.

Demonstrates some interest in acquiring knowledge for

practical ends such as exams or grades but not in its

application or significance.

Demonstrates little to no interest in or is apathetic toward the acquisition,

application or significance of

knowledge.

Connections across

disciplines

Integrates and synthesizes examples, facts, or theories from more than one field of study or

takes a multi-disciplinary approach to draw well-reasoned conclusions or creative insights.

Connects examples, facts, or theories from more than one point of view or discipline to

gain insight.

Connects examples, facts, or theories from more than one

point of view or discipline. May contain inaccurate or oversimplified statements.

Provides examples, facts, or theories from one point of view

or discipline.

Does not present any facts, examples or

theories.

Connections among

academic, co-curricular,

professional and life

experiences

Meaningfully synthesizes connections among academic,

co-curricular, professional and/or life experiences to enhance understanding, broaden own perspectives, and translate

learning gains to new situations.

Identifies connections among academic, co-curricular, professional and/or life experiences to enhance

understanding and broaden own perspectives.

Identifies connections among academic, co-curricular, professional and/or life

experiences and can state the importance of those

connections.

Can identify significant academic, co-curriculuar, professional and/or life

experiences but has difficulty identifying connections among

them.

Does not identify signficant academic, co-curricular, professional

or life experiences.

Engagement with diverse cultures and communities

Actively engaged with diverse cultures and communities in

order to develop an empathetic understanding of a mutlifaceted world and is eager to continually examine own perspectives in the

face of new experiences or learning.

Demonstrates an empathetic understanding of a

multifaceted world and is willing to examine own

perspectives in the face of new experiences or learning.

Reflects on how own attitudes and perspectives may be

different from those of other cultures and communities and is willing to consider what can

be learned.

Expresses attitudes and perspectives as an individual, respects differences between themselves and those of other

cultures and communities.

Expresses attitudes and perspectives as an

individual and recognizes differences

between themselves and those of other cultures

and communities.

Evaluation of assessment and portfolio management systems• Team of faculty explored multiple systems

• Identified critical parameters – ease of use, cost structure, reporting capabilities

• Scheduled open demonstrations for faculty and staff

• Collected survey evaluations and recommended LiveText

• Implementation of the system began with LiveText in spring 2014

Assessment for learning

Intellectual Skills• Faculty have students submit an assignment that is already

part of their course plan targeting student development toward these outcomes

• Faculty assess in LiveText using Intellectual Skills Rubric

Intellectual Engagement• Faculty engage students in critical reflection throughout term

to deepen student self-awareness, responsibility for own learning

• Faculty have students respond to a reflective prompt/portfolio that is common across all sections at each seminar level

• Faculty assess in LiveText using Intellectual Engagement Rubric

LAR 101 Reflection Prompt

Critical Reflection assignment instructions: First, read the LAR 101 Doane Core learning outcomes…Second, read William Cronon’s essay “‘Only Connect…’ The Goals of a Liberal Education.”

Finally, in an essay of at least three pages address the following: • Identify two Doane Core learning outcomes that you think will be

important in your own development and connect each to one or more Cronon characteristic. Reflect on the personal importance of these connections, supporting your reflection with specific evidence from your own life.

• Looking to the future, what will you do next to continue growing? How will you act, in choosing courses or activities or experiences, in ways that will help you to identify or explore your interests, clarify your values, and strengthen your intellectual skills?

LAR 202 Reflection PromptsDIVERSITY AND DEMOCRACY:

• What do you see as the critical issues that we need to face to create a well-functioning democracy…?

• Where in your range of academic or co-curricular experiences have democratic processes played a role? What have you learned…

• Describe an experience when you have engaged with people from diverse background that are different from your own. In this case, how are you defining diversity? What assumptions did you have going into the experience? How did the experience challenge your perspectives and assumptions? If it did not…

EDUCATION: • Reread your LAR 101 critical reflection on the essay by William

Cronon “‘Only Connect…’ The Goals of a Liberal Education” and reflect on how your perspective(s) on liberal education may have changed based on the work you have done…

LAR 303 Showcase Portfolio

The comprehensive project in this course will require leadership and collaboration. In order to make the collaboration as effective as possible it is important to know the strengths, skills, and values that each member brings to the team.

To open your portfolio, write a cover letter in application for a team that summarizes your strengths, skills, and values. • What are you most passionate about? • What are your goals for the future?• What can you bring to the team to help it construct an

understanding of a critical issue…• How will your engagement with the team over the critical

issue being addressed in your section both further your own goals and help the team to succeed?

LAR 303 Portfolio EvidenceYour portfolio should showcase examples of your work that provide evidence of your development in the areas described below. Each sample should be introduced with a short reflection addressing the prompts below and any other relevant ideas. • Engage in intellectual inquiry – what is your best example from previous

work of developing an argument supported by relevant evidence in order to analyze a critical issue?

• Communicate effectively – what is/are your most effective achievement(s) in writing, speaking, and/or creative work while in college?

• Build insights across disciplines – where have you connected knowledge from two or more courses in different disciplines…?

• Be a productive team member and work collaboratively to solve a problem – what specifically stands out in your contributions to past team efforts…?

• Pursue an understanding of a multifaceted world – where have you engaged with people from different cultures, beliefs, appearances, abilities, backgrounds, or perspectives, etc in order to challenge and expand your world view…?

LiveText training, implementation, analysis of first term of dataTraining• LiveText staff came to campus during August Faculty week• Faculty received accounts and got into system to explore student and

faculty views

Implementation• Fall term 2014: LAR 101 students on our traditional campus and adult

campuses successfully submitted assignments, instructors assessed them

Analysis• Jan 2015 Rubric Review: Team of faculty reviewed initial data,

conducted inter and intra-rater reliability scoring• Jan 2015 Workshop on Outcomes and Assessment: shared initial

results and debrief of processes

Initial results for intellectual skills development in LAR 101 at traditional and adult campuses

Initial results for intellectual engagement in LAR 101 at traditional and adult campuses

Trad.Only

AdultOnly

Next stepsRevise and practice• Revise rubrics, prompts• Foster faculty practice and collaboration regarding rubrics

Support development• Support development of LAR 202 sophomore seminars and LAR 303

junior/senior seminars• Support development of reflective writing and student-centered

teaching practices

Look ahead• Set targets for student development• Collect evidence of development across the sequence of seminars,

analyze, respond

Mission

Outcomes

Evidence

Response

ASSESSMENT & E-PORTFOLIO SOLUTIONS

• A versatile and extensible learning tool from LiveText

THE FUTURE OF LEARNING

33

ExtracurricularExperience

AcademicExperience

ProfessionalExperience

Key Assessments Organizational Achievements

Prior Studies Hallmark Assignments Resume / Profile

Personal Achievements

Professional MembershipHobbies and Interests

Learning Journals

Instructor/Faculty Assessment

Achievement of University Goals Journal / Life-wide Learning

A SIMPLE WORKFLOW

34

Assignment

Correlation of Standards & Outcomes

Custom Rubrics Aligned to Outcomes

Assessors Dashboard

Ease of use for Faculty and Assessors

Customizable Assessment Tools

MORE WORKFLOWS

40

Assignment

INFINITE POSSIBILITIES

41

Aggregate data on Core Competencies

Disaggregated by College & Gender

Trend Reports

Scheduled Reporting Dashboards

CurateD Portfolios

Questions

• Kate Marley• kate.marley@doane.edu

• John Burney• john.burney@doane.edu

• Ida Asner• ida.asner@livetext.com