Assessing Intrapersonal and Interpersonal Competence

Assessing Intrapersonal

and Interpersonal Competence:

Fragments from the Field

George Kuh

Senior Scholar, NILOA

Supporting Students’ College Success Summit

Washington, DC

April 13, 2017

http://www.learningoutcomesassessment.org/index.html


Overview

A word about NILOA

Partial answers to three questions

Discussion



NILOASince 2008, our mission is to discover and disseminate effective use of assessment data to strengthen student learning and advance high quality, actionable assessment work.● SURVEYS ● WEB SCANS ● CASE STUDIES ● FOCUS GROUPS ●OCCASIONAL PAPERS ● WEBSITE ● RESOURCES ● NEWSLETTER ●LISTSERV ● PRESENTATIONS ● TRANSPARENCY FRAMEWORK ● FEATUREDWEBSITES ● ACCREDITATION RESOURCES ● ASSESSMENT EVENTCALENDAR ● ASSESSMENT NEWS ● MEASURING QUALITY INVENTORY ●POLICY ANALYSIS ● ENVIRONMENTAL SCAN ● DEGREE QUALIFICATIONSPROFILE/TUNING ● DQP CAMPUS COACHING ● ASSIGNMENT DESIGNAND ONLINE LIBRARY ● EMERGING LEARNING SYSTEM NETWORKING

www.learningoutcomesassessment.org

http://www.learningoutcomesassessment.org/

Three Questions

1. How does assessment of the competencies identified in this report intersect with the current state of assessment in higher education institutions?

2. What processes are involved in turning data from assessments of these competencies into improvement in higher education? What are the challenges, opportunities, or potential feedback loops to make this work consequential?

3. How might this improvement process work, specifically, when considering assessments of the identified competencies in higher education?

Do we measure what we value?

or

Do we value what we measure?

Wise decisions are needed about what and how to measure the proficiencies demanded by the 21st century

1. How does assessment of intra- and interpersonal competencies intersect with the current state of assessment in higher education institutions?

-- NAS report generally accurate – most institutions do very little assessment of these outcomes despite statements valuing such behaviors by institutions and employers

-- AAC&U Essential Learning Outcomes (ELOs) and Lumina Degree Qualifications Profile (DQP) only indirectly address interpersonal and intrapersonal competence

1. How does assessment of intra- and interpersonal competencies intersect with the current state of assessment in higher education institutions?-- NAS report generally accurate – most institutions do very little assessment of these outcomes despite statements valuing such behaviors by institutions and employers

-- AAC&U Essential Learning Outcomes (ELOs) and Lumina Degree Qualifications Profile (DQP)

-- AAC&U VALUE rubrics re teamwork, ethical reasoning and institution-specific rubrics (e.g., Tidewater CC personal development rubric)https://list.aacu.org/sites/default/files/files/VALUE/VALUE_CulturalAndSocialUnderstandingRubric.pdf

https://list.aacu.org/sites/default/files/files/VALUE/VALUE_CulturalAndSocialUnderstandingRubric.pdf

Valid Assessment of Learning in Undergraduate Education

(VALUE) Rubrics Inquiry and analysis Critical thinking Creative thinking Written communication Oral communication Reading Quantitative literacy Information literacy Teamwork Problem solving Civic knowledge and engagement Intercultural knowledge and competence Ethical reasoning and action Foundations and skills for lifelong learning Integrative learning

1. How does assessment of intra- and interpersonal competencies intersect with the current state of assessment in higher education institutions?

-- NAS report generally accurate – most institutions do very little assessment of these outcomes despite statements valuing such behaviors by institutions and employers

-- AAC&U Essential Learning Outcomes (ELOs) and Lumina Degree Qualifications Profile (DQP)

-- AAC&U VALUE rubrics re teamwork, ethical reasoning and institution-specific rubrics (e.g., Tidewater CC personal development rubric)https://list.aacu.org/sites/default/files/files/VALUE/VALUE_CulturalAndSocialUnderstandingRubric.pdf

-- Behavioral observations and performance assessments via internship/job evaluations, high-impact practices, etc.

https://list.aacu.org/sites/default/files/files/VALUE/VALUE_CulturalAndSocialUnderstandingRubric.pdf

High-Impact Activities

First-Year Seminars and Experiences Common Intellectual Experiences Learning Communities Writing-Intensive Courses Collaborative Assignments and Projects Inquiry & Creative Activity Diversity/Global Learning Service/Community-Based Learning Internships and Field Placements Capstone Courses and Projects

ePortfolio

U of Iowa Student Employment Project“Guided Reflection on Work” (GROW)

Supervisors had two structured conversations with every student employee during spring semester and coded conversations into the following categories:

How the job and academics complement each other (“How is your job fitting in with your academics?”)

Transfer between work and academics (“What are you learning here at work that is helping you in school?”)

Transfer between academics and work (“Are you learning anything in class that you can apply here at work?”)

Transfer between work and future career (“Give me a couple of examples of things that you are learning here at work that you will be using in your future profession?”)

IOWA GROW® participants were more likely (p<.01) to agree/strongly agree that their job was helping them attain the following outcomes of student employment:

1 My supervisor helps me make connections between my work and my life as a student.

6My job has helped me develop more effective time management skills.

2 My job has helped prepare me for

the world of full-time employment.

3 My job has helped me improve

my written communication skills.

4 I can see connections between my job and major/coursework.

7 My job has helped me improve my oral communication skills.

8 My job has helped me develop conflict resolution skills.

9My job has helped me use critical

thinking skills to form opinions and

solve problems.

5 My job has helped me learn about career options. 10

Because of my job, I am able to work effectively with individuals with a variety of backgrounds, experiences, and cultures.


-- The classic assessment cycle

The Assessment Cycle


-- The classic assessment cycle

-- Outcomes statements aligned with course requirements

-- Curricular mapping -- Assignment design

Design assignments to elicit the student behaviors that demonstrate the desired outcomes.

Questions? Comments? Click

here!

Welcome to NILOA’s Assignment Library!The purpose of this website is to provide a searchable online library of collegiate-level course assignments in a wide variety of academic disciplines that link to one or more proficiencies in the Lumina Foundation’s Degree Qualification Profile (DQP). This page allows you to:

Browse assignments Borrow and adapt an assignment to fit your needsSubmit an assignment to be included in our libraryConsult a resourceComment on and ask questions about assignmentsStart a conversation on your own campusGive us feedback!

Learn about the DQP Submit an assignmentBrowse our library Look for resources

What do you want to do?


-- Campaign to mobilize outside-in pressure (accreditors, employers, professional societies) to underscore the value of interpersonal and intrapersonal competence and compel institutional leaders, faculty and others to cultivate these outcomes, and collect and make consequential use of assessment results

-- Measurement approaches (e.g., rubrics, performance assessments) must be developed and widely used to document these competencies

Operational Verbs to Assess Teamwork Skills

Verbs that describe executive functions

lead, operate, administer, control, coordinate, manage, plan, navigate

Verbs that describe forms of deliberative activity

argue, challenge, debate, defend, justify, resolve, dispute, advocate, persuade

Verbs related to modes of communication, in groups

collaborate, contribute, negotiate, feedback

Verbs describing how to explain a position or problem

articulate, clarify, illustrate, interpret, translate, elaborate, elucidate


Verbs describing the cognitive functions underlying analytical reasoning

compare, contrast, differentiate, distinguish, formulate, map

Verbs describing information processing

calculate, determine, estimate, manipulate, measure

Verbs describing ways to format information

arrange, assemble, collate, organize

Verbs describing rethinking or reconstructing

accommodate, adapt, adjust, improve, modify, refine, reflect, review


Verbs describing information processing

calculate, determine, estimate, manipulate, measure

Verbs describing ways to format information

arrange, assemble, collate, organize

Verbs describing rethinking or reconstructing

accommodate, adapt, adjust, improve, modify, refine, reflect, review


-- Outside-in pressure (accreditors, employers, professional societies) is

needed to compel institutional leaders and faculty to take action and make consequential use of assessment results

-- Measurement approaches (e.g., rubrics, performance assessments) must be developed and widely used to document these competencies

-- Bundling interventions (learning communities, peer mentoring, grit/conscientious assignments)

-- Problem-based and small group assignments that actively involve students in applying what they are learning


-- Activities such as frequent low-stakes testing, feedback, and autobiographical writing help facilitate self-regulated learning and minimize stereotype threat

(see Singer-Freeman, K. & Bastone, L. (2016). Pedagogical choices make large classes feel small (NILOA Occasional Paper No. 27). Urbana, IL: University of Illinois and Indiana University, National Institute for Learning Outcomes

Assessment.http://www.learningoutcomesassessment.org/documents/Occasional%20Paper_27.pdf

http://www.learningoutcomesassessment.org/documents/Occasional Paper_27.pdf


-- an essential component is a common or widely shared documentation/record keeping system:

Comprehensive Student Record (AACRAO & NASPA)U of Central Oklahoma Transformative Student

Recordhttp://www.learningoutcomeassessment.org/documents/Hynes_Assessment_in_Practice.pdf

-- ePortfolio, a promising pedagogical innovation and meta framework (see Table 4-1, pp 106-108)

http://www.learningoutcomeassessment.org/documents/Hynes_Assessment_in_Practice.pdf

ePortfolios make student learning visible – to students themselves, to their peers and faculty, and to external audiences.

ePortfolio Resources

Batson, T. (2002). The electronic portfolio boom: What's it all about? Campus Technology.https://campustechnology.com/articles/2002/11/the-electronic-portfolio-

boom-whats-it-all- about.aspx

Cambridge, D. (2010). Portfolios for lifelong learning and assessment. (Chichester, United Kingdom: Wiley.

Clark, J.E., & Eynon, B. (2011/2012). Measuring student progress with e-portfolios. Peer Review, 13(4)/14(1), 6–8.

Eynon, B. & Gambino, L. (2017). High impact ePortfolio practice: A catalyst for student, faculty and institutional learning. Sterling, VA: Stylus.

Eynon, B., Gambino, L.M., & Török, J. (2014). What difference can ePortfolio make? A field report from the Connect to Learning Project. International Journal of ePortfolio, 4(1), 95–114.

https://campustechnology.com/articles/2002/11/the-electronic-portfolio-boom-whats-it-all- about.aspx


-- Faculty and institutional leaders must value interpersonal and intrapersonal competence and address these outcomes explicitly in institutional and program learning goals, curricular mapping, assignments, and out-of-class learning experiences, and in efforts to document student attainment of these competencies

Faculty Engagement in Assessment is Key, But Non-Trivial Challenges Must Be Addressed

Concern that it is a fad

Concern that it is externally imposed

Concern that it is a top-down administrative mandate

Concern over the corporatization of higher education

Challenges to academic freedom / shared governance

Faculty Engagement in Assessment is Key, But Non-Trivial Challenges Must Be Addressed

Misalignment in language / difficulty in communicating warrant and purposes

Fear over misuse of results and data

Fear over lack of use of the data

Skepticism over the ability to assess student learning

Worries that it might affect standards

Concerns related to simplicity / complexity

Concerns over workload of faculty / that it is an add-on

Fundamentals

A sea-change is needed to move interpersonal and intrapersonal competence into the core of desired collegiate learning outcomes for ALL students

Engaging pedagogical approaches used extensively in-class and out-of-class that promote self-regulation and reflection combined with frequent, timely feedback to insure students recognize, practice, and demonstrate these competencies

Alignment of curricular and other learning experiences (e.g., applied, experiential learning, high-impact practices) that require students to practice and apply these competences in multiple contexts

Principles for Consequential Use of Evidence

1. Backward design. That is, begin assessment activity with the end use in mind. What practical questions are of greatest interest to potential partners—faculty, administrators, staff, and intended internal and external end users? How might they use the results? What do partners and end users expect to find?


2. Identify users of evidence when designing assessments and its anticipated uses at different levels within an institution. What institution-wide or program-level goals and audiences can be influenced by results? Present assessment and the questions it explores in a way that persuades people that the results of the inquiry will have practical value in their work, at their level.


3. Leverage the accreditation process for meaningful campus action to improve student learning. Ensure that the campus benefits from all the time and energy that accreditation requires by framing this work as more than an act of compliance—as a process to move the campus toward improvement in learning, teaching, and institutional performance.


4. Connect assessment work to related current national initiatives and projects. Membership associations, national organizations, foundations, and other collective initiatives provide a broader context in which to embed assessment work, to learn from the work of other institutions, and to increase the impact, legitimacy, and value of assessment results.


5. Link assessment work to campus functions that require evidence of student learning (e.g., program review, center for teaching & learning). Action and improvement are more likely if (a) assessment work is sustained, not episodic: (b) campus structures and processes are in place to report results, discuss implications, and plan needed action, and (c) key end users and partners are engaged in the process.


6. Effective student learning outcomes assessment is a continuous process. Taking evidence-informed action is important, but it also marks the beginning of a new cycle. What was the impact of the change? Did student performance improve? Follow-through and taking time to evaluate the impact of evidence-based change is essential in fostering a culture that supports the meaningful use of assessment results.


7. The value of assessment work is a function of the extent to which results are used. Are assessment priorities shaped by questions and concerns for student success that anticipate the productive application of results? Is work too often stalled in the data collection stage? Does the sharing-results phase tend to terminate the cycle? Where does the cycle get stuck, why, and what might free it up?

Questions

&

Discussion

www.learningoutcomeassessment.org

Documents

Assessing Intrapersonal and Interpersonal Competence