Using Institutional Learning Outcomes to Foster Meaningful Dialogue and Decision-Making

Using Institutional Learning Outcomes to Foster Meaningful Dialogue and Decision-Making

Giovanni Sosa, Ph.D.Chaffey College

RP Conference 2013

The Standards (pg.1)Instructional programs, student supportservices, and library and learning supportservices facilitate the achievement of theinstitution's stated student learning outcomes

ACCJC Standards

Improving Institutional Effectiveness (IB)The institution demonstrates a conscious effort to produce and support student learning, measures that learning, assesses how well learning is occurring, and makes changes to improve student learning…

ACCJC Standards

Improving Institutional Effectiveness (IB)…[it] demonstrates its effectiveness by

providing 1) evidence of the achievement of student learning outcomes…

ACCJC Standards

SLO Assessmentis a continual

process!Criteria

Means ofAssessment

Summary ofEvidence

LearningOutcomesStatement

Use ofResults

Adopted Institutional SLOs◦ Communication◦ Critical Thinking & Information Competency◦ Community/Global Awareness & Responsibility◦ Personal, Academic, and & Career Development

http://www.chaffey.edu/general_info/competencies.shtml

Core Competencies



Facione (1990) 46 experts were convened to discuss role of

CT in educational assessment and instruction

Identified Core CT skills and Sub-Skills

Critical Thinking

1) Interpretation – To comprehend/express meaning of wide variety of experiences

2) Analysis – Identify inferential relationships among statements/concepts

3) Evaluation – Assess the credibility of statements; assess logical strength of inferential statements/concepts

4) Inference – Form hypotheses

Critical Thinking: Core CT SKills

5) Explanation – To state results of one’s reasoning;

6) Self-Regulation – Self-reflection of one’s views to question or confirm reasoning

Critical Thinking: Core CT SKills

MSLQ (Pintrich, McKeachie, & Lin, 1987)◦ Developed in process of studying how to make

students more efficient learners

What does the CT assessment specifically measure?

Critical Thinking Assessment

CT assessment found to be statistically associated to course performance (r = .15)

Critical Thinking Assessment

Success Not Successful Total

High CT 57 42 100

Low CT 42 57 100

Total 100 100 200

Pertains to issues of cultural diversity, human rights, and prejudice reduction – both within national borders and across national borders

Knowledge that individual possesses of other cultures (Cognitive)

The extent to which individual empathizes with values of other cultures (Affective)

Willingness to take stand on cultural issues (Participatory)

Global Awareness (Clarke, 2004)

Inspired by Global Perspective Institute (Braskamp et al., 2011)◦ Holistic Human Development:

Cognitive domain (“Whatdo I know?”) Intrapersonal domain (“”Who am I?”) Interpersonal domain (“How do I relate to others?”)

What does our Global Awareness assessment specifically measure?

Global Awareness Assessment

1907 Total Responses 78 Courses 33 Departments

◦ Chemistry (N = 199)◦ English (N = 195)◦ Theatre (N =159)◦ Cinema (N = 154)◦ CIS (N = 137)◦ Economics (N = 102)

Voluntary Participation

Results: Course Characteristics

SampleN = 1,907

DistrictN = 56,216

Transferable to UC/CSU 69.8% 54.8%Transferable to CSU 13.2% 14.5%Not Transferable 16.9% 30.7%

Results: Course CharacteristicsComparison with District Data (SP12)

Transfer Status

SampleN = 1,907

DistrictN = 52,216

Face-to-face 96.5% 94.0%Hybrid 3.5% 2.2%Online 0.0% 3.8%


Distance Learning

SampleN = 1,907

DistrictN = 52,216

Basic Skills 1.8% 16.0%Not Basic Skills 98.2% 84.0%


Basic Skills

SampleN = 1,791

DistrictN = 18,434

Cum GPA 2.92 2.52Units Attempted 42.78 34.42Units Completed 40.80 31.32

Results: Student CharacteristicsComparison with District Data (SP12)

Performance Indicators

SampleN = 1,791

DistrictN = 51,526

Success Rate 84.4 70.53Withdrawal Rate 3.3 11.1

Results: Student CharacteristicsComparison with District Data (SP12)

Performance Indicators

SampleN = 1,673

DistrictN = 43,767

GPA Converted Grade 2.88 2.61

Critical Thinking

CT Level Range PercentileLow 0 – 19 ≤24th PercentileMedium 20 – 28 25-74th PercentileHigh 29 - 35 ≥75th Percentile

Critical Thinking: Levels

Variables Examined:◦ First Generation Status◦ Gender◦ Age Range◦ Parents’ Education◦ Ethnicity◦ UC/CSU Transfer vs. Non-Transferable

CT Differences by Demographics/Course Transfer Status

Variables Examined:◦ Success Rate◦ Withdrawal Rate◦ GPA Converted Grades◦ Units Attempted◦ Units Completed◦ Cum. GPA◦ Assessment Tests (Reading Comp/Sentence Skills/

Math)

CT Differences by Performance Indicators

CT Level Number of Students

Mean Units Attempted

Low 446 44.29Medium 962 41.26High 496 44.79*d = .11 - Medium level vs. Low level d = .13 – Medium level vs. High level

CT Differences by Performance Indicators: Units Attempted


Mean Units Earned

Low 446 42.76Medium 962 39.06High 496 42.66*d = .13 for comparisons of Medium level with either Low or High

CT Differences by Performance Indicators: Units Earned


Mean Cum. GPA

Low 446 2.98Medium 962 2.87High 496 2.94*d = .13 - Medium level vs. Low level d = .09 - Medium level vs. Low level

CT Differences by Performance Indicators: Cum. GPA

Global Awareness

GA Level Range PercentileLow 0 – 25 ≤24th PercentileMedium 26 – 28 25-74th PercentileHigh 29 - 30 ≥75th Percentile

Global Awareness: Levels

Variables Examined:◦ First Generation Status◦ Gender◦ Age Range◦ Parents’ Education◦ Ethnicity◦ UC/CSU Transfer vs. Non-Transferable

GA Differences by Demographics/Course Transfer Status

GA Differences by Age Range

GA Level19 or

Younger 20 to 24 25 or Older Total

Low 149 209 96 454Medium 249 384 204 837High 133 265 215 613Total 531 858 515 1904*r = .14

Variables Examined:◦ Success Rate◦ Withdrawal Rate◦ GPA Converted Grades◦ Units Attempted◦ Units Completed◦ Cum. GPA◦ Assessment Tests (Reading Comp/Sentence Skills/

Math)

GA Differences by Performance Indicators

GA Level Number of Students

Mean Units Attempted

Low 454 39.34Medium 837 41.85High 613 46.92*d = .09 – Low vs. Medium d = .27 – Low vs. High d = .17 – Medium vs. High

GA Differences by Performance Indicators: Units Attempted


Mean Units Earned


GA Differences by Performance Indicators: Units Earned


Mean Sentence Skills


GA Differences by Performance Indicators: Sentence Skills

Relationship between CT and GA?

Low GA High GA Total

High CT 120 217 334

Low CT 117 66 186

Total 237 283 520

r = .15

Enhancing institutional effectiveness◦ Self-reflective dialogue

Utilized assessments serve as reliable tools for measuring CT and GA◦ But may not adequately measure all aspects of

corresponding competency More assessment of foundation/online

courses Implications of units attempted/earned &

GPA

Conclusions

Random sampling for Spring 2013 Inclusion of two additional core

competencies for Spring 2013 Division-wide assessment during Fall 2012 Application to specific course level

assessment First Annual Student Services Poster Session

Institutional Response to Findings

Braskamp, L. A., D. C. Braskamp, K. C. Merrill, & M. E. Engberg (2011). Global Perspective Inventory. Global Perspective Institute, Inc., http://gpi.central.edu

Clark, V. (2004). Students’ global awareness and attitudes to internationalism in a world of cultural convergence. Journal of Research in International Education, 3, 51-70.

Facione, P. A. (1990). Critical thinking: A statement of expert consensus for purposes of educational assessment and instruction. American Philosophical Association (pgs. 13 -19). Newark, DE.

Pintrich, P. R., Smith, D. A. F., Garcia, T., & McKeachie, W. J. (1991). A Manual for the use of the Motivated Strategies for Learning Questionnaire (MSLQ). National Center for Research to Improve Postsecondary Teaching and Learning. Washington, DC.

References

http://gpi.central.edu/

Documents

Using Institutional Learning Outcomes to Foster Meaningful Dialogue and Decision-Making