View
213
Download
0
Category
Tags:
Preview:
Citation preview
LESSONS LEARNED from a MULTI-CAMPUS DIVERSITY INITIATIVE
THE USE OF EVALUATION & ORGANIZATIONAL LEARNING FOR EFFECTIVE DIVERSITY
INITIATIVES:
“Organized Anarchy”
The American college or university is a prototypic organized anarchy. It does not know what it is doing. Its goals are either vague or in dispute. Its technology is familiar but not understood. Its major participants wander in and out of the organization. These factors do not make a university a bad organization or a disorganized one; but they do make it a problem to describe, understand, and lead [and as a result evaluate].
-- Michael D. Cohen & James G. March inLeadership and Ambiguity (1974)
Assessment:
• Systematic collection, review, and use of information about educational programs undertaken for the purpose of improving student [and campus] learning and development.
Marchese, T. in Assessment Essentials, Palomba, C.A. & Banta, T.W. (1999)
Empowerment Evaluation:
• The use of evaluation concepts, techniques, and findings to foster improvement and self-determination…it is designed to help people help themselves and improve their programs using a form of self-evaluation and reflection.
David Fetterman in Empowerment Evaluation (1996)
Purposes of Assessment
• Test Assumptions“Our bridge program is highly successful”“We are / are not diverse because…”“Diversity is central to our mission”
• Initiate & Guide Dialogue• Identify Gaps, Disconnects & Potential
Solutions• Develop a sense of ownership and
empowerment• Develop and sustain capacity for
organizational learning
Steps in effective assessment of diversity
initiatives
1. Define the Purpose
2. Primary Audience
3. Evaluation Team
4. Identify Context5. Target Topic
6. Formulate Questions
7. Obtain Data8. Assess Data9. Analyze Data10. Report
Findings
Source: Assessing Campus Diversity Initiatives:A Guide for Campus Practitioners, Garcia, M. et al. (2001)
Utilize Existing Data
• Institutional Research Office
• Vice-President for Planning / Research
• Varied Campus Offices
• Past Strategic Plans
• Self-Studies for Accreditation Agencies
• Past Diversity Initiatives
• Integrated Postsecondary Education Data System (IPEDS) www.nces.ed.gov/ipeds/
Benefits of Varied Types of Data
• Quantitative Data– Helps us explore the “what” and the “who”
(e.g. campus-based surveys)
• Qualitative Data– Helps us understand the “why” and the “how”
(e.g. focus groups; interviews; observations; document analyses)
• Process Data– Helps us understand and explore where pockets
of resistance, inertia, or hope are located– And, whether “IT” is resistance…inertia…or
hope
Assess and Analyze the Data
• Response Rates• Recognize and Qualify Data
Limitations• Background Variables
(e.g. race/ethnicity, gender, socioeconomic status, major etc.)
• DisaggregationGuided by mission / purpose of program
or initiative
Report the Findings
• Review target audience• Format for dissemination
Written reports; oral presentations; internal/external dissemination
• Guide the discourse• Involve critics• Balanced Analysis that emphasizes the
good, the bad, and the ugly• Connect findings with overall institutional
mission
Research must function as a tool for appropriating the codes and cultural symbols of institutional power in an effort to transform institutional environments in the interest of cultural democracy…In this way, researchers who carry out their work in the interest of cultural democracy can function as social advocates, facilitating a production of knowledge that is committed to the creation of institutional conditions where people find their voices and their rightful places as full and equal participants.
-- Antonia DarderInstitutional Research as a Tool for Cultural Democracy
References
• Assessing Campus Diversity Initiatives: A Guide for Campus Practitioners by Garcia, M.; Hudgins, C.A.; Musil, C.M.; Nettles, M.T.; Sedlacek, W.E.; & Smith, D.G.
• Leadership and Ambiguity by Cohen, M.D. & March, J.G.• Assessment Essentials: Planning, Implementing, and Improving
Assessment in Higher Education by Palomba, C.A. & Banta, T.W.• Empowerment Evaluation: Knowledge and Tools for Self-
Assessment & Accountability edited by Fetterman, D.M.; Kaftarian, S.J.; & Wandersman, A.
• Institutional Research as a Tool for Institutional Democracy by Darder A. in Studying Diversity in Higher Education edited by Smith, D.G.; Wolf, L.E.; &Levitan, T.
iALTO!BREAK !!
Nuts and Bolts of Evaluation: How to Use the Process
to Advance Institutional Learning about
Diversity
http://www.aacu.org/irvinediveval/index.cfm
IRVINE FOUNDATION GOALS FOR THE CAMPUS DIVERSITY INITIATIVE
Increase the institutionalization of diversity in the context of educating all students for participation and leadership in a diverse society.
Increase the educational success of disadvantaged and ethnic minority populations.
GOALS AND PURPOSE
http://www.aacu.org/irvinediveval/index.cfm
http://www.aacu.org/irvinediveval/index.cfm
RESULTS
Comparative Six-Year Graduation Rate for 1997 Entering Cohort
0
20
40
60
80
100
CDI-1 CDI-2 CDI-3 National UC CSU
Institutional Sources
Perc
ent
All American Indian African American Latino/a AA/PI White
GOALS AND PURPOSE
PURPOSE OF AN EVALUATION EFFORT:ORGANIZATIONAL LEARNING
Provide ongoing information about the implementation so that mid-course corrections can be instituted.
Build the capacity of campuses to assess and learn from their own progress.
Identity and provide opportunities for campuses to share problems and solutions.
http://www.aacu.org/irvinediveval/index.cfm
ORGANIZATIONAL LEARNING
http://www.aacu.org/irvinediveval/index.cfm
Using institutional data to inform progress
Interrupting the usual
Engage myths that become self-fulfilling prophecies
Maintaining the link between campus diversity efforts and institutional goals for effectiveness and excellence
Taking time for reflection and self-learning
Keeping all campus constituencies informed
Building synergy among many efforts on campus
Approaches evaluation from an organizational learning point of view
Manageable for campus and capable of being maintained
Monitors key goals and elements of proposal
Focuses on institutional issues/change, not simply project-specific issues
Reveals success and problems along the way in both results and processes
Guides interim reports to senior leadership, campus,
board → →
PRINCIPLES
http://www.aacu.org/irvinediveval/index.cfm
PRINCIPLES
• Takes into account:– Institutional differences and stages with respect to
diversity– That institutions vary in mission, needs, goals and
culture– That strategies, goals and emphasis differ– The possibility of taking some risks and learning from
them– Differences within institutions (Disaggregation of
information)• Encourages institutional sharing• Uses outside resources as appropriate
INSTITUTIONAL AUDIT
Education and Scholarship
Access and Success
Institutional Viability and
Vitality
Climate and Intergroup Relations
CONTEXTGlobal Local
http://www.aacu.org/irvinediveval/index.cfm
ACCESS AND SUCCESS
http://www.aacu.org/irvinediveval/index.cfm
Transfer among fields (esp.
SMET)
Success of students -- graduation, persistence,
honors, performance
Pursuit of advanced degrees
Undergraduate/graduate
population by fields and levels
ACCESS and
SUCCESS
Disaggregated
Independent U. Student Retention Rates,
2000-03 (%)(Note: Non resident information not available.)
86%
76%
70%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Fall 2 Fall 3 Fall 4
All
Four & Six-year Graduation Rates by Race/Ethnicity (%)
Figure 20. Independent U. Student Retention Rates by Race/ethnicity,
2000-03 (%)(Note: Non res ident inform ation not available .)
86%
76%
70%72%
62%59%
79%
71%66%
87%
78%
72%
88%
82%79%
82%
72%67%
92%
78%
69%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Fall 2 Fall 3 Fall 4
All
American Indian
African American
Latino/a
Asian American/P I
White
Unknown
Independent U. Graduation Rates, 2000-03 (%)(Note: Non resident information not available.)
52%56%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
4-Year Grad 6-Year Grad
All
Figure 21. Independent U. Graduation Rates by Race/ethnicity, 2000-03 (%)
(Note: Non resident information not available.)
52%56%
40%
46%44%
61%
52%
63%
58%
67%
51% 51%
42%
36%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
4-Year Grad 6-Year Grad
All
American Indian
African American
Latino/a
Asian American/PI
White
Unknown
CLIMATE AND INTERGROUP RELATIONS
Type and quality of interaction among groups
Quality of experience/
engagement on campus
Perceptions of institution (climate,
commitment, engagement)
CLIMATE AND
INTERGROUPRELATIONS
Disaggregated
http://www.aacu.org/irvinediveval/index.cfm
% “Satisfied with Climate for Minorities” by Attendance in Racial
Cultural Awareness Session:
Racial cultural awareness workshop
noyes
% S
atisfa
ied
with
clim
ate
fo
r m
ino
ritie
s
100
90
80
70
60
50
40
30
20
10
0
47
57
% “Satisfied with Climate for Minorities” by Attendance in Racial Cultural Awareness
Session and Race/Ethnicity:
Race/Ethnicity
% S
atisfie
d w
ith
Clim
ate
fo
r M
ino
ritie
s
100
90
80
70
60
50
40
30
20
10
0
Racial/Cultural Wksh
yes
no
60
38
60
80
62
70
46
62
73
56
EDUCATION AND SCHOLARSHIP
Experience
Presence of diversity-related courses, requirements
Degree to which courses include diversity issues and the placement of such courses
Quantity and substance of student learning about diversity
Course taking patterns of students
Level of faculty expertise on diversity-related matters
Level & diversity of faculty participating in diversity efforts
Availability
Faculty CapacityLearning
EDUCATION AND
SCHOLARSHIP
http://www.aacu.org/irvinediveval/index.cfm
INSTITUTIONAL VIABILITY AND VITALITY
http://www.aacu.org/irvinediveval/index.cfm
Constituency perceptions of institutional
commitment to diversity
Perceptionsof access, equity and inclusion among all
constituents
Public perceptions of the institution
Centrality of diversity in the
planning process, mission
statements
Institutional Strategies and
dedicated resources
Institutional history on
diversity issues and incidentsDiversity of
faculty and staff by level
INSTITUTIONAL VIABILITY AND VITALITY
http://www.aacu.org/irvinediveval/index.cfm
RESULTS
Core Faculty by Race/Ethnicity, 2000 & 2004
0.4 3 4 7 7
83
1 10.6 4 59 8
1 1
80
0102030405060708090
100
Per
cen
t (%
)
2000
2004
Faculty Turnover Quotient
http://www.aacu.org/irvinediveval/index.cfm
RESULTS
TQ = 0%: No Turnover
TQ = 100%: 100% of URM new hires replaced URM faculty who left the institution
)]x100Hires URMNew
URM Period StartURM Period End([1TQ
http://www.aacu.org/irvinediveval/index.cfm
RESULTS
0
50
100
150
200
250
300
Turnover Quotient*
Turnover Quotient* of URM Faculty (Replacement Rate), Individual Campuses, 2000-2004
Institution
http://www.aacu.org/irvinediveval/index.cfm
FACULTYOverall --DISAGGREGATED--over timeNew hiresRetention/turnover--TQLocation --departments/fields
EVALUATION PROCESS
Institutional Goals
ProcessStrategies(Objectives)
REPORTING
Interpretations/PerceptionsLessons Learned
What has been the impact on for example, curriculum, student success, etc.?
What has been done?What has happened?What is working?What is not working?
http://www.aacu.org/irvinediveval/index.cfm
iALTO!BREAK !!
Principles of Evaluation & Institutional Learning
• Framework and Monitoring Progress
• Centrality to Mission• Leadership & Communication• Organizational Learning• Inclusive and Differentiated
Approach• Alignment
Recommended