Promoting Educational Research Capacity through Systematic Evaluation of Curricular Innovations

Inspire

.Lead

.En

gage

.

Promoting Educational Research Capacity through Systematic

Evaluation of Curricular Innovations

Janet Landeen, Lynn Martin, Charlotte Noesgaard, & Kirsten Culver, McMaster

Donna Carr, ConestogaNancy Matthew-Maich, Mohawk

Inspire

.Lead

.En

gage

.

Changing Perspectives

Inspire

.Lead

.En

gage

.

Overview of Approaches

Inspire

.Lead

.En

gage

.

Today’s Session Setting the Context-Curriculum Renewal Gathering the Research Team Identifying the Big Questions Using Interpretive Description to Organize

Approaches Narrowing the Questions, Finding Resources,

Inviting Others to the Dance Sharing Progress to Date

Inspire

.Lead

.En

gage

.

The Context

Inspire

.Lead

.En

gage

.

Designing the Kaleidoscope Curriculum

Experiences of the McMaster Mohawk Conestoga BScN Program

Inspire

.Lead

.En

gage

.

Honouring the many authors of the Kaleidoscope

Inspire

.Lead

.En

gage

.

Inspire

.Lead

.En

gage

.

Kaleidoscope…

Kalos…beautifulEidos…shape

Scopeo…to reflect on

Inspire

.Lead

.En

gage

.

Key Areas for Curricular Revision

PBL/PBL Thinking Like a Nurse

Ways of Knowing

Integrated Knowing/Acting

/Being

Retention of Program Philosophy & Goals

Inspire

.Lead

.En

gage

.

Inspire

.Lead

.En

gage

.

The Person in the Middle Encounter the person (individual,

family, group, community) Use of narrative to have person come

alive

Inspire

.Lead

.En

gage

.

Turn the Kaleidoscope to a different aspect

Inspire

.Lead

.En

gage

.

Clinical Reasoning & Judgment in Nursing

(Tanner, 2006, p.208)

Inspire

.Lead

.En

gage

.

Turn the Kaleidoscope to a different aspect

Inspire

.Lead

.En

gage

.

Pedagogical Approach: Ways of Knowing in Nursing

Empiric (scientific, evidence based) Ethical (moral component-Codes of Ethics) Personal (consistent with therapeutic use of

self) Aesthetic (art of nursing) Emancipatory (critical reflection & action based

on inequities)(Carper, 1978; Chinn & Kramer, 2008)

Inspire

.Lead

.En

gage

.

Turn the Kaleidoscope to a different aspect

Inspire

.Lead

.En

gage

.

Integrated Approaches to Learning

Pathophysiology & Evidence Informed Decision Making

Scaffolded across multiple courses versus stand-alone courses

(AACU, no date)

Inspire

.Lead

.En

gage

. So What?Or

Who Cares?

Inspire

.Lead

.En

gage

.

Gathering the Research Team Interest in educational reform and research Identification of key individuals-

– Chair of Program Evaluation Committee– Director of Nursing Education Research Unit– Leaders within Program

Individuals looking for career growth opportunities

Graduate Students Undergraduate Students

Inspire

.Lead

.En

gage

.

Identifying the Big Questions

Inspire

.Lead

.En

gage

.

Program Evaluation vs. Educational Research

Evaluation of all aspects of program

Real life/real time curriculum-as-lived versus curriculum-as-planned

Used to inform local decision-making

In-depth exploration of specific topics

Comparison group, before/after design, or systematic qualitative exploration of experience or process

Uncover broader knowledge relevant to others

Inspire

.Lead

.En

gage

.

CASN Accreditation Standard3.2.5 Ongoing comprehensive evaluation

continually improves program outcomes. Descriptor: The program is deliberative and

responsive in making timely improvements based on data from ongoing rigorous and ethical evaluations from faculty, learners, graduates, employers, and others, through the application of the Key Elements.

Key Element: The program and curriculum are monitored and evaluated to ensure currency and relevance to nursing practice.

Inspire

.Lead

.En

gage

.

Program Evaluation at McMaster Mohawk Conestoga

Comprehensive Program Evaluation Plan Use of Stufflebeam’s CIPP Model (Context,

Input, Process, & Product) to develop Evaluation Matrix

Matrix critiqued for congruence with program philosophy, Stake’s Responsive Evaluation Model, & Aoki’s suggestion of multiple perspectives

(Aoki 1991; Stake 2003; Stufflebeam & Shinkfield 2007)

Inspire

.Lead

.En

gage

.

Program Evaluation

Specific Questions

Situational Issues

Institutional Priorities

Accreditation Standards

Student Expectations

Inspire

.Lead

.En

gage

.

Program of Research

Specific Research Project

Informed by Program Evaluation

Funding Opportunities

Faculty Interest

Inspire

.Lead

.En

gage

.

Evaluating the Impact of the Kaleidoscope Curriculum

Have the educational innovations of the Kaleidoscope Curriculum made a difference in the graduates of the BScN Program?

Are there unintended consequences of the curricular changes?

Inspire

.Lead

.En

gage

.

Sub-questions Do students achieve the same or better

outcomes on standard measures of performance?

Do clinical faculty and clinical preceptors notice a difference in graduating (Level 4) students of the Kaleidoscope Curriculum? If so, are those differences consistent with the educational innovations?

Inspire

.Lead

.En

gage

.

Sub-questions cont’d. Do graduating students integrate knowledge of

pathophysiology and evidence informed decision making into their clinical reasoning and judgement?

Do stories of the educational experience that graduates and faculty tell focus on the person (individual, family, group, community)?

Do graduating students utilize ways of knowing in their approaches to nursing practice?

Inspire

.Lead

.En

gage

. Interpretive Description

Inspire

.Lead

.En

gage

.

Integrative description as organizing approach

When purpose is to gain knowledge that can be directly applied to enhance nursing education.

“requires an integrity of purpose deriving from two sources: (1) an actual practice goal, and (2) an understanding of what we do and don’t know on the basis of the available empirical evidence (from all sources).”

(Thorne, 2008, p. 35)

Inspire

.Lead

.En

gage

.

Developing the Approach

Data Source Data Analysis Strategy Rationale

Student performance on pathophysiology and EIDM examinations pre and post Kaleidoscope.

Per item analysis on equivalent or identical exam questions, and grouped by content areas.

Teaching methodology is different but overall content to be learned has remained the same. Where appropriate, direct comparisons will be exam on anonymized examination responses.

Do students achieve the same or better outcomes on standard measures of performance?

Inspire

.Lead

.En

gage

. Progress to Date

Inspire

.Lead

.En

gage

.

Evaluating the Impact of Pedagogical Innovations:  Does the Kaleidoscope Curriculum Make a Difference in the

 Clinical Practice of Final Year Undergraduate Nursing Students?

Kaleidoscope Research TeamFunded by Centre for Leadership in Learning,

McMaster University $8,000

Inspire

.Lead

.En

gage

.

Evaluating Clinical Learning Outcomes

Integrative description Focus groups or individual interviews with 30

Clinical Faculty who taught students pre and post Kaleidoscope

Individuals likely to be sensitive to change in students but least involved in curriculum discussions

Semi-structured interview questions Graduate student Research Coordinator Poised to begin recruitment

Inspire

.Lead

.En

gage

.

Understanding the Experiences of Students Who are Enrolled in

Joint College/University Collaborative Educational Programs

Nursing: J. Landeen (Mac), N. Mathew-Maich (Moh), M. Parzen (Moh), & L. Hagermann (Con)

Bachelor of Technology: L. Bolan (Mac) & D. Bender (Moh)Medical Radiation Sciences: M. Faquharson (Mac) & L.

Marshall (Moh)

Inspire

.Lead

.En

gage

.

“Dual Identity” Project McMaster President’s “Forward with Integrity” Fund

$5,000, matched by Dean of Engineering Uses interpretive description methodology 8 undergraduate students (2 per program & site)

paid to develop full proposal, recruit participants, conduct focus groups, analyze results, & present findings

Nursing student researchers receive course credit prior to assuming paid role

Interdisciplinary faculty team mentoring & teaching students

Inspire

.Lead

.En

gage

.

Examples of Other Research Projects

Evaluation of written clinical evaluation forms 2 years pre and 2 years post Kaleidoscope to assess degree of uptake of curriculum renewal-in proposal writing stage (Noesgaard et al)

Exploring the Meaning of Consistency in PBL Programs-study completed (Landeen & Jewiss)

What Makes an Effective Teacher? Listening to the Voice of Our Students- in data analysis (Matthew-Maich et al)

Inspire

.Lead

.En

gage

.

Barriers to conducting educational research

Work load of faculty Cultural & institutional differences between

universities & college partners Funding sources for educational research Perceptions of value of educational research

versus clinical research

Inspire

.Lead

.En

gage

.

Facilitators to Educational Research

Resurgence of interest in educational research Organizations such as International Society for

Teaching and Learning in Higher Education (ISSOTL)

Accreditation standards Multiplicity of venues for presenting and

publishingEnriching the experiences of our

students!

Inspire

.Lead

.En

gage

.

References Aoki, T. T. (1991). Layered understandings of orientation in

social studies program evaluation. In Pinar. W.F. & Irwin, R.L. (Ed.), (2005). Curriculum in a new key: The collected works of Ted. T. Aoki. .(pp.167-185) Mahwah, NJ: Lawrence Erlbaum..

Association of American Colleges & Universities. (no date) Integrative Learning VALUE Rubric. Retrieved from: http://www.aacu.org/value/rubrics/integrativelearning.cfm

Carper, B. (1978). Fundamental patterns of knowing in nursing. Advances in Nursing Science, 1(1), 13-23.

Chinn, P., & Kramer, M. (2008). Integrated theory and knowledge development in nursing (7th ed.). St. Louis: Mosby Elsevier.

Inspire

.Lead

.En

gage

.

References cont’d. Stake, R. (2003). Responsive evaluation. In T.

Kellaghan & D.L Stufflebeam (Eds). The international handbook of educational evaluation, . (pp. 63-68). Dordecht: Klower Academic Publishers.

Stufflebeam, D.L. & Shinkfield, A.J. (2007). Evaluation theory, models, and applications. San Francisco: Jossey-Bass

Tanner, C.A. (2006). Thinking like a nurse: A research-based model of clinical judgment in nursing. Journal of Nursing Education,45(6), 204-211

Thorne, S.E. 2008. Interpretive description. Walnut Creek, CA: Left Coast Press

Inspire

.Lead

.En

gage

.

Thank you

landeen@mcmaster.ca