FACULTY DEVELOPMENTMODULE ON ASSESSMENT

Harford Community College

Nursing Faculty Orientation

2007 - 2008

INTRODUCTION

Evidence-based teaching

If learning is essential, using effective teaching strategies is a key skill needed by educators

Evidence-based teaching is the conscientious, explicit, and judicious use of current best evidence in making decisions about the education of professional nurses.

A useful reference for faculty interested in teaching with an evidence-based approach is: Stevens, K. & Cassidy, V. (1999). Evidence-based teaching. Sudbury MA: National League for Nursing.

Evidence-based teaching practice (EBTP)Should involve systematic study of:

teaching strategies; learning processes; learner characteristics; learner outcomes; academic success; effects of technology; organization of curriculum; theory verification; measurement and methodological issues; educational program standards; and administrative effectiveness.

(NLN, Shaping the Future, spring 2004)

INTRODUCTION

Value of the scholarship of teaching

The Scholarship of Teaching and Learning (SOTL) We invite you to read more about SOTL to explore

its value to your practice as an educator. Read and listen to the audio comments on the following SOTL web page by copying the following into your web browser address bar……..

http://www.issotl.org/tutorial/sotltutorial/u1a/u1a4.html You will be invited to discuss your own goals for

scholarship of teaching with Director Putland and Dean Wrobel as you continue your orientation. Now please continue this module by continuing to the next slide.

INTRODUCTION

Transforming Nursing Education

(NLN, 2005) Innovation in Nursing Education: A Call to Reform (NLN, 2003)

Innovation

“Learning and innovation go hand in hand. The arrogance of success is to think that what you did yesterday will be sufficient for tomorrow”

William Pollard

“Innovation is the ability to see change as an opportunity – not a threat.”

Steve Jobs

Transformation

“When complexity and ambiguity make progress difficult, transformational leaders seek novel solutions to problems using creativity and intuition. They facilitate the work of others and empower their success.”

Joel, L. in Andersen, C., 1999.

The National League for Nursing Board of Governors, after input from nurse educators in all types of programs, issued the following two position statements to lay a framework for national debate and to guide curriculum development initiatives in the early 21st century. Please read the two position statements found at the web addresses below and then complete Activity 1 included in your workbook of this module presentation.

http://www.nln.org/aboutnln/PositionStatements/transforming052005.pdf

http://www.nln.org/aboutnln/PositionStatements/innovation082203.pdf

INTRODUCTION

Challenges Facing Nurse

Educators Today

Most masters programs in nursing have focused on preparation for advanced clinical practice rather than for advanced practice as a nurse educator.

So many new nurse educators, while they are excellent clinicians, have had little if any preparation in education theory, curriculum development, teaching and learning, and assessment strategies.

These issues combined with a growing faculty shortage in Maryland means that faculty development initiatives are needed to help new educators prepare for their academic role.

Purpose of this Module on Assessment

Therefore the purpose of this orientation module is to provide a beginning framework for self-assessment of one’s knowledge base related to effective teaching and learning.

The Purpose of Assessment

Competency assessment is always outcome oriented; the goal is to evaluate performance for the effective application of knowledge and skill in the practice setting. Competency assessment techniques address psychomotor, cognitive, and affective domains. Competencies can be generic to clinical skills and leadership skills needed in practice, specific to a clinical specialty, at either basic or advanced levels (Benner, 1982; Gurvis & Grey, 1995).

Principles of Good Assessment

identifies readiness to learn and unique learning needs

utilizes resources and empowers

success for life-long learning uses appropriate assessments

Unique Learning Needs Adult Learners- bring experience, benefit from active

participation, expect a high degree of influence on what they are to be educated for, need to see applications for new learning, expect their responses to be acted on when asked for feedback on the progress of the program

More Diversity- need for students to gain a culturally competent education, need for diversity in faculty role models, and need for all students to have access to resources that can empower their success

Multiple Life Stressors- the majority of nursing students today work and have significant family responsibilities while attending nursing school

Individual Learning Styles- all students do not learn the same way, so use of only one or two teaching methodologies can result in ineffective learning for many students

Resources Available at HCC Resources for Faculty

via web resources to promote student success on HCC website at:

http://www.harford.edu/Advising/advinfo.asp?FA=StudentLife

Resources for Students and Faculty available regarding pre-nursing requirements for admission at:

http://www.harford.edu/Advising/prenursing.asp?FA=Employee

۩ Adult Literacy and ESL resources at HCC at:

http://www.harford.edu/cet/literacy/default.asp?FA=Employee

FERPA Family Educational Rights and Privacy Acts

Go to Harford Community College

website for this important orientation

at:

http://www.harford.edu/registration/FERPA/FERPA_Faculty/FERPA_Faculty_files/frame.htm

Types of Assessment Diagnostic: a set of tools and interventions to assess the teachers

readiness to teach and the students readiness to learn in ways that promote learning based on the complexity of the content, the level of competency required, the individual learning styles of the student, and the facilitators and barriers to successful assimilation and application of knowledge

Formative: will provide information to faculty about the process toward student success, quality of experience, and success of innovation in the curriculum; Data which reflects program competencies that can be used for summative evaluation should be collected

Summative: will help in determining how the innovation facilitates student achievement of course and program goals and outcomes. These data will allow benchmarking outcomes across similar programs given similar sample populations

Teaching & Learning for Different Levels of Thinking Skills

Cognitive…... Psychomotor….. Affective

Adapted from University of Hawaii, Honolulu Community College (HCC) new faculty training courses

Cognitive Domain of Learning:Knowledge The cognitive domain has three practical instructional

levels including fact, understanding, and application. The fact level is a single concept and uses verbs like define, identify, and list. The understanding level puts two or more concepts together. Typical verbs for this level include describe, compare and contrast. The application level puts two or more concepts together to form something new. Typical verbs at this level include explain, apply, and analyze. Delivery in this domain is typically a lecture or creative teaching presentation and the evaluation will be subjective and objective test items.

Psychomotor Domain of Learning: Skills In the psychomotor domain the student will

produce a product. The three practical instructional levels include imitation, practice, and habit. The first level, imitation, will simply be a return of the demonstration under the watchful eye of the instructor. The practice level may occur without direct oversight of the instructor. The habit level occurs when the student can perform the skill accurately, but in twice the time or less that it takes the instructor or an expert to perform. The evaluation will be a performance or skill test since content to do the skill is cognitive.

Affective Domain of Learning:Perceptions or Values The affective domain is based upon behavioral

aspects and may be labeled as beliefs. The three levels in the domain are awareness, distinction, and integration. The verbs for this domain are generally limited to words like display, exhibit, and accept and these apply at all levels. The first two levels are really cognitive; integration is behavioral and requires the learner to evaluate and synthesize. The content in this domain will usually involve discussions. The testing in the first two levels will be cognitive, whereas the third level will require an affective checklist.

Assessing and Evaluating Domains of Learning

Cognitive…... Psychomotor….. Affective

Adapted from NLN educational program, Innovative Teaching Strategies, by Mahoney, P. & Ortelli, T. NSNA Pre-Convention Faculty Workshop in Anaheim, CA, April 11, 2007.

Assessing and Evaluating the Cognitive Domain

group work directed paraphrase concept/mind mapping writing one minute papers subjective test items objective test items

Assessing and Evaluating the Psychomotor Domain return demonstration video tape peer critique peer coaching simulations pick out of a hat

Assessing and Evaluating the Affective Domain self-assessment journaling attitude surveys individual and group assessment

Major Categories in the Taxonomy of Educational Objectives(Bloom 1956)

Adapted from http://faculty.washington.edu/krumme/guides/bloom.html

Hierarchy of cognitive/thinking skills determine objective terminology It is important for new faculty to understand the taxonomy behind objective levels used in education. The most widely used taxonomy of cognitive domains of learning was developed in the mid-1950’s by Benjamin Bloom.

Knowledge level objectives:

Knowledge of terminology; specific facts; ways and means of dealing with specifics (conventions, trends and sequences, classifications and categories, criteria, methodology); universals and abstractions in a field (principles and generalizations, theories and structures):Knowledge is (here) defined as the remembering (recalling) of appropriate, previously learned information.

Outcome verbs in knowledge level objectives: define; describe; enumerate; identify; label; list; match; name; read; record; reproduce; select; state; or view.

Comprehension level objectives:

Comprehension of grasping (understanding) the meaning of informational materials.

Outcome verbs in comprehension level objectives: classify; cite; convert; describe; discuss; estimate; explain; generalize; give examples; make sense out of; paraphrase; restate (in own words); summarize; trace; or understand.

Application level objectives:Application of previously learned information in new and

concrete situations to solve problems that have single or best answers.

Outcome verbs in application level objectives: act; administer; articulate; assess; chart; collect;

compute; construct; contribute; control; determine; develop; discover; establish; extend; implement; include; inform; instruct; operationalize; participate; predict; prepare; preserve; produce; project; provide; relate; report; show; solve; teach; transfer; use; or utilize.

Analysis level objectives:Analysis is the breaking down of informational materials

into their component parts, examining (and trying to understand the organizational structure of) such information to develop divergent conclusions by identifying motives or causes, making inferences, and/or finding evidence to support generalizations.

Outcome verbs in analysis level objectives: break down; correlate; diagram; differentiate;

discriminate; distinguish; focus; illustrate; infer; limit; outline; point out; prioritize; recognize; separate; or subdivide.

Synthesis level objectives:Synthesis is creatively or divergently applying prior

knowledge and skills to produce a new or original whole.

Outcome verbs in synthesis level objectives: adapt; anticipate; categorize; collaborate; combine;

communicate; compare; compile; compose; contrast; create; design; devise; express; facilitate; formulate; generate; incorporate; individualize; initiate; integrate; intervene; model; modify; negotiate; plan; progress; rearrange; reconstruct; reinforce; reorganize; revise; structure; substitute; or validate.

Evaluation level objectives:Evaluation is judging the value of material based on

personal values/opinions, resulting in an end product, with a given purpose, without real right or wrong answers.

Outcome verbs in evaluation level objectives:

appraise; compare and contrast; conclude; criticize; critique; decide; defend; interpret; judge; justify; reframe; or support.

Hierarchy of objective levels (Bloom, 1956) As the level of cognitive complexity required

increases, the objective level for Bloom’s taxonomy also increases.

Level I are the lower levels of knowledge, comprehension and application objectives;

Level II are the higher levels of knowledge, application, analysis, synthesis, and evaluation objectives

Test blueprints for courses should reflect progressive complexity of objectives, with test questions and evaluation tools reflecting that progression of complexity across the program.

Bloom’s Taxonomy as a Basis for Writing Objectives and Assessments in HCC Courses

Activity Two and Three

Activity Two Request a copy from Director Putland of the current

course objectives for the HCC nursing class you have been hired to teach

Identify in writing, on the copy, the level of objective (Bloom’s Taxonomy) that corresponds to each objectives in your course

Identify in writing, on the copy, the domain of learning that currently is guiding the teaching and learning in the class based on the course objectives

Activity Three Write a proposed new list of course objectives for

the class you have been hired to teach that reflect the appropriate domain of learning (cognitive, psychomotor, or affective) and are listed in an order that progresses from least complex to most complex across the course

=====================================

Set and appointment to review and discuss with Director Putland your work on Activity Two and Activity Three

HCC Curriculum Review ProcessCurriculum content is guided by requirements of accrediting bodies (NLNAC and Middle States Association of Colleges and Schools) and the Maryland State Board of Nursing. To review the HCC Curriculum Review Process go to:

http://docushare.harford.edu/dsweb/Get/Document-224674/Educational%20Procedures%20and%20Practices.doc

Important Additional Aspects of Assessment to Consider

Authenticity- course objectives are clear, are shared with each student at the beginning of the course, and are consistently applied in ways that are culturally competent and meet student’s individual learning needs. Faculty provide assessment of each student’s progress toward achieving course objectives at frequent and regular intervals, and offer resources and remediation as needed to empower student success at Harford Community College.

Important Additional Aspects of Assessment to Consider Validity:

– Content: degree to which items in assessment instruments adequately represent the content

– Criterion-related (concurrent or predictive): degree to which scores in an instrument are correlated with an external criterion

– Construct- degree to which an instrument measures the construct under investigation

Evidence of validity comes from a triangulated view of goals/outcomes that confirms quality outcomes for students, faculty, and agency or staff employees.

Important Additional Aspects of Assessment to Consider Reliability

– These measures assure the stability, consistency, and equivalence of course outcomes

– Statistical evaluations vary based on the type of data collected. Item analysis, internal consistency measures, and other descriptive statistics with comparison of means can assist faculty to measure whether change has occurred related to benchmark outcomes

Other Considerations and Synthesis and Application of ContentThank you for participating in this Faculty Development Module on Assessment. While we cannot give you a graduate education in nursing education in one brief module, our hope is that we have provided a review and a baseline framework of terminology to help as you assess the course objectives in your class to guide your teaching, and prepare to assess and evaluate learning, as you begin your career at HCC. We’re glad you’re here! The nursing faculty and administration at Harford

Community College in Bel Air, MD