1

Section

4 Teaching Effectiveness

Effective Testing Begins with Course Design

A key component of Kolb’s experiential learning cycle/model is that learning should be relevant and applicable to the learner’s needs. Therefore, my assessments normally allow choice and expansion or linking of knowledge and skills being explored in other courses. Assessments typically have a practical aspect that is useful in either their profession or allows for them to apply a new skill.

When designing course outcomes, I use Bloom’s Taxonomy as my template. My research into the effective development of learning outcomes in post-secondary education has lead me to the conclusion that if we want our students to be able to critically analyze and synthesis information, then we must write our course outcomes at least the analysis level. Given that research also suggests that multiple-choice and true/false assessment instruments are more appropriate when assessing a student’s knowledge, comprehension, and application, it is rare that I have multiple-choice or true/false types of questions on an assessment instrument. An exception is when I am designing curriculum for programs that have a national assessment process that involves multiple choice questions or when I have created a self-assessment tool in an online learning environment.

A comment by a recent student demonstrates the type of feedback and assessments that I provide learners as they progress through my courses: “M. Burns set up the class to be extremely user-friendly. Her responses to problems or misunderstanding were lighting quick as were all e-mail activity. I enjoyed her professional demeanor and intellectually articulate instruction and communication.”

Development and use of meaningful assessment

When designing student assessments I create a matrix as illustrated below. The matrix illustrates how to link Bloom’s Taxonomy to topics and course outcomes in my course. The matrix also allows me to determine the number of assessments and weightings depending upon the number of course outcomes and learning objectives I have for each topic. My assessments of students’ progress within a course are designed to get them to critical think or reflect on specific topics as well as providing for direct application of a new skill or knowledge subset. See Appendix B for assessment examples.

Design of an Assessment Matrix

The design of an assessment matrix requires the course outcomes and learning objectives be linked to specific topics, modules or units within a course. Using the related learning objectives, the instructor can create a matrix for each topic, module

2

or unit. The steps and matrix indicated below is the design assessment that I typically use and teach others to create.

Step 1: Review Course Outcomes and Learning Objectives

By the end of this course you will be have: 1) Explored the relationships between culture, diversity and health 2) Examined cultural competence within a health context 3) Examine the impact of diversity of health care organization 4) Examine the impact of diversity on health education programs

Step 2: Separate Course Outcomes into topics, modules or units

Unit 2: Culture and Medicine- Weeks 2-5 (Goals 1-4)

1. Explore the relationships between culture, diversity and health a. Define culture and cultural competence. b. Define diversity, both visible and invisible; define diverse populations.

2. Explore the characteristics of diversity and diverse populations a. Discuss how varying cultural values and beliefs among clients can

affect health care delivery. 3. Examine cultural competence within a health context.

a. Recognize cultural models of health, disease and illness; in particular the common models in the UAE community.

4. Examine the impact of diversity on health care organizations a. Describe the major positive and negative impacts of a diverse

workforce in health care. 5. Examine the impact of diversity on health education programs

a. Describe the challenges of health promotion/ education in a diverse community.

Step 3: The Matrix

Goal 1 Goal 2 Goal 3 Goal 4 Goal 5 Bloom’s Taxonomy

Level Relationships Populations Competence Health Care

Education

Total/ Weighting

Knowledge 5 3 8

Comprehension 2 3 6 6 17

Application

Analysis 6 2 1 1 1 11

Synthesis

Evaluation

Total 13 5 4 7 7 36

3

Step 4: Creation of the Assessment Instruments Example (Learning Objectives 4a and 5a):

4a. Describe the major positive and negative impacts of a diverse workforce in health care. 5a. Describe the challenges of health promotion/ education in a diverse community.

EXAMPLE OF A CUMULATIVE UNIT ASSESSMENT

4

Effective teaching and learning as substantiated by student feedback

Examples of student feedback for courses taught within the post-secondary education system are outlined below. Student feedback for workshops were collected and read, but copies were not kept. In the future, a written reflective summation on workshops including examples of student feedback will be kept. See Appendix C for full analysis of student feedback for courses in which feedback was provided.

Higher Colleges of Technology

The Higher Colleges of Technology evaluated its faculty primarily through student feedback using a computer-based evaluation form that provided instantaneous feedback to the professor. Below is an example of the questions and format that were used for evaluation purposes. My reflection per area is located under each section. Analysis and reflection of other courses are provided in Appendix D.

HLED 105

HLED 105 introduces the concepts of health and wellness. Determinants of health are explored with reference to the UAE, and to the background theories of health and illness. The structure of the UAE Health Care workforce, roles and responsibilities in health care organizations, including standards of professional practice is examined. The influence of international philosophies of health care on the UAE system is explored. Students are introduced to some criteria for planning to meet community health service needs.

Student-Faculty Evaluation

Teacher Name Margo Burns Evaluation Date 10/Dec/2005 09:00 AM

Course Code HLED-105 - Intro to Health Care Systems Section 12B1HLED01

Department Health Science Number of Votes 7 First Evaluation

SAgr-Strongly Agree (10) Agr.-Agree (7.5) NAD-Neither Agree nor Disagree (5) Dis.-Disagree (2.5) SDis.- Strongly Disagree (0)

Category : Interaction skills

Ser Question SAgr

10 Agr. 7.5

NAD. 5

Dis. 2.5

SDis. 0

Avg. (10)

1 The teacher clearly explained the college's policy on cheating and plagiarism.

6 1 0 0 0 9.64

2 The teacher is available to meet and help me outside class. 3 4 0 0 0 8.57

3 The teacher treats me with respect. 3 4 0 0 0 8.57

4 The teacher makes clear to me what is expected of me on a daily/weekly basis.

4 3 0 0 0 8.93

5 The teacher explains the subject well. 4 2 1 0 0 8.57

6 The teacher treats me well. 3 4 0 0 0 8.57

7 The teacher is helpful to me when I have difficulty in class or with assignments. 4 3 0 0 0 8.93

8 The teacher speaks clearly and in a manner that is easy to understand. 3 4 0 0 8.57

9 The teacher treats me fairly. 2 4 1 0 0 7.86

10 The teacher provides timely feedback on assignments. 3 4 0 0 0 8.57

11 The teacher keeps me informed of my progress. 3 4 0 0 0 8.57

Interaction skills Total and % Average 37 36 2 0 0 85.4

5

Reflection: Interaction Skills An above overall average in this section of the evaluation indicates that I am on track in most areas. Unfortunately due to my heavy involvement with 4

th year students, I was less available to them then

some of their other professors. I will suggest that students be provided with a larger block of time within each week that more closely matches those professors teaching 4

th year students. Also finding

classrooms space that we could use for a “drop-in” format might be helpful in allowing the more shy students easier access to my time.

Category : Teaching Skill

Ser Question SAgr

10 Agr. 7.5

NAD. 5

Dis. 2.5

SDis. 0

Avg. (10)

1 The teacher clearly stated the objectives of the subject and the requirements at the beginning of the semester.

3 4 0 0 0 8.57

2 The teacher clearly explained the grading system for the subject. 5 2 0 0 0 9.29

3 The teacher uses materials that help me understand the subject. 5 2 0 0 0 9.29

4 The teacher enjoys teaching. 4 3 0 0 0 8.93

5 The teacher creates a classroom environment that is comfortable, and encourages learning.

5 2 0 0 0 9.29

6 The teacher presents the lesson in a clear and organized way. 4 2 1 0 0 8.57

7 The teacher encourages me to think for myself. 3 4 0 0 0 8.57

8 The teacher gives me homework and assignments that are useful for my understanding of the subject.

4 2 1 0 0 8.57

9 The teacher gives clear instructions for homework and assignments.

4 2 1 0 0 8.57

10 The teacher helps me prepare for assessments. 3 2 2 0 0 7.86

11 The teacher encourages me to express and defend my opinions. 5 2 0 0 0 9.29

12 The teacher helps me improve my communication skills (writing, speaking)

5 2 0 0 0 9.29

13 The teacher makes the class interesting. 2 4 1 0 0 7.86

14 The teacher encourages me to learn. 3 4 0 0 0 8.57

Teaching Skill Total and % Average 55 37 6 0 0 87.5

Reflection: Teaching Skill While the overall score within this section is above average, the average score in “teacher helps prepare me for assessments” indicates that students may have felt unprepared for the written assignments in this course. A lower score is also noted in the next section, interaction skills. More time needs to be given in the first year, first semester courses for students to grasp the assessment concept. Fewer assignments need to be provided which would allow for more class time to build the skills necessary to be successful in this course.

Category : Technological skills

Ser Question SAgr

10 Agr. 7.5

NAD. 5

Dis. 2.5

SDis. 0

Avg. (10)

1 The teacher encourages me to use computer technology (e.g. internet, PowerPoint)

4 3 0 0 0 8.93

2 The teacher uses educational technology (computer, OHP, etc..). 5 2 0 0 0 9.29

3 The teacher knows how to use educational technology in a way that helps me learn.

3 4 0 0 0 8.57

Technological skills Total and % Average 12 9 0 0 0 89.3

6

Reflection: Technology Skills Unfortunately, no student made any written comments and therefore it’s difficult to determine what kind of technology within this course could have been used to help them learn better. This particular class is not scheduled for use within the computer-based classroom, therefore limiting access to computers for their use. We did have several interactive sessions using the computer (i.e. Health Promotion Detectives Game) but this was teacher-led not student-centered; perhaps allowing them to work on this assignment in pairs would have helped.

Category : Organizational skills

Ser Question SAg

r 10

Agr. 7.5

NAD. 5

Dis. 2.5

SDis. 0

Avg. (10)

1 The teacher follows the course guide. 4 3 0 0 0 8.93

2 The teacher is well prepared for each class. 5 2 0 0 0 9.29

3 The teacher uses class time well. 3 4 0 0 0 8.57

Organizational skills Total and % Average 12 9 0 0 0 89.3

Category : Content Knowledge

Ser Question SAgr

10 Agr. 7.5

NAD. 5

Dis. 2.5

SDis. 0

Avg. (10)

1 The teacher knows the subject well. 4 2 1 0 0 8.57

Content Knowledge Total and % Average 4 2 1 0 0 85.7

Comments posted to Margo Burns for HLED-105

Ser. Comments

No comments posted.

7

University of San Diego

I began teaching for the University of San Diego in December of 1999. While I have continually been evaluated by the students since then and on one occasion through a peer process, much of the original evaluation process was hosted on their old online learning system and therefore access to this material is no longer available. I do however have data from 2005 onwards, although a few sessions are missing due to their system being down. I have plotted out the data in a graph form and provided an analysis of the written comments.

Example of Closed-ended Questions

Question 1: Did you find that the course material was well organized?

Students are asked to rate the course material using a 5 point scale as follows:

• All of the time

• Most of the time

• About half of the time

• Some of the time

• No opinion

University of San Diego Faculty Evaluation

0

20

40

60

80

100

Aug-

05

Sep-

05

Mar-

06

Sep-

06

Mar-

07

Jun-

07

Sep-

08

Course Date

Qu

esti

on

Resp

on

se %

Organization ofCourse

Reflection: In review of my notes I see that the September 2006 class had 4 failures, 3 whom failed to submit work within the timeframe. This was also the month in which the new software was introduced. Together these two items may have contributed to the dip in the ratings. As my peer evaluation on the set up of my course indicates a well organized layout of material, I am satisfied with this rating.

8

Written Comment Analysis While student feedback using closed-ended questions is useful, particularly as it provides graphical information that can be used to quickly see unusual dips in student ratings, I find it is the written comments that provide the most useful information. For example, as illustrated in Appendix C, when students indicate that they had difficulty find assignments in the new learning management system (in Blackboard there was a tab called assignments), I immediately provided a course timetable that indicated the course layout, assignment names and due dates. In the next student feedback after this change was made, students indicated that they wanted to see the value on the timetable sheet for each assignment, even though the grade book provided this information. An immediate change, as illustrated below, was made to the course timetable.

E-mailed Comments In addition to the written comments provided in the formal student-faculty evaluation process, I receive about 5 e-mailed comments per course that provides me with very quick feedback about certain aspects of the course that a student has enjoyed. The comments have been important aspects of my course design and reflection, particular those received in the January courses following major revisions to the layout of a course. Examples of these comments are outlined below.

9

Wearp, Self-Discovery#6, January

Wednesday, February 20, 2008 7:12 PM

……After I took this quiz, I played the videos with my class and had them take the quiz. We did it

as a whole group and they loved it. They were already familiar with the characters. They also all

got 100% on all three of the quizzes. Now they want to watch more of the human body stuff and

take more quizzes!....

10

Development of effective teaching materials

Development of teaching materials begins with the layout of the topic or unit, linking course outcomes and learning objectives followed by the development of lesson plans. Lesson plans are developed using several different methods including Gagné’s 9 steps of instruction. Samples of teaching material are provided below. Full examples of lesson plans and associated teaching materials are located in Appendix D.

University of San Diego-Online Teaching

Example of the Discussion Board for Week 1 (Sept 2005)

Notice the change from the one above to the one posted below. The discussion board now includes graphics, videos, or audio as required to fully engage the learner.

11

Higher Colleges of Technology

1. Online Learning Activity

Below is an example of an adaptive learning material associated with an online learning activity. Students worked through what an epidemic is through the use of the online activity and completed a test as a game.

12

2. PowerPoint Slide Example