Everything you wanted to know about course evaluations but were afraid to send an email to ask… Shari L. Jorissen, EdD, PhD Director of Assessment Office

Everything you wanted to know about course evaluations but were

afraid to send an email to ask…

Shari L. Jorissen, EdD, PhD

Director of Assessment

Office of Institutional Research & Assessment

Welcome to Today’s Webinar!

Confidential – Internal Use Only2

• Click the arrow to view panel.• Adjust audio setup as needed.• Ask questions throughout the webinar.• Technical Support: 800-263-6317• Closed Captioning is available through the link

in the Questions area.

Presenter Background


Dr. Shari Jorissen, EdD, PhD is the Director of Assessment in the Office of Institutional Research and Assessment (OIRA) at Walden University.

As the Director of Assessment, Dr. Jorissen is responsible for coordinating the course evaluation process for Walden including administration and reporting. In addition, she is responsible for Program level learning outcomes assessment and reporting across Walden as well as completing research projects with different stakeholders throughout the University.

She was a member of the Walden University HLC Assessment Academy committee and currently serves as a mentor for two other universities going through this process. She is an active member of the Association for Institutional Research (AIR) and the Association for the Assessment of Learning in Higher Education (AALHE).

Dr. Jorissen is a two time alumni of Walden having earned her PhD in Psychology in 2008 and her EdD in Higher Education and Adult Learning in 2012.


Session Learning Outcomes

At the end of this session, participants will be able to:

• Be familiar with the types of course evaluations used at Walden.

• Be familiar with the course evaluation administration process.

• Locate items, frequencies, and mean scores on a course evaluation report.

• Be familiar with the types of reporting that use course evaluation data.

Course Evaluation Data Collection

Course Evaluation Versions

Evaluation: Courses Included:

StandardAll graduate level courses not included in other categories; 5000 level; WCSS

UndergraduateAll undergraduate level courses not included in other categories

Field ExperienceAll practicum, internship, and field experience courses

SBSF-7100/Research ClassroomAll SBSF-7100 courses; Thesis, Dissertation, Doctoral Study courses


Course Evaluation Deployments

Sunday Monday Tuesday Wednesday Thursday Friday Saturday

Initial Invitation (overnight

between M & T)

Last day of class

Reminder Email

Grades due

Deployment Ends

Faculty Level Reports

Deployed*


Student Access to Course Evaluation

Students receive an email inviting them to take the course evaluation

To complete your survey:• Go to the My Announcements area within your Blackboard classroom and open the

"Invitation to respond" announcement to submit a response. • If you do not see the invitation link in the My Announcements box, click on more

announcements…to find the link.• You may have to scroll through the announcements to find the link.• The course evaluation will not be saved, if you stop filling out the evaluation before you

submit you will need to start again.







After 7 days the link moves to “More Announcements”

– Students may need to scroll through the announcements to find the link






Faculty Level Reporting

Individual Faculty Level Reports

Faculty directions:

1.Log into a recent course

2.Find Content Collection in the control panel/menu on the left of your classroom

3.Expand the Content Collection link


2

3


Faculty directions:4.Find link containing your username_content; click on this link

5.Go to folder containing course section (individual faculty) course evaluation reports—folder will have deployment name containing term (201250-1 for example)


4

5


Faculty directions:6.Click on the PDF link to open your report:


56





A: Internal OIRA tracking number

B: Type of course evaluation

C: Term-Part of Term (from Banner)

C1: Year (Fall term begins the new year code: September 2012 = 2013)

C2: Term (Odd=Quarter; Even=Semester)

10=Fall; 30=Winter; 50=Spring; 70=Summer

20=Fall; 40=Spring; 60=Summer

C3: Part of term (Identifies how many weeks and placement in term)

1=Full term (12 wks Quarter; 16 wks Semester)

2=1st 8 weeks of Semester

4=2nd 8 weeks of Semester

A CB

C1 C2 C3

Examples:



1=School2=College

3=Walden Overall

Only includes data from that

deployment

A

B

C

University Level Reporting

Reports Generated with Course Evaluation Data

Reports are compiled by OIRA every 3 months based on course evaluation deployment end dates:

– March 6 through June 5: Spring Report– June 6 through September 5: Summer Report– September 6 through December 5: Fall Report– December 6 through March 5: Winter Report

• Course Evaluation Report: All questions; Frequencies and means over time

• Course Characteristics Report: Mean scores of course questions

• Instructor Quality Report (IQR): Mean scores of instructor questions and instructor scale


Course Designation

• Many programs share courses• Course Designation is used to categorize courses

in quarterly reports


Course Course Designation

COUN-6000; COUN-6510 COUN-6xxx

MGMT-6045/MGMT-6705 Crosslisted SoM

COUN-6700/PSYC-6700/MMPA-6700 Crosslisted CSBS

MMPA-8320/PUBH-8320 Crosslisted University

COMM-1001 COMM (GenEd)

COMM-3001 COMM (SOM)

Quarterly Course Evaluation Report


45.9% 35.5% 41.2% 58.6%

87.0% 88.7% 86.4% 95.3%

89.7% 88.6% 69.8% 94.9%

83.3%-91.4% 84.6% - 85.4% 63.9% - 86.0% 80.5% - 93.9%

Survey Form

RC/SBSF 7100 Field ExperienceTable of Contents

Response Rate

Overall Satisfaction with Instructor

Overall Satisfaction with Course

University Services

Summary Undergrad



Course Designation

EDU

C 8x

xx

EDU

C 7x

xx

EDU

C 6x

xx

EDAD

8xx

x

EDAD

7xx

x

EDAD

6xx

x

EIDT

6xx

x

MAT

H 6

xxx

READ

6xx

x

SCIE

6xx

x

COE

Cros

s-lis

ted

Colle

ge o

f Ed

ucati

on

Number of Respondents 951 200 3325 65 8 98 170 255 62 77 5211 365

InstructorPlease answer the following questions about your instructor:The instructor encouraged mutual respect among students.Strongly disagree 0.5% 0.5% 0.5% 3.1% 0.0% 0.0% 0.0% 0.8% 0.0% 1.3% 0.5% 1.1%Disagree 0.8% 2.0% 0.4% 3.1% 0.0% 3.1% 0.0% 0.8% 0.0% 0.0% 0.6% 1.6%Neutra l 5.0% 5.5% 4.1% 7.7% 12.5% 5.2% 7.1% 2.4% 4.8% 1.3% 4.4% 6.0%Agree 23.6% 34.0% 28.3% 35.4% 12.5% 24.0% 26.0% 35.4% 21.0% 28.6% 27.8% 28.8%Strongly agree 70.0% 58.0% 66.7% 50.8% 75.0% 67.7% 66.9% 60.6% 74.2% 68.8% 66.6% 62.5%Total 943 200 3309 65 8 96 169 254 62 77 5183 365The instructor addressed students’ needs as learners.Strongly disagree 1.4% 1.5% 0.8% 7.7% 0.0% 0.0% 1.2% 2.0% 0.0% 1.3% 1.1% 3.9%Disagree 2.6% 5.5% 2.2% 13.8% 12.5% 4.2% 0.6% 4.0% 1.6% 0.0% 2.6% 5.8%Neutra l 4.9% 5.5% 6.1% 15.4% 12.5% 7.3% 11.9% 7.5% 4.8% 1.3% 6.2% 6.9%Agree 24.8% 29.5% 29.6% 27.7% 12.5% 24.0% 24.4% 33.2% 29.0% 33.8% 28.6% 26.3%Strongly agree 66.4% 58.0% 61.2% 35.4% 62.5% 64.6% 61.9% 53.4% 64.5% 63.6% 61.5% 57.1%Total 941 200 3299 65 8 96 168 253 62 77 5169 361

Table A9: Summary Course Evaluation

Spring 2012



Sum 2011

Fall 2011

Wtr 2011

Spr 2012

Sum 2012

Spr'12 - Sum '11

Spr '12 - Fall '11

Spr '12 - Wtr '11

Cross-sectional SampleWalden Instructor Scale (Range 8 - 40) 35.63 35.49 35.58 35.61 No -0.02 0.12 0.03Classroom Characteristics Scale (Range 12 - 60) 40.10 39.99 40.08 40.10 No 0.00 0.11 0.02Course Balance Scale (Range 3 - 15) 9.34 9.35 9.36 9.36 No 0.02 0.01 0.00University Services Scale (Range 3 - 15) 13.24 12.93 12.96 12.75 Yes -0.49 -0.18 -0.21Walden Instructor Sca le (Range 8 - 40) 35.60 35.85 35.81 36.60 Yes 1.00 0.75 0.79Classroom Characteristics Sca le (Range 12 - 60) 39.82 39.70 39.78 40.03 No 0.21 0.33 0.25Course Balance Sca le (Range 3 - 15) 9.36 9.37 9.37 9.30 No -0.06 -0.07 -0.07Univers i ty Services Sca le (Range 3 - 15) 13.48 13.10 13.10 12.88 No -0.60 -0.22 -0.22Walden Instructor Sca le (Range 8 - 40) 36.32 34.75 36.23 35.02 No -1.30 0.27 -1.21Classroom Characteristics Sca le (Range 12 - 60) 40.36 39.98 39.77 40.21 No -0.15 0.23 0.44Course Balance Sca le (Range 3 - 15) 9.33 9.42 9.25 9.40 No 0.07 -0.02 0.15Univers i ty Services Sca le (Range 3 - 15) 12.96 12.41 12.94 12.88 No -0.08 0.47 -0.06Walden Instructor Sca le (Range 8 - 40) 36.03 35.97 35.97 36.20 No 0.17 0.23 0.23Classroom Characteristics Sca le (Range 12 - 60) 40.78 40.45 40.56 40.77 No -0.01 0.32 0.21Course Balance Sca le (Range 3 - 15) 9.25 9.34 9.30 9.28 No 0.03 -0.06 -0.02Univers i ty Services Sca le (Range 3 - 15) 13.41 12.93 13.29 13.13 No -0.28 0.20 -0.16Walden Instructor Sca le (Range 8 - 40) 36.54 36.19 34.94 31.87 Yes -4.67 -4.32 -3.07Classroom Characteristics Sca le (Range 12 - 60) 39.73 39.48 38.68 37.30 No -2.43 -2.18 -1.38Course Balance Sca le (Range 3 - 15) 9.32 9.47 9.51 9.48 No 0.16 0.01 -0.03Univers i ty Services Sca le (Range 3 - 15) 12.97 12.50 13.32 12.67 No -0.30 0.17 -0.65Walden Instructor Sca le (Range 8 - 40) 34.95 34.93 37.48 34.00 No -0.95 -0.93 -3.48Classroom Characteristics Sca le (Range 12 - 60) 39.07 38.43 40.80 39.67 No 0.60 1.24 -1.13Course Balance Sca le (Range 3 - 15) 9.50 9.06 9.50 10.00 No 0.50 0.94 0.50Univers i ty Services Sca le (Range 3 - 15) 12.73 12.71 13.44 n<5 No N/A N/A N/A

University Overall

Table A10: Summary Course Evaluation Scales Over Timeby Course Designation

Means by Term Difference by Term

Changes Over Time

EDUC 8xxx

EDUC 7xxx

EDUC 6xxx

EDAD 8xxx

EDAD 7xxx

Quarterly Course Characteristics Report


NEW FEATURETo filter by

College/School, USE This built in Filter

Course Filter

To filter by Course, USE This built in

FilterCourse Filter

Course Term SurveyIDSurveys

Completedq39 q40 q44 q45 q46

College of Education CoursesRWRCoEL EDUC-1001 EDUC-1001 201230-3 58580 48 4.28 4.39 4.30 4.35 4.38RWRCoEL EDUC-1002 EDUC-1002 201230-5 59211 61 4.39 4.37 4.38 4.29 4.38RWRCoEL EDUC-1003 201230-3 58580 31 4.66 4.62 4.70 4.71 4.70RWRCoEL EDUC-1003 201230-5 59211 12 4.11 4.20 3.90 3.75 3.90RWRCoEL EDUC-1003 43 4.39 4.41 4.30 4.23 4.30RWRCoEL EDUC-1004 EDUC-1004 201230-5 59211 29 4.50 4.50 4.48 4.46 4.54RWRCoEL EDUC-1005 EDUC-1005 201230-3 58580 32 4.23 4.36 4.36 4.40 4.39RWRCoEL EDUC-1006 EDUC-1006 201230-5 59211 30 4.36 4.32 4.33 4.28 4.39RWRCoEL EDUC-1011 EDUC-1011 201230-1 59201 48 4.63 4.64 4.55 4.57 4.65RWRCoEL EDUC-1012 EDUC-1012 201230-1 59201 32 4.32 4.37 4.41 4.33 4.41RWRCoEL EDUC-1013 EDUC-1013 201230-1 59201 24 4.32 4.23 4.18 4.10 4.32RWRCoEL EDUC-1014 201230-3 58580 13 4.85 4.85 4.69 4.50 4.77RWRCoEL EDUC-1014 201230-5 59211 15 4.07 4.13 4.40 4.20 4.40RWRCoEL EDUC-1014 28 4.46 4.49 4.55 4.35 4.59RWRCoEL EDUC-1015 201230-3 58580 17 4.81 4.69 4.63 4.53 4.56RWRCoEL EDUC-1015 201230-5 59211 24 4.71 4.71 4.54 4.41 4.74RWRCoEL EDUC-1015 41 4.76 4.70 4.59 4.47 4.65RWRCoEL EDUC-1016 EDUC-1016 201230-3 58580 10 3.90 3.60 3.00 3.11 3.40RWRCoEL EDUC-2001 EDUC-2001 201230-3 58580 28 4.60 4.58 4.48 4.44 4.46

EDUC-1003Combined Score

Quarterly Instructor Quality Reports (IQR)


Instructor Scale Mean

Survey ID CourseQ1

MeanQ2

MeanQ3

MeanQ4

MeanQ5

MeanQ6

MeanQ7

MeanQ8

MeanCompleted

SurveysTime Course Designation

29.1574. Standard Course Evaluation_201310-1

PUBH-8330-x 3.92 3.31 4.00 3.77 3.92 3.31 3.62 3.31 13.00 15.54 PUBH 8xxx


NURS-6210-x 4.33 4.00 4.06 2.00 3.61 3.39 4.28 3.83 18.00 19.39 NURS 6xxx


NURS-8210-x 4.21 3.93 4.21 3.07 3.21 3.36 4.23 4.00 14.00 21.29 NURS 8xxx


NURS-6150-x 4.00 3.60 4.00 3.80 4.00 3.50 3.90 3.60 10.00 18.10 NURS 6xxx


HINF-6205-x/MMHA-6115-x

3.86 3.57 3.86 4.29 3.86 3.57 4.29 3.43 7.00 20.29 Xlist SHS


NURS-6025-x 4.20 3.87 3.86 4.07 3.33 3.73 4.07 3.73 15.00 17.40 NURS 6xxx


NURS-6410-x 4.50 3.40 4.50 3.90 3.90 3.70 3.80 3.70 10.00 23.60 NURS 6xxx

Quarterly Instructor Quality Reports (IQR)


Fall 2011

Wtr 2011

Spr 2012

Sum 2012

Fall 2011

Wtr 2011

Spr 2012

Sum 2012

Fall 2011

Wtr 2011

Spr 2012

Sum 2012

Q1The instructor encouraged mutual respect among students.

4.52 4.54 4.54 0.34 0.35 0.36 2,270 3,190 2309

Q2The instructor addressed students’ needs as learners.

4.39 4.40 4.41 0.43 0.48 0.48 2,270 3,190 2309

Q3The instructor encouraged independent thinking in discussions.

4.51 4.51 4.52 0.35 0.39 0.40 2,270 3,190 2309

Q4The instructor gave me feedback on submitted assignments within 10 calendar days after the due date

4.42 4.43 4.45 0.53 0.56 0.55 2,270 3,190 2309

Q5The instructor responded to my emails within two business days.

4.36 4.37 4.38 0.48 0.51 0.53 2,271 3,190 2308

Q6The instructor’s feedback showed me how to improve my work.

4.30 4.31 4.32 0.49 0.52 0.54 2,269 3,189 2308

Q7The instructor evaluated my assignments using stated grading policies.

4.50 4.51 4.51 0.38 0.40 0.41 2,270 3,189 2307

Q8The instructor challenged me to improve the depth of my learning.

4.41 4.41 4.43 0.41 0.44 0.45 2,270 3,190 2308

Scale Instructor Scale (8-40) 35.42 35.49 35.55 3.08 3.29 3.34 2,269 3,188 2304

IQR MeansMeans by Term Standard Deviation by Term # Course Sections by Term

All IQR Courses (Graduate

Level/Received Summary Course

Evaluation)

Questions

Course Evaluations

[email protected]

[email protected]

IQR

[email protected]

30

mailto:[email protected]



Documents

Everything you wanted to know about course evaluations but were afraid to send an email to ask… Shari L. Jorissen, EdD, PhD Director of Assessment Office