Students’ Ratings of University Teaching:A Data Mining Analysis
Chuck DziubanResearch Initiative for Teaching Effectiveness
Ida Cook Department of Sociology and Anthropology
Faculty Senate
Morgan WangDepartment of Statistics
University of Central Florida
Rationale for the Study
• Teaching evaluation data for a 3-year period were available to allow a comparison of two different sets of items (UCF/BOR).
• Responding to faculty interest, the UCF Faculty Senate requested that an evaluation of the Student Evaluation of Instruction measures be performed. (FS 1995-96-11)
The Instrument: UCF Items
• Feedback on your performance in this course
• The instructor’s interest in your learning
• Use of class time
• The instructor’s overall organization of the course
• Continuity from one class meeting to the next
• The pace of the course
• The instructor’s assessment of your progress
• The text and supplemental learning materials used
The Instrument:Board of Regent Items
• Description of course objectives and assignments
• Communication of ideas and information
• Expression of expectations for performance
• Availability to assist students In or outside of class
• Respect and concern for students
• Stimulation and interest in the course
• Facilitation of learning
• Overall assessment of instructor
The Study LayoutApproximately 450,000 student responses
Five Colleges:• Arts and Sciences
• Business Administration
• Education
• Engineering
• Health and Public Affairs
Three Levels:• Lower Undergraduate
• Upper Undergraduate
• Graduate
Three Years:• 1996-97
• 1997-98
• 1998-99
Overall Rating of the Instructor
Binary Decision TreeSAS Enterprise Miner
Remaining15 Items
• Level • Year• College
DevelopmentalSample 1
DevelopmentalSample 2
DevelopmentalSample 3 Validation
Sample
Predictors
Decision Tree Example
85.9%n=11,286
85.8%n=6,460
91.5%n=2,079
72.7%n=378
86.7%n=2,369
86.5%n=5,639
74.8%n=821
94.1%n=1,036
89.1%n=1,043
64.7%n=148
79.6%n=230
88.4%n=3,263
84.1%n=2,376
68.9%n=298
78.5%n=526
Arts & Sciences,Business Admin.,Hospitality Mgmt. Education Engineering
Health & Pub. Affairs
F2F, E, M W
females males A&S BA & Hosp. mgmt
F2F E, M, W E, MF2F
Overall
Rules Leading to Overall Rating of Excellent
Questions Ex VG G F P
Facilitation of Learning ●
Communication of Ideas and Information
●
Probability = .96N=46,805
Rules Leading to Overall Rating of Excellent
Questions Ex VG G F P
Facilitation of learning ●
Communication of ideas and information
●
Organization of the course ●
Assessment of student progress
● ●
Probability = .85N=3,462
Rules Leading to Overall Rating of Excellent
Questions Ex VG G F P
Facilitation of learning ●
Communication of ideas and information
●
Organization of the course ●
Instructor interested in your learning
●
Probability = .78N=6,215
Overall Ratings Unadjusted and Adjusted for ‘Excellent’ Decision
Rules
Adjusted for Rule
College Unadjusted 1 2 3
CAS 42% 95% 85% 80%
CBA 36% 95% 83% 77%
COE 54% 97% 86% 75%
CECS 32% 96% 84% 78%
COHPA 48% 96% 86% 74%
For more information…
Research Initiative for Teaching Effectiveness
Dr. Chuck Dziuban407-823-5478
Dr. Patsy Moskal407-823-0283
http://rite.ucf.edu
Recommended
