Upload
melanie-white
View
217
Download
3
Tags:
Embed Size (px)
Citation preview
Assessing SAGES with NSSE Assessing SAGES with NSSE datadataOffice of Institutional ResearchSeptember 25th, 2007
IntroductionIntroductionToday’s talk presents an analysis of
the impact of SAGES participation on a number of NSSE survey items
Focus on first-year responses only as no full-implementation classes have graduated
Report will first focus on NSSE benchmark scales followed by a discussion of individual NSSE items related to SAGES learning goals
AnalysisAnalysisThe seven years of NSSE data collected
by Case has been divided into four groups: ◦ Pre-SAGES (2000, 2001)◦ Students in the SAGES pilot (2002-2004)◦ Students not in the SAGES pilot (2002-2004)◦ Full-Implementation Classes (2005-2006)
An analysis of covariance (ANCOVA) was conducted to determine group differences on NSSE benchmarks and survey items
All analyses control for first-year reported major
NSSE BenchmarksNSSE Benchmarks In order to condense NSSE’s 80+ survey
items into easily discussed and analyzed scales, NSSE has developed five conceptually and statistically valid “benchmark” scales.
Scales include “Active and Collaborative Learning,” “Academic Challenge,” “Student-Faculty Interactions,” and “Supportive Campus Environment.”
In 2005 NSSE changed the calculation of the fifth benchmark, “Enriching Educational Experiences,” making pre-2005 longitudinal comparisons of this scale unreliable.
Active and Collaborative Active and Collaborative LearningLearningMeasures the extent to which
students engage in classroom activities and collaborate with others to solve problems.
Items include: asked questions in class, made class presentations, worked with other students on projects during class, worked with other students outside of class.
Active and Collaborative Learning
• Results indicated that those in the SAGES pilot had significantly higher scores than all other students.
• Additionally, those in the full-implementation of SAGES had, on average, significantly higher scores than pre-SAGES students.
Active and Collaborative Learning
• Results indicated that those in the SAGES pilot had significantly higher scores than all other students.
• Additionally, those in the full-implementation of SAGES had significantly higher scores than pre-SAGES students.
Academic ChallengeAcademic ChallengeMeasures the extent to which
students exert—and institutions demand—academic effort.
Items include: time spent preparing for class, number of assigned textbooks, and the extent to which the campus environment is perceived to emphasize academics.
Academic Challenge
• Results revealed no group differences on the Academic Challenge measure
Student-Faculty Student-Faculty InteractionInteractionMeasures the extent to which
students learn and solve problems by interacting with faculty members.
Items include: the extent to which students discussed grades with faculty, worked with faculty members on activities other than coursework, and discussed career plans with faculty.
Student-Faculty Interaction
• Results indicated that those in the SAGES pilot had significantly higher scores than those in the full-implementation class
• There were no significant differences among the remaining three groups
Student-Faculty Interaction
• Results indicated that those in the SAGES pilot had significantly higher scores than those in the full-implementation classes
• There were no significant differences among the remaining three groups
Supportive Campus Supportive Campus EnvironmentEnvironmentMeasures the extent to which students
believe that the institution is committed to their success and cultivates positive relations among different groups on campus.
Items include: quality of relationships with faculty, quality of relationships with peers, and the extent to which the campus environment is perceived to provide support to succeed socially and academically.
Supportive Campus Environment
• Results indicated that those in the SAGES pilot had significantly higher scores than pre-SAGES students and students in the full-implementation classes
• There were no significant differences among the remaining three groups
Supportive Campus Environment
• Results indicated that those in the SAGES pilot had significantly higher scores than pre-SAGES students and students in the full-implementation classes
• There were no significant differences among the remaining three groups
Individual ItemsIndividual ItemsSAGES goals focus on classroom
participation, developing writing and speaking skills, and academic advising.
The following slides examine 6 individual NSSE items relevant to these goals:◦ Asked questions in class◦ Gave a class presentation◦ My experience at Case has contributed to my
ability to write clearly and effectively◦ My experience at Case has contributed to my
ability to speak clearly and effectively◦ Satisfaction with University administration◦ Satisfaction with academic advising
Individual ItemsIndividual Items
• For both of these items, those in the SAGES pilot outperformed all other groups; however, students in the full-implementation classes had significantly higher scores than pre-SAGES students.
Individual ItemsIndividual Items
• For both of these items, there were no differences between those in the SAGES pilot and those in the full-implementation classes.
• Those in the SAGES pilot and the full-implementation classes outperformed the other two groups.
Individual ItemsIndividual Items
• For “Satisfaction with Advising” there were no year-to-year differences.
• For “Satisfaction with Administration,” those in the SAGES pilot had significantly higher scores than all other groups. Those in the full-implementation classes had significantly lower scores than all other groups
Conclusions – BenchmarksConclusions – BenchmarksFor three of the four benchmark
scales, students in the SAGES pilot significantly outperformed at least one other group.
Of the four benchmark scales, only one—Active and Collaborative Learning—significantly increased from pre-SAGES to the full-implementation class.
Conclusions – Individual Conclusions – Individual itemsitemsStudents in the full-implementation
classes scored significantly higher than pre-SAGES students on a number of items:◦ Asked questions in class◦ Gave a class presentation◦ My experience at Case has contributed to
my ability to write clearly and effectively◦ My experience at Case has contributed to
my ability to speak clearly and effectively
There were no group differences on ratings of satisfaction with advising
There was a significant drop in ratings of satisfaction with administration from pre-SAGES to the full-implementation classes.
LimitationsLimitationsNSSE is a survey of the entire
experience at Case, not an assessment of SAGES.
These analyses statistically control for first-year expected major only.
Significant differences from pre- to post-full-implementation of SAGES can be inferred to be due in part to the change in curriculum, but may also be due to unmeasured—or un-measurable—influences.
Thank You!Thank You!Questions? Concerns?
Contact: Tom [email protected]