Embed Size (px)
Citation preview
1
Outcome Based Innovation and
Engagement in STEM Courses: A
Model for Success
2
UNT Student Engagement Index
1. I spend my own time to learn more about things taught in my class.
2. I believe that I am an active participant in my learning.
3. I become immersed when working on challenging activities.
4. In my class I notice when a new idea is introduced.
5. I am willing to help other students with their learning.
3
Presenters and Participants
• Rob Petros, Ph.D. | Assistant Professor, UNT
Department of Chemistry
• Mike Simmons, Ph.D. | Senior Associate
Director, CLEAR
4
Participant Outcomes
• Identify relevant student and faculty data related to STEM course redesign
• Describe effective, learner focused instructional practices
• Describe the connection between outcome based education and improvement of instruction
• Describe outcome based assessment strategies
• Identify basic milestones for success
• Discuss how various assessment and instructional best practices can be applied to a variety of settings and institutions
• Develop strategies for implementing instructional and assessment best practices at course, department and institutional level.
• Develop a basic outline of ideas for incorporating outcome based, learner centered instruction and assessment into their own courses, programs, or institutions.
5
Roadmap of Presentation
• The state of STEM/recent results
• Course data and information
• NextGen concepts
• Chemistry Redesign
– Engagement
– Learner Centered and Strategies
• Questions and Feedback
6
Dr. Rob Petros: Chemistry
The Problem
• Only 40% of entering college students that declare STEM majors complete degrees in STEM disciplines
• Economic projections indicate 1 million more STEM professionals will be needed in the US than will be produced over the next decade at current rates
Why are we losing these students?
1. Lack of inspiration in low level science classes
2. Frustration with the ‘weed out’ courses
3. STEM graduate programs prepare researchers not educators
7
Problem Solved!?
• Redesign has increased enrollment from 145 to 192 (32%), while reducing actual class size from 145 to 96 ($120K+ in annual tuition).
• Retention rate has increased dramatically ($250K?)
* first year incorporating engaged learning activities (peer group learning)
• Percent effort for teaching course has decreased dramatically
# of students
receiving D, F, or W
# of students
registered
retention
rate (%)
2013/2014* 39 377 90
2012/2013 83 285 71
2011/2012 83 291 71
2010/2011 87 248 65
2009/2010 77 218 65
8
Student Performance During Redesign
0
10
20
30
40
50
60
May-11 May-12 May-13 May-14† May-14‡ May-14*
% o
f cl
ass
Results from ACS Standardized Final Exam
Above Natl. Average Bottom quartile
† All students taking second semester with Petros
‡ Students that were in Petros' section for both OChem I and II
* Students that were in Petros' section for only OChem II
podcasts
introduced
lecture
eliminated
NextGen
launched
9
Brainstorming: What do you know about your
students?
Identify three types of data/information that
is available about your students now.
Identify three types of data/information about
your students that you wish you had!
10
Importance of Course Data
• We hope to show:
– What data can come from courses?
– Why is course data important?
– How can course data be used?
– What are the issues and challenges?
– What’s happening at UNT and other
institutions?
– Implications and opportunities
11
NextGen Course Redesign
Course Objectives
Assessment*Instructional
Strategies
*Carriveau, R. S., Connecting the dots : developing student learning
outcomes & outcome based assessments. Fancy Fox Publications:
Denton, TX, 2010.
12
Student Learning Outcomes
• What the student is expected to know and be able to do
– Useful in the classroom• To students
• To Faculty
– Connects to “larger” goals• Program
• Institution
• Accreditation
13
1. Goal: The student will understand literature (by)
1.1. General Learning Outcome (GLO): The student will demonstrate
accurate, critical, analytic reading of literature (by)
specific learning outcome statement (sLO):
The student will (be able to)
1.1.1 Identify important and supporting details
1.1.2 Recognize assumptions and inferences
1.1.3 Identify sequence of events
1.1.4 Determine the main idea/theme of a passage or
piece of literatureSource: Carriveau, R.S. (2011), Connecting the Dots: Developing Student Learning Outcomes and Outcome-Based Assessments
Example of Three Level SLO Structure
14
Code each item to a specific
learning outcome
1 Goal: The student will understand literature (particular period or genre)
1.1 General Learning Outcome (GLO): The student will demonstrate
accurate, critical, analytic reading of literature.
Specific learning outcome statements (sLO):
1.1.1 Determine the main idea/theme of a passage or piece of literature.
Test Item or Rubric Dimension
1.1.1 What was this passage mostly about? (Could also be CR)
A. Bias and prejudice can affect intellectual growth.*
B. Economic growth is impacted by bias and prejudice.
C. Current thoughts on bias and prejudice lack insight.Source: Carriveau, R.S. (2011), Connecting the Dots: Developing Student Learning Outcomes and Outcome-Based Assessments
15
The Link Between Outcomes and ItemsCalculating Outcome Attainment Values
Goal
1
Avg = 83
Source: Carriveau, R.S. (2011), Connecting the Dots: Developing Student Learning Outcomes and Outcome-Based Assessments
1. 87
2. 90
3. 65
4. 58
5. 63
6. 52
7. 66
8. 77
9. 84
10 93
11. 96
12. 88
13. 82
14. 88
15. 90
16. 80
17. 92
18. 81
19. 81
20. 82
Specific Outcome
1.1.1
Avg = 81
Specific Outcome
1.1.2
Ave = 60
Specific Outcome
1.1.3
Avg = 88
Specific Outcome
1.2.1
Avg = 85
Specific Outcome
1.2.2
Avg = 85
General Outcome
1.1
Avg = 81
General Outcome
1.2
Avg = 85
16
ExamSoft “Categories”
Use the “Categories” feature to create three levels
of outcome statements
17
Tag Learning
Outcomes for
each row
Weight
rows
18
Reporting Outcome Attainment
at the Program Level
Program
Goal#1
% Correct
Goal 1
% Correct
Goal 2
% Correct
GLO 1.2
% Correct
GLO 2.3
% Correct
sLO 1.2.3
% Correct
# Exams 40 35 17 11 5 3
# Items 615 282 21 71 2 14
Group Average 85.23% 84.3% 83.8% 86.0% 92.9% 86.5%
Student A 75.77% 71.84% 72.73% 50% 100% 100%
Student B 89.71% 88% 90.91% 100% 100% 92.3%
Student C 83.82% 82% 100% 100% 84.6% 50%
Student D 77.45% 80% 72.73% 75% 100% 84.62%
19
Activity
• Write a Three Level Outcome Statement
20
Core Principles
• What are the essential principles every student should take away from the course?
• Who are your students and what are you preparing them for?
• Can your course teach important lessons beyond your own discipline-specific content?
• How will you assess student attainment?
21
Course Objectives
• Goal 1 – Students will understand molecular structure and its implication
for basic chemical reactivity
• Goal 2 – Students will understand the concept of isomerism in organic
chemistry
• Goal 3 – Students will understand various mechanistic pathways
commonly encountered in organic chemistry
• Goal 4 – Students will understand how spectroscopy can be used to
determine molecular structure
• Goal 5 – Students will understand the relationship of organic chemistry
to their everyday lives
• Goal 6 – Students will be aware of the impact of the globalization of
scientific research on US competitiveness in science and technology and
be motivated to pursue and obtain degrees in STEM majors
• Goal 7 – Students will demonstrate elements of collaboration, leadership,
innovation, problem solving, creativity, teamwork, and critical thinking
22
Objectives and Professional Standards
• American Chemical
Society’s Criterion
Referencing Project
underway, but not
completed.
• Lumina Foundation – DQP
• University Goals
• Currently based on
important topics from ACS
study guide
23
24
25
Online Content
252 podcasts totaling ~23h of recorded material produced covering all topics
from both semesters posted on UNT’s iTunesU site
26
STEM Incentives Program
• Information on graduate school in the sciences
• Careers in chemistry
• Where my classmates are now
• Need for diversity in STEM
• Science and Engineering Indicators 2012
• Activity demonstrating the globalization of scientific research
– List authors’ home institution for first 20 articles in the first issue of JACS in 1982, 1992, 2002, and 2012
27
STEM Incentives Program
28
Engaged Learning Activities
*Office of Science and Technology Policy – Engage to Excel (2012)
29
Class Time – Small Group Work
• 192 students divided into 32 groups of 6
• Each group meets 2hr/wk
• Can observe skills related to course goals 5-7
30
Group Assignments
31
32
Formative Assessment On Demand
33
More Robust Analysis
Learning Objective # Assessments # Items Group Average
1. Students will understand molecular structure and its
implication for basic chemical reactivity 11 66 73.36%
1.1 Students will know and apply the naming system for
organic compounds 11 23 71.06%
1.1.1 Students will recognize and correctly name
molecules containing functional groups 9 23 71.24%
1.1.2 Students will correctly name stereoisomers 2 1 70.56%
1.2 Students will understand the relationship between
structure, hybridization, resonance, and aromaticity 2 21 81.80%
1.2.1 Students will identify the correct hybridization
state for C, N, O and other relevant atoms 2 18 81.19%
1.2.2 Students will identify factors that lead to
stabilization in resonance structures 2 3 84.86%
1.3 Students will understand the role of acidity⁄basicity in
reactions 4 23 70.67%
1.3.1 Students will predict products of an acid⁄base
reaction2 2 73.89%
1.3.2 Students will identify acid⁄base conjugate pairs 2 6 89.73%
1.3.3 Students will use pKa values to predict relative
acidity 4 15 66.86%
34
Tracking Student Learning in Real Time
First Half of
Semester
Second Half of
Semester
Number
of Items
Group
Average
Number
of Items
Group
Average
1.1.1 Students will recognize and correctly name
molecules containing functional groups10 91% 5 97%
1.3.3 Students will identify acid⁄base trends for
common functional groups10 82% 2 69%
2.1.1 Students will recognize the mechanisms for
electrophilic and nucleophilic aromatic substitution
reactions2 38% 1 84%
2.1.2 Students will predict the site of substitution
based on substituents present on the aromatic ring2 44% 1 85%
2.1.5 Students will predict products of reactions given
specific starting materials2 80% 12 87%
3.1.1 Students will predict reaction products based on
starting materials35 76% 29 83%
3.1.2 Students will differentiate between reversible
and irreversible addition reactions8 87% 4 93%
5.2.1 Students will analyze spectral data to predict
molecular structure12 84% 9 88%
35
Rob Petros – Improved from 56% to 71%
Fourth Exam Performance (split into two section)
Remediation Assignment Performance (sections combined)
36
Rob Petros – Early intervention
Student Feedback
• Student feedback
released on
learning outcome
performance and
areas of
strength/weakness
• Displayed score
upon exam exit
37
Lessons Learned
1. More students can be reached through engaged learning environments
2. Real-time analysis of student attainment data can be used to improve student performance
3. Setting expectations for students is critical
4. Participation must be part of students’ grades
5. Groups of all male or all female students appeared to be less engaged
6. Ideally, roles should be assigned to each group member that can then be rotated over the course of the semester
38
Engagement in the Class
• The purpose of the UNT Student Engagement Inventory (SEI) is to provide information about student engagement that instructors can use to make meaningful continuous improvements to their course.
• Part 1, Personal Internal Motivation, is a measure of the students’ personal internal predisposition to be engaged in learning.
• Part 2, Classroom Interaction is a measure of the students’ perception of engagement based on interactions in the teacher-created classroom learning environment.
• Part 3, Social Interaction, is a measure of the social interaction among classmates in interactive classroom experiences.
39
SEI – Classroom Interaction
• Part 2 – Classroom Interaction total scale
score.
– I am satisfied with what I learned in my class.
– Opportunities to interact with my instructor
were provided in my course.
– I interact positively with my instructors.
– I am encouraged in my class to reflect on
what I learn.
– I work harder in my class than I expected to.
40
SEI – Social Interaction
• Part 3 – Social Interaction in the classroom total scale score.
– I actively participate when a group project is assigned in a class (in-class or on-line).
– Opportunities to collaborate with fellow students are provided in my class.
– I actively participate in study groups.
– My class requires the use of electronic media for discussions.
– I contribute to class discussions (in-class or on-line).
41
Review
• Identify relevant student and faculty data related to STEM course redesign
• Describe effective, learner focused instructional practices
• Describe the connection between outcome based education and improvement of instruction
• Describe outcome based assessment strategies
• Identify basic milestones for success
• Discuss how various assessment and instructional best practices can be applied to a variety of settings and institutions
• Develop strategies for implementing instructional and assessment best practices at course, department and institutional level.
• Develop a basic outline of ideas for incorporating outcome based, learner centered instruction and assessment into their own courses, programs, or institutions.
43
Planning
• Learning Strategies worksheet
• Teaching Strategies worksheet
• Assessment Strategies worksheet
