Influential Interventions: Improving STEM Learning Outcomes for Underrepresented Students

James Diedrick, Associate Dean of the CollegeDrew Homa, Academic Assessment Coordinator

25th International Conference on The First-Year Experience Vancouver, B. C. • July 17, 2012

Agnes Scott CollegeAtlanta, GeorgiaStudent Body as of Fall 2011:

883 students, representing 41 states and territories and 29 countries89% of traditional students live on campus11% are international studentsMore than 40% are students of color.About 40% will study abroad before they graduateHistorically and presently, Agnes Scott students have earned academia’s most prestigious scholarships including the Marshall, Rhodes, Fulbright, Goldwater, the Pickering Fellowship and the Gates Millennium Scholarship.

Students in the Generating Excellence in Math & Science Summer Scholars Program (GEMS), Summer 2011

Our Commitment to STEM Education

Majors: Biology, Biochemistry and Molecular Biology, Chemistry, Mathematics, Mathematics-Physics, Mathematics-Economics, Neuroscience, Public Health, Physics & Astronomy

Dual degree programs in Engineering, Computer Science and Nursing with Emory University & Georgia Tech

State-of-the art facilities in new science building, observatory on campus

Emphasis on supporting & encouraging women in pursuit of STEM fields

Increasing Ethnic Diversity 1992-2011

74.2%

3.9%4.0%

17.9%

White Non-resident International

1992

57.9%

5.2%4.5%

32.5%

White

2002

40.4%

10.8%4.7%

44.1%

2011

Increasing Ethnic Diversity 1992-2011

1992 2002 20110%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

American Indian/Alaska Native

Unknown

Hispanic

Two or more races

Asian/Pacific Islander

Non-resident International

African American

White

Diversity on Campus: Total

UndergraduatePopulation

1992 2002 20110%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

American Indian/Alaska Native

Unknown

Hispanic

Two or more races

Asian/Pacific Islander

Non-resident International

African American

White

Diversity on Campus: Total

First-YearPopulation

Achievement Gap for URM STEM Students

Percentage of STEM Majors Having a GPA ≥3.33 (2010)

Analysis of performance in introductory STEM courses clearly demonstrated a trend that faculty had long noticed (data from 2004-2009)

Achievement gap persists even for students who choose to major in a STEM discipline

GEMS Program

Gen

erat

ing

Exce

llen

cein

Mat

h &

Sci

ence

B

egan

200

7

Yea

r-ro

und

prog

ram

aim

ed a

t su

ppor

ting

the

su

cces

s of

stu

dent

s pu

rsui

ng m

ajor

s in

mat

hem

atic

s an

d/or

the

nat

ural

and

phy

sica

l sc

ienc

es

GEM

S Su

mm

er S

chol

ar P

rogr

am,

2009

-201

1 

A

ctiv

itie

s in

clud

e bi

-mon

thly

mee

ting

s, e

xcur

sion

s,

serv

ice

proj

ects

and

col

labo

rati

on w

ith

ASC f

acul

ty

Stu

dent

s en

cour

aged

to

take

adv

anta

ge o

f ac

adem

ic s

uppo

rt c

ente

rs.

L

ivin

g-Le

arni

ng c

ompo

nent

ini

tiat

ed f

or 2

012-

13

year

.GEMS Summer Scholars on field trip to Georgia Aquarium, 2011

Factors influencing success & persistence in STEM

Family Background

Skills & Abilities

Prior Schooling

Classroom ClimateStereotype Threat

Feedback on Learning

Inquiry-Based Learning

Exposure to Real-World Applications &

Careers

Extracurricular Activities

Peer Group Interactions

Academic Integration

Social Integration

Re-evaluate commitments

& goals

Decision to enroll in STEM

class/majorDecision to persist or

depart from STEM

Potential for peer-led Supplemental Instruction in gateway math & science courses to have a positive influence

University of Michigan Center for Research and Learning

Teagle Foundation Support for Interventions to Promote STEM Student Success

Two-year grant received in collaboration with Davidson College to develop academic support initiatives that particularly benefit first generation and URM students

Initiative runs from August 2011-May 2013

Goal includes use of student learning outcome data to continue the levels of academic support beyond the funding period

Initiative includes joint meetings of two institutions, workshops on campus climate issues, stereotype threat, STEM pedagogy

Project Objectives

Increase student success and persistence in STEM disciplines by adding peer-led supplemental instruction (SI) sessions to gateway math and science courses All students are encouraged to attend SI sessions Specific impact to URM and first generation students determined through

assessment SI has been associated with more dramatic gains among URM students than

among their peers*

Addressing/improving campus/classroom climate issues (Diverse Learning Environments Survey (DLE) administered Spring 2012, comparative data to be released September 2012)

* Rath, Kenneth, et. al. “Supplemental Instruction in Introductory Biology I: Enhancing the Performance and Retention of URM Students” (CBE – Life Science Education 6 [2007]:203-216).

New forms of academic supportCoordinator for Resource Center for Math & Science (RCMS)

Assisting in facilitating the implementation and assessment of the SI program

Supporting & coordinating the work of all LA’s and peer tutors (individual peer tutors have been in place for two decades)

Increasing the level & variety of academic support available to our students in math and the sciences

Peer Learning Assistants (LA’s) Juniors or seniors selected by faculty

Training: 2-day session in August, ½ day session in January, and on-going training at regular meetings throughout each semester

Responsibilities: attending course lectures, leading SI sessions, developing workshop content (to varying extents), holding 1:1 tutoring hours & attending regular staff meetings

Grant support allowed for hiring students not eligible for traditional work-study

Math & Science Learning Center CoordinatorDr. Molly Smith, Math & Science Learning Center Coordinator

Ph.D. in Chemical Engineering from University of Michigan

6 years experience working in a university environment as a graduate student, including voluntary service roles in coordination, leadership and planning of education programs and events

5 years corporate experience managing projects and programs involving team members in multiple locations and partners at other companies and universities; included teaching intensive mini-courses on biology and chemistry fundamentals for employees

Dr. Molly Smith

Ethnic & Racial Diversity of LA’s

Total of 11 LA’s for 2011-12 and 12 LA’s for 2012-13

55%36%

9%

White

2011-12

50%

25%

8%

17%

White

2012-13

SI Implementation: Overview

FALL 2011Biology 191: Cell and Animal

BiologyChemistry 101: Fundamental

Concepts of Matter and Reactions

Math 118: Calculus I

Implemented SI in 6 gateway math & science courses (13 total sections, 194 individual students) during 2011-2012

SPRING 2012Biology 192: Molecular Biology

and GeneticsChemistry 102: Periodicity and

Chemical ReactionsMath 118: Calculus IMath 119: Calculus II

COMMON All students encouraged to

attend SI

SI sessions offered weekly, in the evenings

SI sessions led by peer Learning Assistants (LA’s)

LA’s attend course lectures as often as possible

VARIED Method of encouraging

attendance

Number of sessions per week

Level of guidance professors provided LA’s for SI content

Some aspects of implementation were common across courses, while othersvaried by course/professor

SI Implementation: Overview

Supplemental Instruction Rubric

Session Date & Day of Week: __________________ SI Leader: ______________

Course: ____________________ Course Instructor: _______________________

Objective: What are the two most difficult concepts the students need to work on today?__________________________________________________________________________________________________________________________________

Beginning reminders:

Arrange seats in a circleHand out Participation LogSet agenda with groupRemember to relax and be flexible!

Possible Processes: Informal Quiz, Matrix, Reciprocal Questioning, Paired Problem-Solving, Turn to Use Your Partner, Note Processing, Problem-Solving Rubric, Formal Definition (or ID’s), Text Review (Divide and Conquer), Pictorial Representations, Sequencing

Closure options: Predict next lecture, summarize session, informal quiz, 1-minute writing

--Rubric adapted from Deanna, Martin C. and David R. Arendale, Supplemental Instruction: Improving First-Year Student Success in High-Risk Courses. (National Resource Center for the Freshman Year Experience, University of South Carolina, 1992).

Content to cover Processes to use*

Supplemental Instruction Rubric - continued

SI Implementation: Details Fall 2011

Definitions for level of guidance provided to LA’sLow: LA’s develop nearly all content, based on their observations during lecture & recollections from when they took the course

Mid: Profs & LA’s meet weekly to discuss possible SI topics and profs provide some guidance on specific exercises/problems to do

High: Profs provide detailed worksheets and meet regularly with LA’s

Course Enrolled# Sections &

Profs LA’s SI/weekAttendanceIncentive

Guidance to LAs*

BIO 191 953 sections

3 profs4 4 Encourage mid

CHEM 101 763 sections

2 profs2 4 Require high

Math 118 291 section

1 prof2 2

Exam Bonus Points

(starting½ way)high

* Each CHEM 102 professor used a different approach to encourage attendance

Course Enrolled# Sections &

Profs LA’s SI/WeekAttendanceIncentive

Guidance to LAs

BIO 19263

2 sections2 profs

3 3 Encourage Low

CHEM 102 602 sections

2 profs2 4

Encourage*

Require*

Mid*

High*

MATH 118 301 section

1 prof2 2 Exam bonus

pointsHigh

Math 119 291 section

1 prof1 2

Exam bonus points High

SI Implementation: Details Spring 2012

Assessment Methodologies

Quantitative DataSI session attendance logs

Mid-term and Final grades

Descriptive student variables (class year, URM, FGEN, etc.)

Qualitative DataFocus group interviews with First-Year students in STEM gateway courses

Meetings and focus groups with Learning Assistants

Workshops and discussions with STEM faculty using LA’s and SI

Session Attendance Trends

194 Individual students participated

Of those, 135 (70%) were “Target” students, either FGEN or URM

Since some students took more than one course, we have a total of 346 class-students who we studied.

Of those 346 students, 254 (73%) of them participated in at least 1 workshop

The ratio of Target students who participated in workshops is the same as those attending the class (108 of 156 = 70%.)

SI Attendees Perform Better Overall

191 192 101 102 118 FA 118 SP 1190

1

2

3

4

SI GroupNon-SI Group

Average Final Grade

BIO CHEM MATH

SI Group: attended 2 or more sessions Non-SI Group: attended 0 or 1 session(s)

Times Attended

Semester Class # students 0 1-2 3-5 6+

11/FA BIO-191 28 46% 11% 32% 11%11/FA BIO-191 41 22% 24% 20% 34%11/FA BIO-191 17 41% 29% 6% 24%11/FA CHE-101 24 8% 8% 33% 50%11/FA CHE-101 27 37% 15% 30% 19%11/FA CHE-101 23 17% 22% 22% 39%11/FA MAT-118 26 31% 42% 19% 8%12/SP BIO-192 21 33% 19% 14% 33%12/SP BIO-192 33 12% 15% 30% 42%12/SP CHE-102 23 30% 13% 17% 35%12/SP CHE-102 37 41% 19% 16% 22%12/SP MAT-118 25 24% 16% 16% 44%12/SP MAT-119 24 4% 25% 33% 38%

Varied Levels of Session Attendance

Grade Target Students Non-Target StudentsImproved or was an A at mid-term (40%) 95 (55%) 55No Change (22%) 53 (43%) 43Worsened (38%) 89 (16%) 16

From Mid-term Grade to Final Grade...

...Based on Number of Sessions Attended

Positive Impact of Regular SI Attendance

I mproved or A at mid-term0

0.1

0.2

0.3

0.4

0.5

0

Percentage of Students

# times attended

I mproved or A at mid-term0

0.1

0.2

0.3

0.4

0.5

0

Percentage of Students

# times attended

From Mid-term Grade to Final GradeFrom Fall 2011 to Spring 2012

Of the 102 students who took an SI class in both semesters:

 

More improvement in Spring than

Fall

Same improvement in Spring and

Fall

Less improvement in Spring than

Fall

Earned A's at Mid-term and

Final, both Fall and Spring

More sessions attended in Spring than Fall 19 6 14 2

Same sessions attended in Spring as in Fall 4 5 2 2

Fewer sessions attended in Spring than Fall 10 3 21 3

No sessions attended Fall or Spring 11

Comparison of Fall and Spring Performance

Samples of Student Feedback—first years

[My LA] helps me to restore my faith in chemistry and want to study more. She has been very helpful, kind, patient, and honest. She explains topics in real world scenarios by making references to reality TV shows, elementary schools, and everyday life situations. At times when the textbook and word problems become to wordy she breaks them down into smaller simpler pieces that help me to piece things together and think through the question in a more useful, sensible way instead of getting stressed out and giving up. She teaches us with authority while showing humility and compassion, which makes it easier to relate to her and ask her questions that might seem trivial. I do not know what I would have done this semester without her help.

The [LAs] were complete life savers- always so friendly and able to help. I absolutely loved them and was privileged to work with them.

“What were some of your favorite things about being a Teagle Learning Assistant?”

Getting to work with students and feeling a sense of accomplishment knowing I helped them learn something.

Students trusted and respected me and I had an active role in helping them succeed.

Helping others understand concepts and seeing them apply them to all areas.

Having a close bonding with first year calculus students Having the experience of teaching college material. Having fun creating new forms of teaching and activities Sharing the excitement of every “aha” moment; and of course

introducing the world’s coolest subject to a new generation

Samples of LA Feedback

Next Steps/Future Plans…

SI Rubric used by all faculty

Move from encourage to require attendance at SI sessions

Facebook page to facilitate contact between students and LAs

Sharing positive impacts we’ve seen with students

Analyzing DLE data and comparing ASC with national data

Adding additional classes to SI/LA initiative (Physics)

Working with Davidson College to compare findings and best practices

Selected ReferencesBrown, Marie Kendall, et al. “Teaching for Retention in Science, Engineering and Math Disciplines: A

Guide for Faculty” University of Michigan Center for Research and Learning Occasional Paper No. 25, 2009.

Davis, Jeff. The First-Generation Student Experience: Who They Are, Their Characteristics, and Strategies for Improving Their Persistence and Success. Stirling, Virginia: Stylus, 2010.

Deanna, Martin C. and David R. Arendale. Supplemental Instruction: Improving First-Year Student Success in High-Risk Courses. The Freshman Year Experience: Monograph Series Number 7. 3rd Ed. Columbia, South Carolina: National Resource Center for the Freshman Year Experience, 1992.

Eagan, Kevin & Jessica Sharkness. “A Strong Start in the Sciences: Factors Influencing Minority Students’ Academic and Social Engagement.” Conference Presentation, 28th Annual Conference on the First Year Experience, Orlando, FL February 2009.

Freeman, Scott, et. al. “Prescribed Active Learning Increases Performance in Introductory Biology. CBE—Life Sciences Education, 6 (2007): 132–139.

Jones Taylor, Valerie and Gregory M. Walton. “Stereotype Threat Undermines Academic Learning (Personality and Social Psychology Bulletin 37 (2011): 1055–1067.

Rath, Kenneth A., et. al. “Supplemental Instruction in Introductory Biology I: Enhancing the Performance and Retention of Underrepresented Minority Students.” CBE—Life Sciences Education 6 (2007): 203–216.

Steele, Claude. “A Threat in the Air: How Stereotypes Shape Intellectual Identity and Performance” American Psychologist 52 (1997): 613-629.

Thanks to:

Dr. Molly Smith, Math & Science Learning Center Coordinator

Dr. Lerita Coleman Brown, Director of the Science Center for Women

Jennifer Cannady, AssistantDean of the College,Director of AcademicAdvising & Student Success

Agnes Scott students, staff& STEM faculty

Agnes Scott ITS StaffBradley Observatory

Dr. Laura Palucki-Blake, Assistant Director of the Cooperative Institutional Research Program (CIRP) at the Higher Education Research Institute, UCLA

Dr. Valerie Jones Taylor, Assistant Professor of Psychology at Spelman College

The National Center for the First-Year Experience and Students in Transition

The Teagle Foundation

Dr. Laura Palucki Blake

And also thanks to: