Jay DubnerColumbia University

2008AGU Fall MeetingDecember 16, 2008

Teacher Research Programs:An Effective Form of Professional Development to Increase

Student Achievement & Benefit the Economy

Practice What You TeachColumbia University’s Summer Research Program for Science Teachers was established in 1990 by Dr. Samuel Silverstein to contribute to the improvement of science achievement of students by providing New York metropolitan area middle and high school teachers with experiences in the practice of science. Teachers become members of research teams for 2 consecutive summers.

3

1990-2008• 241 middle and high

school science teachers

• 88% public schools• 55% women• 46% minorities

7

- Program participation open to New York metropolitan area public, private & parochial science teachers

- Program receives 5 to 7 applications for each available Fellowship

- 12 new participants are accepted each summer, joined by 12 returning participants from previous summer (peer coaches)

Program BackgroundProgram Background

Two Summer Commitment - 8 Weeks immersion in each of two consecutive summers

- 4 days per week in research lab- 1 day per week in pedagogical meeting

Weekly summer seminar series serves several purposes:- Broaden Science Knowledge: Columbia’s faculty or a speaker from outside the University gives a talk on a topic of scientific interest. - Develop a Professional Learning Community: Meetings provide protected time for teachers’ energy to be directed towards solving persistent and common problems encountered by students. Teachers lead hands-on demonstrations, engage in discussions of common classroom problems, and exchange ideas on solutions that work.- Peer Coaching: Second-year participants provide guidance and encouragement to first-year participants- Program Performance: Meetings enable the members of the Advisory Committee to become acquainted with the teachers personally, and to learn about their professional needs and concerns. SRP culminates with a three-day End-of-Summer Symposium at which time the program’s Advisory Committee is able to assess each teacher's grasp of the principles on which his/her work is based.

Program SummaryProgram Summary

• Stipend - $6,000 each summer

• Classroom Enhancement Funds - $1,000 following each of the two summers

• Research Group Funds – $1,000 in each of the two summers

• Travel to Professional Conference – Science education and/or scientific society conferences in each of the school years following participation in the program

• International Program – Following completion of program, 2 teachers are selected to participate in an exchange of science teachers with Singapore’s Ministry of Education and 2 teachers are selected to participate in a 3rd summer (winter?) of research in Australia

Program ‘Perks’

School Year Classroom Support

Teachers are afforded the opportunity to have the graduate student with whom they worked alongside in the lab, visit their classrooms one day a month.

Evaluation

A unique aspect of Columbia University’s Summer Research Program is its emphasis on evaluation. Evaluation has informed and guided the Program’s evolution from its inception, and enabled it to respond to the changing needs of teachers

Student Outcomes Studies1. Columbia’s Summer Research Program

(1993-present)

– Collected data from the NYC public high schools, the program’s largest cohort, with the assistance of the school science department chairs and the NYC Dept. of Ed.’s Division of Assessment & Accountability

2. NSF Multi-site Study (1998-2002)– 8 science teacher research programs – Arkansas, California, Georgia, Idaho, New York,

Oregon, Texas, & Washington State

Instruments AdministeredNSF Multi-site Study

1999-2000• Pre-program survey*

• Post-program survey*

• Mentor survey*

• Student Attitudinal Survey*

• Student Cognitive Tests– Biology & Chemistry

*Surveys available at www.SweptStudy.org(Revised versions of program and mentor surveys available at

www.RETNetwork.org)

Comparison of Study and Control Science Teachers (1)

(1) Data for 32 Study teachers and 32 Control teachers

StudyTeachers

ComparisonTeachers

Undergraduate or graduate major or minor in biology, chemistry physics, earth science or math

94% 87%

Advanced degree 63% 62.5%

Hours course credit beyond highest degree

34 hours 54 hours

Years of teaching experience 7.5 13.2

Years of science teaching experience

6.5 11.8

Hours/week spent outside of class preparing for teaching

9.7 hours 9.25 hours

Assigned textbook covered in science course

60% 59%

Used textbook publisher’s tests rarely or never

68% 68%

Used textbook publisher’s tests sometimes

18% 26%

Used textbook publisher’s tests frequently

14% 6%

Self-reported Changes in Attitudes and Classroom Practices

Scale

Study teachers Comparison Teachers

Pre-teaching survey

Post-teaching survey

ChangePre-

teaching survey

Post-teaching survey

Change

Traditional Goals & Objectives 247 256 7 254 254 0

Inquiry Goals & Objectives 244 250 6 258 232 -26*

Traditional Teaching Methods 248 250 2 260 249 -11*

Traditional Student Activities 245 251 6 258 256 -2

Inquiry Student Activities 247 257 10 249 235 -14*

Teacher Efficacy 240 255 15 247 252 5

Number of teachers = 58 58

* Difference in two change scores is significant at the p<0.05 level

• The items that constitute each scale were identified through a confirmatory factor analysis and the scale scores are standardized factor scores with a mean of 250 and a standard error of 50. Information about item wording and factor loadings can be found at www.sweptstudy.org.• These comparisons are based on 58 teachers and comparison teachers who completed all of the items comprising all of the scales. Five teachers failed to complete all of the items in one or more of the scales.

Student Assessments of Classroom Practices

Study Comparison

Number ofStudents

MeanNumber ofStudents

Mean

Reflected on course material by writing in a notebook.2 2,817 2.55 1,469 2.39*

Used primary sources such as journals.12,817 1.93 1,469 1.8*

Listened to guest speakers or went on field trips.12,817 1.66 1,469 1.54*

Explored career opportunities in math or technology.1 2,817 1.86 1,469 1.69*

Teacher has encouraged me to think about math/ science career.2 2,849 2.84 1,481 2.64*

• Difference between Study and Comparison teachers’ students was statistically significant at the p<0.05 level

1 Measured on a 5 point scale where 1 = never, 2 = 1-2 times a month, 3 = 1-2 times a week, 4 = almost every class, 5 = every class in response to the question “Approximately how often did you engage in the following learning activities in the mathematics/science class that you are currently taking?”

2 Measured on a 5 point scale from strongly disagree (1) to strongly agree (5)

Changes in Student Achievement in Science

Subject

Study Teachers’ Students Comparison Teachers’ Students

Pre-coursePost-

courseChange Pre-course

Post-course

Change

Second year

Biology 267 292 25* 263 284 21*

Chemistry 282 310 28* 275 295 20*

* Difference in two change scores is significant at the .05 level

The test scores were derived through IRT methods and converted into a scale score with a mean of 250 and a standard error of 50.

The student achievement scores are derived from tests constructed by Westat, and the Westat tests were built on sets of items that were used previously in the National Assessment of Educational Progress (NAEP), the Third International Mathematics and Science Study (TIMSS), and the Longitudinal Study of American Youth (LSAY).

2008 Multi-Program Surveys(5 programs participated)

www.RETNetwork.org

Summer Research Program1993 – 2008

• Pre-program survey• Post-program survey• Mentor survey• Spring implementation survey• Student participation in Science

Clubs and Intel-type competitions• Scores on NYS standardized science

exams (Regents)

2008 Teacher Survey Data• 100% reported developing new or revised content to lessons and/or labs*

*341 standards-based lesson plans are on the program’s website: www.ScienceTeacherProgram.org

• 91% reported increasing hands-on activities in their classrooms and/or new laboratory exercises in response to their experiences at Columbia

• 78% reported including lessons on science careers and related job requirements with their students

• 74% reported introducing new technologies in their classroom instruction (e.g.; chromatography, pipetting, PowerPoint)

• 61% reported reading scientific journals more frequently

• 52% reported increased requirements for formal written reports and/or oral presentation requirements

• 48% reported assuming new leadership roles/responsibilities in their school/district/region

• Columbia’s Summer Research Program collects quantitative student data from NYC public high schools

- Study Group -- Students in science classes of program teachers

- Control Group -- Students in science classes of non-participating teachers from the same schools

Intel Science Talent Competition

0.44%

0.72%

0.59%

0.40%

0.00%

0.20%

0.40%

0.60%

0.80%

Percen

t

Prior After 2nd Year

Year in Program

18 Complete Data Sets

Teachers

Control Group

(20 datasets)

NYC Economic Impact

• Regents Diploma – Must pass 5 Regents exams1 (includes 1 Science Exam)

• NYC spends $12,930 per public school student2

• Student takes 5 courses/year = $2,586 per course• 20 more students passing Regents over 2 years• 20 students X $2,586 = $51,720 each teacher/year

1 New York State Education Department, New York State Total Public Report Card, 20052 The New York Sun, New York Outpacing Other States in School Spending, April 4, 2006

National Economic Impacts

• High School dropouts earn $392,000 less in lifetime earnings than a high school graduate thereby paying less taxes1. At a tax rate of 20%, loss of $78,400 in local and federal revenues.

• High School graduates are less likely to commit crimes. Increasing HS completion of males by just 1% would reduce criminal justice system costs by $1.4 billion per year

• High School graduates receive higher salaries thereby not needing food stamps, housing assistance, etc. Estimated annual savings are $8 billion per year

1 Fogg, N., Harrington, P., & Khatiwada, I., The Longterm Labor Market Consequences of Dropping out of High School in Pennsylvania, Center for Market Labor Studies, Northeastern University, October 2007

Publishing Our Findings• “Scientific Work Experience Programs for Teachers: A multi-

site evaluation of impact on teachers and on student achievement”– Status: Very final draft form. Expected to be submitted in the next few

weeks for peer review: Educational Evaluation and Policy Analysis

• “High School Science Teacher Participation in Columbia University’s Summer Research Program Improves Student Interest and Achievement in Science”– Status: Readying for final draft. To be submitted for publication in the

next 2 months. Publication to be determined.

Current Funding Sources

• Ambrose Monell Foundation• Bay and Paul Foundations• Hebrew Technical Institute• Mellam Family Foundation• National Institutes of Health – SEPA Program• National Science Foundation• New York Times Company Foundation

Columbia University’sSummer Research Program for

Science Teachers

www.ScienceTeacherProgram.org