89
ASSESSMENT OF TEACHER EDUCATION PROGRAMS IN HIGHER EDUCATION: ALUMNI/EMPLOYER SURVEY RESULTS, K-12 ACHIEVEMENT RESEARCH FINDINGS, AND WHAT REMAINS TO BE INVESTIGATED Dr. Mike Alfano, Dr. Michael Faggella-Luby, Dr. Rachael Gabriel, Dr. Marijke Kehrhahn, Dr. Mary Yakimowski University of Connecticut Transition to the Connecticut State Standards and System of Assessments Third Annual Connecticut Assessment Crown Plaza, Cromwell Rocky Hill, CT August, 2012 PowerPoint available at: HTTP://WWW.EDUCATION.UCONN.EDU/ASSESSMENT/

Dr. Mike Alfano, Dr. Michael Faggella-Luby , Dr . Rachael Gabriel, Dr. Marijke Kehrhahn,

  • Upload
    hailey

  • View
    29

  • Download
    1

Embed Size (px)

DESCRIPTION

Assessment of Teacher Education Programs in Higher Education: Alumni/Employer Survey Results, K-12 Achievement Research Findings, and What Remains to be Investigated. Dr. Mike Alfano, Dr. Michael Faggella-Luby , Dr . Rachael Gabriel, Dr. Marijke Kehrhahn, Dr . Mary Yakimowski - PowerPoint PPT Presentation

Citation preview

Page 1: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

ASSESSMENT OF TEACHER EDUCATION PROGRAMS IN HIGHER EDUCATION:

ALUMNI/EMPLOYER SURVEY RESULTS, K-12 ACHIEVEMENT RESEARCH FINDINGS, AND WHAT REMAINS TO BE INVESTIGATED

D r . M i k e A l f a n o , D r . M i c h a e l F a g g e l l a - L u b y ,

D r . R a c h a e l G a b r i e l , D r . M a r i j k e K e h r h a h n ,

D r . M a r y Y a k i m o w s k i

U n i v e r s i t y o f C o n n e c ti c u t

Transition to the Connecticut State Standards and System of Assessments

Third Annual Connecticut AssessmentCrown Plaza, Cromwell

Rocky Hill, CTAugust, 2012

PowerPoint available at: HTTP://WWW.EDUCATION.UCONN.EDU/ASSESSMENT/

Page 2: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

ASSESSMENT OF TEACHER

EDUCATION PROGRAMS IN

HIGHER EDUCATION:

ALUMNI/E

MPLOYER SURVEY

RESULTS, K-12 ACHIEVEMENT

RESEARCH FINDINGS, AND W

HAT

REMAINS TO BE INVESTIGATED

This panel from UConn’s Neag School of Education will discuss major findings from research on their teacher preparation programs. This session will include a description of the Neag School of Education’s Assessment Plan and an overview of studies that have been completed recently. Specifically, panelists will describe major findings from our alumni and employee surveys, staffing research, and K-12 studies (including recent results from studies of student achievement in math and reading). Then, with audience participation, we will open a discussion of possible directions for future research to meet CT’s K-12 district needs.

Page 3: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

Introducing …

Mary E. YakimowskiNeag School of EducationDirector of Assessment

The Neag Assessment PlanThe Alumni SurveysThe Employer Surveys

Page 4: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

Introducing …

Michael AlfanoFormally, UConn Neag School of EducationExecutive Director of Teacher EducationCurrently, Southern CT State UniversityProfessor & Chair, Dept. of Sp Ed & Reading

The Placement of Alumni

Page 5: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

Introducing …

Michael Faggella-LubyNeag School of EducationAssociate Professor, Special Education

The Evidence-based Survey Studies

Page 6: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

Introducing …

Dr. Rachael Gabriel

Neag School of EducationAssistant Professor, Reading/Language Arts

The Pupil Performance Studies

Page 7: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

Introducing …

Dr. Marijke Kehrhahn

Neag School of EducationAssociate Dean

Where This Leads Us &Generating Ideas from You

Page 8: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

Mary E. YakimowskiNeag School of EducationDirector of Assessment

The Neag Assessment PlanThe Alumni SurveysThe Employer Surveys

Page 9: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

NEAG SCHOOL OF EDUCATION

Page 10: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

PURPOSE OF ASSESSMENT PLAN

Assessment/Evidence-based culture leading to continuous

improvement Accreditation

Page 11: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

CYCLE OF CONTINUOUS IMPROVEMENT

Page 12: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

NEAG SCHOOL OF EDUCATION ASSESSMENT

PLAN

Page 13: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

ASSESSMENT PLAN

Page 14: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,
Page 15: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

NEAG ASSESSMENT PLAN HIGHLIGHTSIncorporated many best practices including:

Focus on facilitating an assessment culture. A system of participatory participation in assessment

development and reporting. Formative and summative assessments at both the candidate

and program level to embrace ongoing feedback. Efforts made to ensure that assessments are credible, fair,

consistent, accurate, and unbiased, allowing for multiangulation.

Page 16: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

Information available from external sources such as state licensing exams, evaluation through clinic experiences, employer reports, and alumni studies.

Alignment of all accreditation processes from the university to program levels.

A concerted effort to provide a spotlight on assessment.

A system for reviewing and approving the assessment plan.

Assessment-related research opportunities.

(CONTINUED)

Page 17: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

ALUMNI AND EMPLOYER SURVEYS

Every 2 years for select programs, every 4 years by school

Page 18: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

PURPOSE

Collect information from stakeholders (10 years of alumni) for: Continuous improvement of the Neag School Dissemination to school, depts, unit, programs

Commitment to “high quality programs of study and to conduct meaningful research that speaks to the critical issues in education, technology, sports, and health and wellness” (http://www.education.uconn.edu)

4,244 total alumni identified

3,818 of those had valid addresses

758 responded17% of total identified alumni20% response rate from contacted alumni

Page 19: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

29 REPORTS

School-wide, Each Department, and Teacher Education Report

School-wide ReportDepartment of Curriculum & Instruction

Bilingual EducationElementary EducationEnglish/Language ArtsMathematicsMusicScienceSocial StudiesWorld Language

Department of Educational LeadershipEd Leadership Education AdminEd Leadership Executive LeadershipEd Leadership UCAPP

Department of Educational PsychCognition and InstructionCounseling EducationGifted & TalentedSchool PsychologySpecial Education

Department of KinesiologyAthletic TrainingExercise SciencePhysical TherapySports Management

Teacher Education UnitIB/MTCPCG

Page 20: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

SATISFACTION AND IMPORTANCE

Six-point Likert-type scale

Scale Satisfaction Importance1 Very unsatisfied Very unimportant2 Moderately unsatisfied Moderately unimportant3 Somewhat unsatisfied Somewhat unimportant4 Somewhat satisfied Somewhat important5 Moderately satisfied Moderately important6 Very satisfied EssentialN/A No opinion No opinion

Page 21: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

TEACHER EDUCATION: SATISFACTION WITH EDUCATIONAL QUALITYEducational QualityTwo highest itemsaccessibility of facultyoverall quality of instruction

Three lowest itemsrange of coursescourse contentchallenged to meet academic potential

Sample Items Rated• Quality of advising• Up-to-date equipment• Course content• Quality of teaching• Accessibility of faculty• Program challenged you to meet your

fullest academic potential• Faculty experience as practitioners• Sense of community with other students• Clinic/Practicum/Internship experiences• Job readiness

Page 22: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

OVERALL PREPARATION

Mean: Satisfaction Importance Gap

Combine multiple approaches to solve problems 5.20 5.47 -0.26Collaborate effectively with others 5.56 5.69 -0.13Adapt to changes in your working environment 5.32 5.61 -0.29Think analytically and logically 5.40 5.62 -0.22Learn on your own, pursue ideas and find necessary information 5.52 5.67 -0.15Lead and/or supervise groups of people 5.19 5.44 -0.25Formulate creative and original ideas 5.32 5.59 -0.27Effectively use technology 4.86 5.49 -0.63Consider the perspectives of others 5.43 5.56 -0.13Conduct inquiry and/or research 5.26 5.09 +0.17Understand research in professional journals 5.05 4.92 +0.13

Page 23: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

Mean: Satisf. Import. Gap

Creating meaningful learning experiences for students. 4.53 4.85 -0.32The content and/or area specialty 4.44 4.76 -0.32Working effectively with parents. 4.42 4.60 -0.18The degree of preparation for working in the profession. 4.19 4.84 -0.65Challenging students to meet their fullest potential. 4.15 4.80 -0.65The difficulty level of the program. 4.12 4.52 -0.40Standardized assessment skills. 4.05 4.16 -0.11Educating students from diverse cultural backgrounds. 4.04 4.55 -0.51Teaching English language learners. 3.84 4.35 -0.51Integrating technology into classroom instruction. 3.81 4.46 -0.65Classroom management skills. 3.79 4.85 -1.07Teaching students who are both in spec ed& ELL. 3.58 4.32 -0.74Educating students from diverse socioeconomic backgrounds. 3.55 4.57 -1.02Formative classroom assessment skills. 3.45 4.58 -1.13Teaching special education students 3.27 4.64 -1.37Teaching gifted and talented students. 3.04 4.33 -1.29

Preparation for the Teaching Profession

Page 24: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

Top 3 themes:Clinical

My student teaching experience changed my life and affected my teaching more than I ever thought possible. I was able to implement many of the teaching strategies that I learned at NEAG during this experience. Having spent the first half of the year visiting the classroom, and the second half student teaching in that same classroom was incredibly beneficial.

Courses The individual methods courses offered for each content area during the TCPCG program has been the most relevant and useful of all courses. Furthermore, the courses on Multicultural education and Students with special needs continue to be important in my career and I often refer to materials for information.

Faculty I think the most valuable experiences I had in the Neag School were the connections I made

with my professors. I always felt well-supported and mentored by the professors I had, and I still e-mail with several of them for advice and help. These professors are not only experts in their fields, but valuable resources and friends to all students in the Neag School.

WHAT WOULD YOU CONSIDER THE MOST VALUABLE EXPERIENCE OFFERED BY THE NSOE?

Page 25: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

Top 3 themes: Courses

The course about special education was not effective in preparing me for the teaching field. While this is an important topic to be aware of, the material was not presented in a manner that I was able to retain information, and I did not feel prepared to handle situations involving special education upon entering the teaching field.

Clinical The clinicals that were outside of your concentration-- for me, high school social

studies did not help me much with elementary school. Instead give us time in primary vs. intermediate elementary.

Technology During my time there, the technology component was least valuable. We basically

just demonstrated proficiency with Microsoft Office programs.

WHAT DID YOU FIND LEAST VALUABLE?

Page 26: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

RECOMMEND NSOE?

95.9% of respondents would recommend graduate study at the

Neag School of Education to others

Page 27: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

SUMMARY

Positive responses to survey regarding alumni perceptions of the Teacher Education programs

Most are employed in field for which they received training and satisfied with their employment

Parts of Teacher Education to improve were most often identified as courses and clinical experiences

Page 28: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

Michael AlfanoFormally, UConn Neag School of EducationExecutive Director of Teacher EducationCurrently, Southern CT State UniversityProfessor & Chair, Dept. of Sp Ed & Reading

The Placement of Alumni(via district and CSDE records)

Page 29: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

HOW MANY ALUMNI ARE EMPLOYED BY CONNECTICUT PUBLIC SCHOOLS?

ANSWER: 3 ,090 165/166

DISTRICTS

Page 30: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

Distribution of Our Alumni Across Connecticut School Districts

Page 31: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

Alumni Teachers Employed by Connecticut School Districts (map does not include related services and administrators)

Page 32: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

Alumni Elementary Teachers Employed in by Connecticut School Districts

Green= alumni

Page 33: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

Alumni Secondary Teachers Employed by Connecticut School Districts

Orange= alumni

Page 34: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

Alumni Special Education Teachers Employed by Connecticut School Districts

Pink = alumni

Page 35: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

Where are our alumni employed as related service personnel

Page 36: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

Alumni Employed as School Psychologists in by Connecticut School Districts

Red = alumni

Page 37: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

Alumni Employed as School Counselors in 2009-2010 by Connecticut School Districts

Yellow = alumni

Page 38: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

Top Employers of Alumni as Related Service Personnel

Employer nWest Hartford School District 62South Windsor School District 45Trumbull School District 21Newtown School District 20Madison School District 12Monroe School District 12New Fairfield School District 7Capitol Region Education Council 6

Page 39: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

Top Employers of Alumni as School Psychologists

EmployerEast Hartford School DistrictWest Hartford School DistrictGlastonbury School DistrictEnfield School DistrictWindham School District

Page 40: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

Top Employers of Alumni as School Counselors

EmployerManchester School DistrictNew London School District

Page 41: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

Where are our alumni employed as administrators?

Page 42: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

Alumni Employed as Elementary Administrators by Connecticut School Districts

Areas shaded represent alumni

Principals

Asst. Principals

Page 43: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

Alumni Employed as Middle School Administrators by Connecticut School Districts

Areas shaded represent alumni.

Asst. PrincipalsPrincipals

Page 44: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

Alumni Employed as High School Administrators by Connecticut School Districts

Principals

Asst. Principals

Areas shaded represent alumni.

Page 45: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

Alumni Employed as Central Office Personnelby Connecticut School Districts

Pink = alumni

Page 46: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

WHO EMPLOYED THE MOST OF OUR ALUMNI?

Page 47: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

TOP EMPLOYERS OF ALUMNI

DRG Employer TotalI Hartford School District 149H East Hartford School District 99B Glastonbury School District 89F Manchester School District 83B West Hartford School District 82I New Britain School District 61F Vernon School District 59B South Windsor School District 57I Windham School District 57D Tolland School District 53I Waterbury School District 52D Windsor School District 49Other Connecticut Technical High School System 49H Bristol School District 45Other Capitol Region Education Council 45F Enfield School District 39I New Haven School District 39A Simsbury School District 37H Stamford School District 37B Farmington School District 36C Mansfield School District 35D Newington School District 33C Ellington School District 32B Greenwich School District 32H Meriden School District 32F Groton School District 31

Page 48: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

Top Employers of Alumni Secondary English/LA Teachers

Employer nGlastonbury School District 13Vernon School District 11East Hartford School District 10Windham School District 10Connecticut Technical High School System 10

Page 49: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

TOP EMPLOYERS OF ALUMNI SPECIAL EDUCATION TEACHERS

Employer nManchester School District 12Glastonbury School District 12East Hartford School District 12West Hartford School District 9

Page 50: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

Michael Faggella-Luby Neag School of EducationAssociate Professor, Special Education

The Evidence-based Survey Studies

Page 51: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

Evidence-based Examination of Classrooms: Do Pre- or In-service Teachers and Your Field Make a Difference?

Page 52: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

PURPOSE & RESEARCH QUESTION

Purpose: to examine whether variations in response to an evidence-based instrument can be attributed to:

• group membership (pre- on in-service teacher), • field (elementary, secondary, special; education), and/or • the interaction between group membership and field.

Research Question: Is there a significant interaction between group (pre- and in-

service teachers) and field (elementary, secondary, special education) with respect to the overall score and factor scores on an instrument designed to measure confidence of evidence-based practice use?

Page 53: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

SRBI/RTI FRAMEWORK

Tier 1:Comprehensive &

Coordinated Instruction

for All Students

Tier 2:SupplementalInstruction for

Students at Some Risk

Tier 3:Specialized,

IndividualizedIntervention for

Students at High Risk

80% of Students

15%

5%

Page 54: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

5 DOMAINS OF SPECIAL EDUCATION TEACHER PREPARATION

1. Planning and Preparation2. Evidence-based Classroom and Behavior

Management3. Evidence/standards-based Instruction4. Evaluation5. Professional Behaviors and Responsibilities

Page 55: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

THE DOMAINS

Domain Performance IndicatorDomain 1: Planning and preparation Student teachers will…

1A. physically prepare space and materials needed to deliver instruction 1B. design lesson plans to provide all learners access to the general curriculum 1C. modify lesson plans to address needs of students with disabilities

Domain 2: Evidence-based Classroom and Behavior Management

Student teachers will…2A. maintain a structured learning environment 2B. use a small number of positively stated expectations 2C. reinforce appropriate behavior 2D. respond to inappropriate behavior 2E implement individualized behavior strategies for students with disabilities

Page 56: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

THE DOMAINS (CONTINUED)

Domain Performance IndicatorDomain 3: Evidence-based Instruction

Student teachers will…3A. introduce lesson content 3B. maximize student engagement 3C. provide performance-based feedback 3D. review lesson content at the end of instruction3E. teach lesson content relevant to student population

Domain 4: Evaluation Student teachers will…4A. assess student ability and/or knowledge prior to instruction4B. assess student outcomes related to IEP during instruction4C. assess student response to instruction

Domain 5: Professional Standards and Responsibilities

Student teachers will…5A. uphold high standards of competence and integrity and exercise sound judgment in the practice of the profession 5B. engage in professional activities related to continuous learning and advocacy 5C. respectfully with all stakeholders

Page 57: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

METHODSubjects: n=484

• 282 Pre-service IBM & 202 In-service TCPCG

Procedures:

• Online survey

• NSOE Current and Alumni students invited participation

Measure: The Student Teaching Evaluation and/or Self-Assessment Survey (STE-SAS)

• 21-item instrument

• Six-point Likert rating scale on STE-SAS, with “1” indicating Not At All Confident to “6” indicating Very Confident

• Intended to provide common language for professional conversations with the university faculty about evidence-based teaching, learning, and assessment

• Examination of the technical properties of the STE-SAS including reliability and evidence-based four-factor structure

Page 58: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

Pre-Service In-Service Total N % N % N %

Elementary 79 28.0 89 44.1 168 34.7

Secondary 160 56.7 88 43.6 248 51.2

Sp Ed 43 15.2 25 12.4 68 14.0

Total 282 100 202 100 484 100

Descriptive Statistics Overall, Pre- and In-service Groups, and by Field

Page 59: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

Factor 1 : Planning and Assessment (α = .886) 1 Physically prepare space and materials needed to deliver instruction 2 Design lesson plans to provide all learners access to the general curriculum 9 Introduce lesson content 10 Maximize student engagement 11 Provide performance-based feedback 12 Review lesson content at the end of instruction 13 Teach lesson content relevant to student population 14 Assess student ability and/or knowledge prior to instruction 16 Assess student response to instruction

STE-SAS FOUR FACTORS

Page 60: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

Factor 2 : Professional Standards & Responsibilities (α = .879)17 Uphold high standards of competence in the practice of the profession18 Uphold high standards of integrity in the practice of the profession

19 Use evidence to guide exercise/exercising sound judgment in the practice of the profession

20 Engage in professional activities related to continuous learning and advocacy21 Collaborate respectfully with all stakeholders

Factor 3 : Instructional Delivery (α = 0.845)3 Modify lesson plans to address needs of students with disabilities8 Implement individualized behavior strategies for students with disabilities15 Assess student outcomes related to IEP during instruction

Page 61: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

Factor 4: Behavior Management (α = 0.769) 4 Maintain a structured learning environment5 Use a small number of positively stated expectations6 Reinforce appropriate behavior7 Respond to inappropriate behavior

Page 62: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

RESULTS

Mean total STE-SAS was 5.39 (SD = 0.56) indicating overall confidenceRespondents highest in:

• professional standard and responsibilities (M = 5.65, SD = 0.56), followed by

• maintaining classroom control (M = 5.47, SD = 0.62), • general teaching/assessment tasks (M = 5.42, SD = 0.59), and

Least confident in individualizing their teaching (large SD)• instructional flexibility/individualization (M = 4.90, SD = 0.96).

Page 63: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

Ele Sec Sped Total M SD M SD M SD M SD

General teaching/assessment tasks

5.40 0.53 5.38 0.63 5.62 0.52 5.42 0.59

Professional standards and responsibilities

5.62 0.52 5.63 0.62 5.78 0.38 5.65 0.56

Instructional flexibility/individualization

4.79 0.88 4.73 1.00 5.77 0.41 4.90 0.96

Maintaining classroom control

5.58 0.47 5.31 0.71 5.74 0.38 5.47 0.62

STE-SAS Descriptive Statistics by Field

Page 64: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

RESULTS

ANOVA to test significant interaction between group and field• significant main effect for field [F (2, 450) = 13.791, p =.000] • neither significant effect for type of service, nor interaction

effect between field and type. • Scheffee’s analysis yielded special education exhibited

significantly higher global scores than other respondent groups

Secondary Analysis also demonstrated: • Special education rated general teaching/assessment tasks

and instructional flexibility/individualization higher than both elementary and secondary respondents

• Secondary education reported significantly less confidence than either elementary or special education respondents

Page 65: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

Source SS df MS F Sig.Corrected Model 9.694a 5 1.939 6.445 .000Intercept 9797.352 1 9797.352 32569.417 .000Type .063 1 .063 .211 .646Field 8.297 2 4.148 13.791 .000Type * Field .958 2 .479 1.593 .204Error 135.367 450 .301 Total 13400.785 456 Corrected Total 145.061 455

ANOVA Results for STE-SAS Total Score for Group Membership (Pre- vs. In-Service) and Field (Elem, Secondary, Special Education)

Page 66: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

SIGNIFICANCE AND FUTURE RESEARCH1. Respondents in special education feel significantly more confident than respondents in both

elementary and secondary education, irrespective of type of service, in their knowledge of and confidence for these factors

• Could this be the result of the redesign of the Special Education Program? • Perhaps special education teachers practice in each of the four factors relates to higher levels

of confidence?

2. Given the significant main effect for type and the positive correlation results, it could be implied that there may exist more complicated relationship in specific areas such as instructional flexibility/individualization or general teaching/assessment tasks. More research is necessary

3. Findings are significant as teacher education programs reevaluate curricula toward evidence-based models of service delivery such as RtI.

• How does the teacher education program use this data to drive course revision? • How might qualitative examination of students in individual programs yield deeper

understanding? • How are practitioners implementing evidence-based practices in relationship to confidence?

4. The findings raise important questions about the changing role of the special educator in K-12 schools, signaling a potential change in how schools leverage interventionists to support multiple tiers of school-wide support.

Page 67: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

Dr. Rachael Gabriel

Neag School of EducationAssistant Professor, Reading/Language Arts

The Pupil Performance Studies

Page 68: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

OUR TEACHER EDUCATION PROGRAM

Integrated Bachelors/Masters (IB/M) Program Entering students in the Junior year

Exiting with a Masters Special feature - Students participate in

1,200 hours in Clinics, Student Teaching, and Internship

Teacher Certification Program for College Graduates (TCPCG) Program

Masters level studentsTraining for Secondary EducationSpecial feature - Shortages areas

(Mathematics, Science, Special Education) is a focus

Page 69: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

A Great Education Begins with Great Teachers

Teachers for a New Era (TNE) - An initiative designed to improve teacher quality by

reforming outstanding teacher preparation programs

Page 70: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

DESIGN PRINCIPLES 1. Emphasize to preservice teachers the

importance of demonstrating student achievement through evidence.

2. Fully integrate faculty from the liberal arts and sciences, enriching future teachers' general and subject matter knowledge.

3. Support will be extended to beginning teachers from their individual colleges and universities.

Page 71: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

Examining K-12 performance to inform teacher preparation

These studies examine achievement patterns of 3rd-8th grade pupils of graduates of our Teaching Education Program in Reading and in Mathematics

Scott Brown
same comment as previous one
Page 72: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

LITERATURE REVIEW

There are significant interests in examining growth achievement models (e.g., Barone, 2009)

High-quality teacher education programs take on an important role (Bransford, Darling-Hammond & LePage, 2005; Darling-Hammond, 2006)

Lack of empirical evidence connecting teacher education programs with student outcomes (Crowe, 2010; Grossman, 2008)

Page 73: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

PURPOSE OF THESE STUDIES

Examine the impact of teacher education programs on pupil performance in content areas (i.e., reading, mathematics)

Compare a program of interest (that is UConn Neag School of Education) with other programs to investigate the impact of unique characteristics

Page 74: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

SAMPLING 5 public school districts in Connecticut

Approximately 12,00 students from grades 3 through 8

INSTRUMENTATION The fourth generation of Connecticut Mastery

Test (CMT-4)

Grades 3 through 8 in the spring at each year

Page 75: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

DATA ANALYZED

• Total (Raw) Score• Domain Scores• Strand Scores • Proficiency Level Scores• Vertical Scale Scores

Page 76: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

Research Questions Scores Type OtherIs overall achievement for teachers prepared by UI alumni any different from pupils of alumni from other institutions?

Overall raw score

Descriptive, t-test

1 year

Is performance in the domains different for pupils taught by teachers from the UI any different than the performance of pupils taught by alumni from other institutions? (Math only, not Reading)

Domain scores

Descriptive, t-test

1 year

Is performance on the strands any different for pupils taught by teachers from the UR any different than the performance of pupils taught by alumni from other institutions?

Strands Descriptive, t-test

1 year

Is the pattern in pupil proficiency status for those educated by UI alumni any different from pupils of alumni from other institutions?

Prof level

Proportion analysis

1 year

Is overall achievement for those taught by UI alumni different after controlling for initial differences in earlier achievement?

Vertical scale

ANCOVA 2 years

Page 77: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

Overall Score in MathematicsUConn Alumni Pupil Performance

Overall Mean was 106 (SD = 22.8)Those not taught from UConn

Overall Mean was 95.3 (SD = 26.8)

Page 78: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

DOMAIN 1 – NUMERICAL / PROPORTIONAL

Similar results attained across each domain in mathematics

We also looked at strands within domain

UCONN (UI-University of Interest) 53.3

Non-UCONN (Non-UI) 46.4

Page 79: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

Domain 1 - Strand Score Results

Page 80: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

PROPORTIONAL ANALYSIS SHOWING PROFICIENCY LEVEL IN MATHEMATICS

Non-UI UI

Level % %Below Basic 9.1 4.2Basic 10.7 4.4Proficient 20.4 15.4Goal 32.6 36.2Advanced 27.2 39.8

Page 81: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

Groups n Unadjusted 2007-2008

Unadjusted 2008-2009

Adjusted2008-2009

Non-UI 9072 513.0 541.3 542.6

UI 816 534.2 564.2 549.1

F pBetween-Subjects Effects Intercept 3914.3 0.001*** MAVS2007-2008 32283.6 0.001*** Teacher Group 36.6 0.001***

MATHEMATICS PUPIL PERFORMANCE AS MEASURED BY VERTICAL SCALE SCORES

Page 82: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

UI Non-UI Overall

N M SD N M SD N M SD

Overall Score 964 32.4 5.1 10644 29.8 6.7 11608 30.0 6.7

Strand 1 839 8.7 1.8 7453 7.9 2.4 8292 8.0 2.4

Strand 2 839 8.2 1.7 7453 7.3 2.1 8292 7.4 2.0

Strand 3 839 6.2 1.4 7453 5.4 1.8 8292 5.5 1.8

Strand 4 839 9.4 2.0 7453 8.0 2.4 8292 8.2 2.4

Overall and Strands Scores in Reading

Strand 1: Forming a General Understanding, 2: Developing Interpretation, 3: Making Reader/Text Connections, 4: Examining the Content and Structure

Page 83: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

strand1_UI strand1_NON strand2_UI strand2_NON strand3_UI strand3_NON strand4_UI strand4_NON0.00

2.00

4.00

6.00

8.00

10.00

12.00

14.00

Strand by Teacher Group

Stra

nd S

core

An example of CMT-4 reading strands for UI and non-UI groups

Page 84: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

Level UI Non-UI Overall

n % n % n %Below basic 44 4.6 1508 14.2 1552 13.4

Basic 37 3.8 790 7.4 827 7.1

Proficient 85 8.8 1199 11.3 1284 11.1

Goal 447 46.4 4681 44.0 5128 44.2

Advanced 351 36.4 2451 23.1 2802 24.2

Total 964 8.3 10629 91.7 11593 100

Proportional Analysis Showing Performance Level in Reading

Page 85: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

Groups NUnadjusted 2007-2008

Unadjusted 2008-2009

Adjusted 2008-2009

Overall 11832 473.9 497.0 496.6

Non-UI 10863 471.9 495.2 501.0

UI 969 497.1 517.1 498.8

Reading Pupil Performance as Measured by Vertical Scale Scores

Page 86: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

Source SS df MS F Sig.Corrected Model 21593277 2 10796638.70 9130.48 .001Intercept 5001714 1 5001714.21 4229.84 .001 Covariate (2007-08 Rd) 21166675 1 21166675.99 17900.19 .001 UI vs. Non-UI 16484 1 16484.60 13.94 .001 Error 13987591 11829 1182.48 Total 2958138675 11832 Corrected Total 35580868 11831

ANCOVA Results: Reading Pupil Performance on the Adjusted Vertical Scale Scores Based on Initial Differences R Squared = .607 (Adjusted R Squared = .607)

Page 87: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

Dr. Marijke Kehrhahn

Neag School of EducationAssociate Dean

Where This Leads UsGenerating Ideas from You

Page 88: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

Where we have beenAlumni Survey

Employer SurveyDistrict/CSDE Examination

Evidence-based StudiesPupil Performance Studies

Where we think we have more to do

Help us determine what we should further explore

Page 89: Dr. Mike Alfano, Dr. Michael  Faggella-Luby ,  Dr .  Rachael  Gabriel, Dr. Marijke Kehrhahn,

As Education Secretary Duncan discussed the goal from Obama administration’s Race to the Top legislation at an annual meeting of the American Association of College of Teacher Education in February of 2010 in Atlanta, “To put it in the simplest terms, we believe teacher-preparation programs should be focused on results.”

We continue to strive to build an evidence-based

teacher preparation model for our own teacher preparation program directly linked to pupil academic performance.