Florida Department of Education Bureau of Educator Recruitment and Professional Development

Institutional Program Evaluation Plan 2002

Checklist Submission Date: October 2003

Institution: University of South Florida

Part I. Evaluation Section (Plan Update):

Indicate in the table below whether the institution has made any substantive changes to the Evaluation Section of the 2001 IPEP. If changes are substantive, provide a brief summary in the table or in an attached narrative. Supporting documentation is not necessary for IPEP submission, but it must be kept on file for review by the Site Visit Team during the five-year review.

Standard Changes Made Brief Summary of Changes Made to Evaluation Section if an 1 0 _Yes Development and implementation of a new assessment o No system (e.g., in Childhood Ed, plans for electronic portfolios) 2 o Yes o No 3.1 o Yes Changed admission requirements to be in compliance. u No 3.2 0 _Yes Changes made to be in compliance with state o No mandates. 3.3 o Yes u No 3.4 o Yes o No 4 o Yes o No 5 u Yes o No

Part II. Data Section (Annual Report): Section 240.529(4)(e), Florida Statutes, (new Section 1004.04(4)(c), Florida Statutes) requires that each institution conduct an annual review of all state-approved educator preparation programs. Using the Continued Program Approval Standards and the Standard 3 Mandates (Attachment 3) as a guide, indicate with a checkmark on the list below, all elements that have been reviewed for compliance in the 2001-2002 academic year. For any elements not reviewed,

attach an explanation. Again, be sure to keep accurate and detailed records of the institutional annual review for the five-year review by the Site Visit Team. For the on-site review in Year 5, institutions are required to prepare an assessment of compliance report for all the elements listed below, and it is recommended that the appropriate personnel gather and file the data for this report annually as part of the annual review process. Do not send the data to the DOE with this IPEP unless specified. Standard 1: Demonstration of Knowledge and Skills at the Point of Program Completion Assessments or Data Sources Information Management or Decision-Making System Graduate Decisions and Transcript Endorsement Standard 2: Performance on the Florida Teacher Certification Examination Pass Rates (See attached Chart 1) Deviations Standard 3: Inclusion of Components Mandated by State Statute and Rule (Reminder: Refer to the Continued Approval Standard 3 Mandates when completing checklist)

Indicator 1: Candidates Admissions Requirements Retention and Progression Program Completion: Passing FTCE Program Completion: Demonstrating content standards Program Completion: Demonstrating instructional strategies for all learners, assessing K-12 learning, and using technology

Indicator 2: Curriculum Design Inclusion of Content Standards and Accomplished Practices General Requirements for All Teacher Education Majors Subject Area Requirements for All Teacher Education Majors Professional Education Requirements for All Teacher Education Majors Instruction Related to Sunshine State Standards Instructional Strategies, Assessment of K-12 Learning, Technology, ESOL Values Instruction Higher Level Mathematics and Technology

Indicator 3: Clinical and Field-Based Component Postsecondary Instructors District Personnel Site Selection Content and Focus Length of Final Internship

Indicator 4: Annual Review and Continuous Improvement

2

Institutional Program Evaluation Plan (See explanation) Stakeholder Involvement n Remediation of Prior Weaknesses (See Standard 3, Indicator 2, Curriculum Design, "Instructional Strategies, Assessment, Technology, and ESOL")

Continuous Improvement Employer Satisfaction Survey (See attached sample copy of survey for 2002-2003 year) Data Reporting Requirements

Attached are required data reports including: • Percent of Program Completers obtaining full-time teaching employment

within the first year of graduation-2000-2001 graduates employed in 20012002; • Average length of stay of Program Completers in their full-time teaching

positions; • Summary of results of employer satisfaction survey including percent of

employers rating 2000-2001 Program Completers as satisfactory).

Recency of Faculty Experience

Standard 4: Diversity of Student Population Diversity Characteristics Standard 5: Satisfaction of Employing Districts Satisfaction Eligibility for Rehire (See attached Chart S)

Part IILA. Improvement Section (Report): Academic year 2001-2002 (organized by the Continued Program Approval Standards)

Due to many administrative and personnel changes in the Dean's Office, the USF 2001 IPEP was not submitted to the FLDOE until December 2002 and did not include the section for Standard 5 since we were not able to collect and analyze the necessary "rehire" data from the school districts. The section for Standard 5 was submitted in March 2003. We have not yet received the results of our 2001 IPEP review and cannot address any weaknesses which may have been noted by the reviewers.

Standard 1: All students who complete the program will demonstrate the 12 Accomplished Practices at the Preprofessional Benchmark as described in the institutional evaluation plan. Changes made in the Internship Office database system the previous year have continued to facilitate the documentation of students' demonstration of the Accomplished Practices (AP). In addition, increased training provided to individuals serving as university supervisors has clarified their responsibilities and ensured the gathering of this documentation for accountability and program review purposes.

3

Since last year faculty are required to indicate on their course syllabi assignments that are aligned with the Florida Accomplished Practices and the ESOL competencies. This has afforded students with better communication regarding the connection between their individual courses and their demonstration of the Accomplished Practices. For programs using portfolios, this has been very helpful as the students organize the contents of their portfolios.

Standard 2: 90% of the students in the program will pass the CLAST, the Professional Education, and Subject Matter Subtests of the Florida Teacher-Certification Examination.

Annual Review Process: The institution's review process is adequate to ensure compliance with this standard. Please see below.

Pass Rates: CLAST: In 2001-2002 at least 90% of program completers passed the CLAST.

FTCE: The USF College of Education's graduates do extremely well on the statewide teacher certification exams. Successful completion of the FTCE is a requirement for graduation; therefore, 100% of USF graduates successfully pass the FTCE. The 2001-2002 Title lI report for the USF College of Education indicated that 100% of program completers taking the professional education and subject matter subtests of the FTCE passed the exam. Results will be included in the next printing of the College of Education Profile, a brochure which presents and highlights important facts about our institution.

Accomplished Practices: All students in all pre-service programs, whether undergraduate or master's level, must demonstrate minimal competency in all 12 Accomplished Practices. On a 5point scale, with "5" indicating performance proficiency at the level of a competent beginning teacher, interns must achieve at least a " 3" in all areas to successfully complete internship and graduate. For interns who demonstrate weakness in a particular area, processes are in place for them to have an extended internship experience that provides them with opportunities to become competent. In more serious cases, interns may be required to attempt a second internship, often in a different district, to demonstrate competence in all 12 Accomplished Practices. Standard 3, Indicator 1: State-mandated requirements related to candidate component of the program - including admissions, retention and progression, and program completion - are implemented and monitored, with improvements made as appropriate. Admissions Requirements: Last year's improvement plan indicated that the College of Education anticipated using the 10% admission waiver of the CLAST only for students who provide medical evidence documenting an inability to take the CLAST during the semester they are entering the College and would allow them one semester to take and pass all sections of the CLAST. This proposed plan was reconsidered and, therefore, not implemented since it was determined that it too narrowly defined when a 10% waiver can be granted. Admissions requirements changed in Fall 2001 for Elementary Education, Early Childhood Education and Special Education according to the State mandates.

4

Retention and Progression: Students receiving 10% waivers are informed of remedial services available to them so that they can work toward meeting competencies. CLAST workshops are offered every month in the different subject areas. Program Completion - FTCE: Students in all teacher preparation programs at USF are required to pass the FTCE prior to graduation to increase the quality of teachers just entering the profession and to ensure that the students they will be teaching have knowledgeable and skilled instructors. There is no change in this policy. Program Completion - Content Standards: Please see Improvement Report, Standard 3, Indicator 4, Annual Review and Continuous Improvement, "Graduates' Performance on Accomplished Practices," pp. 10-14. Program Completion - Diversity, K-12 Assessment, and Technology: Program completion requirements for undergraduates and graduates include the required ESOL training as mandated by the State of Florida. Language Arts graduate students must also pass a comprehensive ESOL Education examination in order to receive the ESOL endorsement. Please see Improvement Report, Standard 3, Indicator 4, Annual Review and Continuous Improvement, "Graduates' Performance on Accomplished Practices," pp. 11-14 for more information on assessment and technology. Standard 3, Indicator 2 - Curriculum Design: State-mandated requirements related to the curriculum design component of the program - including both content and delivery - are implemented and monitored, with improvements made as appropriate.

During 2001-2002 major curriculum changes have taken place. New Master's in the Arts of Teaching (MAT) programs were approved and implemented to address the demand for alternative certification programs. The following new MAT programs are attracting considerable enrollment: Elementary Education, English Education, Foreign Language Education, Social Science Education, and Special Education. Several new courses and modules were designed: Classroom Management course, Teaching the Adolescent Learner, and Reading in the Content Areas. Students in the Elementary Education, English Education, Foreign Language Education, and Special Education MAT programs can now graduate with full ESOL endorsement. The technology Portal system has been implemented at the university level. It provides a web-based shell for every course offered, enabling faculty to enhance their course instruction with web-based resources. It has considerably eased communication with students via official university e-mail addresses. "Blackboard" is being used with increasing frequency to enhance traditional course delivery or to offer courses on-line in graduate programs. Content Standards and Accomplished Practices: Childhood Education, incorporating the elementary and early childhood programs, has increased reading instruction hours to 12 credits and reexamined the content and organization of its courses. Faculty in this department have taken the lead in developing an instrument organized under the six NCATE standards and correlated to representative behaviors from the Florida Accomplished Practices that may become a common data gathering instrument across the College in pre-service programs.

5

General Requirements for all teacher education majors: Two graduate initial certification programs, English and Foreign Language Education, have received approval from the Florida Department of Education to grant the ESOL endorsement. All students in initial certification programs complete coursework and other requirements in ESOL.

Subject Area Specialization Requirements for all Teacher Education Majors: All programs are in compliance with subject area specialization requirements as currently defined by the Department of Education. Professional Education Requirements for all Teacher Education Majors: No changes have been made since last year. Instruction Related to Sunshine State Standards: There are no changes from the 2001 IPEP Report. All programs continue to be in compliance.

Values Instruction: The administration, faculty and staff are committed to preparing College of Education (COE) students to excel in teaching children from all cultural and linguistic backgrounds. In Florida hundreds of thousands of K-12 students are in ESOL programs, and the Florida Department of Education has mandated that every graduate of a teacher certification program be prepared to teach ESOL students.

Instructional Strategies, Assessment of K-12 Learning, Technology, and ESOL: In response to feedback on the 1999 arid 2000 IPEPs (specifically, the relationship between and among assessments was noted as inconsistent or vague) - In a continuing effort to improve education, the College of Education at the University of South Florida evaluates the performance of graduates on an annual basis. Currently, a set of recently developed instruments is administered to undergraduate, graduate and advanced graduate students. These surveys employ a series of questions designed to provide graduates with an opportunity to make self-ratings on a broad range of educational experiences. All of the newly designed instruments were developed to complement current data collection activities and to aid in the identification of program and unit strengths and weaknesses. Initial Teacher Preparation Assessment In a continuous effort to refine and adapt the variety of learning opportunities, the College of Education seeks information from graduating seniors regarding their perceptions of various aspects of their teacher preparation programs. An entire section of this newly revised instrument (Initial Teacher Preparation Graduating Seniors' Exit Survey) is dedicated to items that are carefully aligned with the Preprofessional Benchmarks for the Accomplished Practices. Further, during the revision of this instrument, consideration was given to the developing College of Education Conceptual Framework to ensure incorporation of the unique characteristics, knowledge, skills, and scholarly dispositions. A pilot test of this newly revised survey is currently planned for fall 2003. Additionally, both an Initial Teacher Preparation Alumni Survey (administered two years after graduation) and a complementary employer survey, Initial Teacher Preparation Principal Survey, have been completely redesigned and pilot tested. Lastly, revisions are also planned for the current Intern Evaluation Form. At present, this assessment is completed by University Supervisors and Cooperating Teachers supervising preservice teachers during their final internship. It is our intent to refine this tool so that it more adequately represents the accomplishment of state standards (i.e., 12 Accomplished Practices), and is more reflective of the reality of today's classrooms.

This set of revised initial teacher preparation instruments coheres around the 12 Accomplished Practices, graduates' perceptions of preparation and the development of professional qualities and scholarly dispositions.

Graduate Level Assessment As an institution dedicated to excellent teaching, the University's fundamental objective is to evoke in its students a lifelong commitment to learning that enables them to become thoughtful, active, productive and compassionate citizens. Advanced studies at the graduate and professional level enable students to work with faculty at the forefront of their disciplines, to increase their depth of knowledge and personal achievement and to contribute to the advancement of their professions and the larger society. The University has high expectations of its faculty and students as exemplary scholars. The College of Education endeavors to graduate and support superior and visionary educators and human and support service professionals who can perform the professional roles required in changing educational environments. Further, the college seeks to expand professional knowledge through interdisciplinary research related to teaching, learning, and teacher education, thereby providing useful information for the design and evaluation of educational systems and the preparation of personnel for those systems. In order to achieve these goals, it is necessary to systematically collect, review, and use information about educational programs. The Department of Educational Measurement and Research has been at the forefront of an effort to develop a comprehensive assessment system for the College of Education that will monitor and improve student learning and development. In this vein, initial effort has been focused on the development of a series of instruments to glean information about graduate educational experiences. The first step of instrument development involved the conceptualization of the relevant domains. Exploration of extant literature and recently developed instrumentation on important aspects of graduate education revealed a wide range of interest and investigation. An examination of the indicators of successful graduate programs was made in concert with the development of the major domains. Further, careful consideration was given to the developing College of Education Conceptual Framework to ensure incorporation of the unique characteristics, knowledge, skills, and dispositions central to graduate education. The domains of primary interest fell within three broad categories: learning environment, preparation, and graduate experiences. Once these domains were established, survey items were constructed based upon existing validated instruments related to these areas. To date, a Master's Program Exit Survey, an Advanced Graduate Rxit Survey, and an Advanced Graduate Alumni Survey have been pilot tested. This set of graduate level assessments coheres around indicators of successful graduate educational experiences and professional commitments and scholarly dispositions. The TECH Initative: PT3 (Preparing Tomorrow's Teachers to Use Technology) is a federallyfunded grant which sponsors the College's new Teacher Education Change Initiative. This initiative aims to prepare faculty to meet the National Education Technology Standards for Teachers (NETS*T) and to infuse technology into its teacher education curriculum. It is a three-pronged

7

approach ensuring that prospective teachers are well prepared to use technology in 1) professional education, 2) pre-professional education and 3) field experiences. The project has been developing and delivering classroom lessons tied to state standards. It provides faculty with opportunities and resources to integrate technology into teaching and learning and is helping some 360 new teachers to become technology-proficient by the year 2004. Over 100 faculty and teachers attended a seminar on electronic portfolios in December 2002.

The USF PT 3 evaluators have contributed valuable information about faculty skills, proficiencies and practices related to technology through periodic course syllabi review, faculty surveys aligned to ISTE (International Society for Technology Education) standards, and student surveys that include faculty and program feedback, also aligned to ISTE standards.

Course Syllabi Review As part of the 3-year PT 3evaluation, each participating faculty member submits syllabi for review. The review was designed to examine faculty and student use of technology within courses taught by participating faculty members. Data on participants, as well as a group of randomly selected individuals, are being gathered each semester during the life of the project.

Faculty Survey Results The evaluators developed and administered an ISTE-aligned faculty survey designed to gather information about faculty proficiency and use of technology, with particular emphasis on how they use technology within the courses they teach. Results from the surveys are being used to plan future faculty development projects. Student Survey Results Student surveys aligned to the ISTE standards for pre-service teachers were developed and administered in Year Two of the PT3 grant. The survey was modified to include issues of interest to the College of Education and their need to gather data for NCATE. Students were asked about both their skills and practices in each area, and about the extent to which these issues were addressed in the education courses they took. The "Preparation" scales address the program and its faculty, while the others represent aggregate student proficiencies. Preliminary results, using a scale of 1 (low) to 5 (high) in each area of the ISTE standards, include (n=458) mean scores of 3.6 in Technology Operations and Concepts, 3.2 in Preparation for Technology Operations and Concepts, 3.1 in Planning and Designing, 3.6 in Preparing for Planning and Designing, 3.2 for Teaching, Learning and Curriculum, 3.4 for Preparing for Teaching, Learning and Curriculum, 3.1 for Assessment and Evaluation, 3.3 for Preparing for Assessment and Evaluation, 4.0 for Productivity and Professional Practice, and 3.8 for Social, Ethical and Legal issues. A rigorous assessment has been developed to assess graduate students' ability to critique a published research study and identify potential strengths and weaknesses in the problem statement, the research design, the statistical analysis, and the interpretation of data, and offer appropriate suggestions for correction. This assessment is currently administered each semester to students enrolled in EDF 6481, Foundations of Educational Research. Standard 3, Indicator 3 - Clinical and Field-Based Component: State-mandated requirements related to the clinical and field-based component of the program are implemented and monitored, with improvements made as appropriate.

8

Postsecondary Instructors: The procedures involving postsecondary (university) supervisors are working well, and no improvements have been made since last year.

District Personnel: One of the components of the Secondary Education Field Experiences and Supervision Model calls for the selection and preparation of clinical faculty from local school districts. Last year a new graduate level Clinical Education course - Seminar for Clinical Educators -- was developed for Clinical Faculty. It was first offered in Spring 2001, and the second cohort of clinical faculty members completed this required semester-long seminar in December of 2001. They currently help to coordinate early field experiences in their respective schools.

The Physical Education department also has a Clinical Supervision course, and nearly 90% of the teachers used as cooperating teachers have taken this course to date which they report has been very beneficial in helping them mentor pre-service teachers.

The new graduate level Clinical Education course, Mentoring Novice Teachers, was developed the previous year for both Special Education and Elementary personnel and was implemented during 2001-2002. The teachers taking the course have identified a number of benefits: closer connections to the College and the mission of teacher preparation, increased professional growth and responsibilities, and improved skills in mentoring pre-service teachers. The course is co-developed and co-taught by university and district personnel from both Hillsborough and Pasco counties that better prepare them for mentoring pre-service students in their schools. Through this collaboration, new teachers are being provided a seamless system of support and guidance from the time they enter these programs thought their initial years as a teacher.

Many clinical faculty members have served on panels of experienced teachers addressing important issues and answering questions from pre-service teachers in courses such as Classroom Management and Teaching the Adolescent Learner. Frequent collaboration and communication with clinical faculty members have facilitated opportunities for important feedback regarding our current courses and programs, and consequently have provided opportunities for updating and improving our teacher preparation programs.

Site Selection: Progress is being made regarding the collaboration between university and school districts in the placements of interns for final internships. Several meetings have been held with area school districts (Pasco, Hillsborough, Pinellas) in order to assess the progress of the model and to discuss the placement of interns. School district content area supervisors are collaborating with USF program area faculty to make the best placements for interns. Content and Focus: Pre-service teachers have a planned series of clinical experiences early in their program and a final internship that meets the content and focus requirements of the State. All students are expected to do their student teaching (final internship) the semester immediately following the completion of early field experiences. An education student recommended for internship has demonstrated professional, ethical, and reflective behavior and is committed to the achievement and success of all students. The current internship evaluation form is correlated to the Accomplished Practices and is used for all final internship students.

Technology - All field experiences include the opportunity for student teachers to design at least one interdisciplinary unit of instruction using technology to enhance student learning. Overseeing this effort is USF's federally funded PT3 TECH Initiative. (Please see Standard 3, Indicator 2, Curriculum Design, The TECH Initiative, pp. 7-8) Several K-12 schools in Hillsborough and Pasco counties are partners in this process, which is giving student teachers invaluable field experience in technology-infused teaching and learning.

Teaching strategies for LEP students - As mentioned in the section on Curriculum Design, all students in a teacher education program must meet the State mandate for ESOL and are instructed in the use of a variety of instructional strategies to teach all learners, including students of different cultural and ethnic backgrounds.

Accomplished Practices (APs) - With regard to the 12 APs, students cannot graduate without having demonstrated the 12 APs. The assessment structure is common across the unit; however, implementation varies within the departments. The Special Education Department, for example, requires that all students develop a portfolio that demonstrates their competency based on coursework and field experiences. A structured review process, which occurs over the five semesters of the program, includes regular feedback and opportunities for students to continuously improve their products. The portfolios are structured around the Accomplished Practices as well as the Council for Exceptional Children competencies. Each of the 12 APs represents one domain of the portfolio. Students complete reflective narrative descriptions of their work describing what they have learned during their program and how they have applied what they have learned. Please see Improvement Report, Standard 3, Indicator 4, Annual Review and Continuous Improvement, "Graduates' Performance on Accomplished Practices," pp. 11-14.

Length of Final Internship: No changes have been made since last year's report. The internship at the undergraduate level and at the Master's Plan II level is 15 weeks. In Counselor Education, students spend 15 weeks (600 hours) in a full-time internship. In School Psychology students complete a full-time full-year internship of at least 1500 clock hours. In Speech Impaired, after inhouse practica, students do an internship in schools for 16 hours/week for 12 weeks and must complete 350 clock hours with a minimum of 192 at a site. Standard 3, Indicator 4 - Annual Review and Continuous Improvement: State-mandated requirements related to annual review and continuous improvement of the program are implemented and monitored Due to many administrative and personnel changes in the Dean's Office, the USF 2001 IPEP was not submitted to the FLDOE until December 2002. Consequently, we have not yet received the results of our 2001 IPEP review.

Institutional Program Evaluation Plan: Changes have not been made on the most current IPEP since we have not yet received feedback from the DOE.

Stakeholder Involvement: In August 2002 the institution sponsored a two-day College-wide retreat. College faculty and program coordinators were invited to participate in a forum to focus on the next accreditation visit.

10

Annual Review - Remediation of Weaknesses: Please see Part III.A. Improvement Report, Standard 3, Indicator 2. The Instructional Strategies, Assessment, and Technology, and ESOL section starting on page 6 addresses the concerns noted in the FLDOE responses to the 1999 and 2000 IPEPs, indicating that the relationship between and among assessments was inconsistent or vague. Continuous Improvement: The most recent results of the FTCE exam indicate that 100% of program completers taking the exam passed.

Graduate’s Performance on Accomplished Practices- In an effort to assess overall performance with respect to the Preprofessional Benchmarks, items on our 2001-2002 Graduating Senior Exit Survey were evaluated with regard to their relationship to the 12 Accomplished Practices. Composite items were created to represent each of the 12 Accomplished Practices, and acceptable performance was based on achieving an average score of 3.0 or higher in each area. If we turn our attention to the reported level of performance with regard to these Benchmarks (see Table 1), we see evidence that our graduates perceive themselves as achieving a high level of success across the broad range of Accomplished Practices. The graduating seniors reported an acceptable level of performance that ranged from 78% to 95%. The highest levels of success were reported for Accomplished Practices 6 (Ethics), 3 (Continuous Improvement), 2 (Communication), 10 (Planning), and 8 (Subject Matter Knowledge) with over 90% of graduates reporting acceptable performance in theser areas. The area revealing the most room for improvement was Accomplished Practice 12 (Technology), suggesting that future efforts should be focused on enhancing graduates’ ability to use available technology at the school site in appropriate ways, to facilitate student access to electronic resources and to use technology to manage, evaluate, and improve instruction. This was the only area where fewer than 80% of the respondents (78%) reported adequate preparation.

han Table I

Percentage of Graduates Reporting an Adequate Level o Preparation in Accomplished Practices

Reporting

Number Accomplished Practice Adequate Preparation

1 Assessment 87

2 Communication 92

3 Continuous Improvement 94

4 Critical Thinking 89

5 Diversity

6 Ethics

7 Human Development & Learning

8 Subject Matter Knowledge 9

9 Learning Environments

10 Planning

11 Role of the Teacher

12 Technology

8

9

90

8

92

8

Note: Percentage of respondents reporting an acceptable level of preparation (adequately prepared, well prepared, or very well prepared).

Alumni Survey: In an effort to assess perceptions of overall performance, with respect to the Preprofessional Benchmarks, items on our 2001-2002 Alumni Survey were examined with regard to their relationship to the 12 Accomplished Practices. Composite items were created to represent each of the 12 Accomplished Practices, and acceptable performance was based on achieving an average score of 3.0 or higher in each area. If we turn our attention to the reported level of performance with regard to these Benchmarks (see Table 2), we see evidence that our graduates are attaining a very high level of success across a broad range of abilities. The alumni reported acceptable levels of performance that ranged from approximately 80% to 100%. If we consider Accomplished Practice 1, Assessment, as represented by item 11 (i.e., design and use of a variety of assessment methods) we see that 94% of the bachelor's level alumni rated themselves as average or above, while 100% of master's level alumni reported average or above performance. Interestingly, the highest levels of performance evidenced for all alumni were observed to be in the areas of Communication and Ethics, with 100% of respondents reporting performing at an acceptable level for each of the aforementioned accomplished practices. The area that appears to have potential for improvement at the bachelor's level is Technology, suggesting that future effort should be focused on enhancing students' use and integration of appropriate technology in teaching and learning processes. This was the only area where acceptable performance was reported to be less than 90%.

12

Table 2

Percentage of Alumni Reporting an Acceptable Level of Per ormance for Accomplished Practices by Degree Level

Number Accomplished Practice Bachelor's Master's

Degree Degree

1 Assessment 94 100

2 Communication 100 100

3 Continuous Improvement 98 100

4 Critical Thinking 92 100

5 Diversity 92 100

6 Ethics 100 100

7 Human Development and Learning 98 100

8 Subject Matter Knowledge 97 100

9 Learning Environments 92 100

10 Planning 91 100

11 Role of the Teacher 94 100

12 Technology 80 96 Employer Satisfaction Survey: In an effort to assess perceptions of overall performance, with respect to the Preprofessional Benchmarks, items on our 2001-2002 Principal Survey were examined with regard to their relationship to the 12 Accomplished Practices. Composite items were created to represent each of the 12 Accomplished Practices, and acceptable performance was based on achieving an average score of 3.0 or higher in each area. If we turn our attention again to the reported level of performance with regard to the Preprofessional Benchmarks (see Table 3), we once again see evidence that our graduates are attaining a very high level of success across a broad range of abilities. The principals reported acceptable levels of performance that ranged from approximately 92% to 100%. An acceptable level of performance was evidenced for more than 90%

13

of alumni (at both the bachelor's and master's level) across all of the 12 Accomplished Practices. Interestingly, the same strengths appeared for graduates at both levels. The principals reported that the greatest percentage of alumni demonstrated acceptable performance in Communication, Ethics, Human Development and Learning, and Subject Matter Knowledge regardless of degree level. These findings suggest that as a whole, principals in schools that employed USF alumni are very satisfied with placement of alumni in Florida schools. On a host of different dimensions, our graduates appear to be performing at admirable levels, casting teacher preparation programs at the University of South Florida in a very favorable light.

Table 3

Percentage of Principals Reporting an Acceptable Level of Performance for

Accomplished Practices by Degree Level

Number Accomplished Practice Bachelor's Master's

Degree Degree

1 Assessment 92 94

2 Communication 98 100

3 Continuous Improvement 92 94

4 Critical Thinking 94 100

5 Diversity 94 100

6 Ethics 97 100

7 Human Development and Learning 97 100

8 Subject Matter Knowledge 97 100

9 Learning Environments 93 94

10 Planning 94 97

11 Role of the Teacher 94 94

12 Technology 92 94

14

Reporting Requirements: The College of Education follows the required reporting requirements. The most recent Title II Report was submitted to the FLDOE. The College's graduates do exceedingly well on the statewide teacher certification exams; they had an eligibility rehire rate of 100% in 2001-2002.

Recency of Experience: This initiative was implemented in the Fall of 2002. A form was developed for faculty to record their involvement in classroom instruction in local school districts and for chairpersons to verify such activity. All departments are in compliance.

Standard 4: The diversity of student population enrolled in each program will, over each fiveyear period, increase in, both the number of minority students and the number of underrepresented classes appropriate to an institution's mission. The percentage of minorities has continued to increase at the undergraduate level and at the graduate level for the past six years. The percentage of minorities graduating from programs has also increased at both levels.

The Interim Dean established a diversity task force to address the College's strategic goals related to diversity: to enhance the valuing of diversity in multiple forms via recruitment, support and retention of faculty, students and support staff; to infuse diversity throughout the curriculum at both graduate and undergraduate levels; and to foster the development of culturally competent graduates, students, faculty and support staff.

Standard 5: Feedback from Florida employing districts will verify that 90% of the program graduates who complete their first year of teaching will be rehired, or in the case of "downsizing," will be eligible for rehiring. The procedure in place for collecting feedback from the school districts regarding "rehirability" for our program graduates is working well, and the districts continue to cooperate with our requests for data. The only improvement we made this year regarding collecting feedback from the school districts on program completers hired in their schools was to revise the table which accompanies the cover letter to the districts. The resulting feedback indicated that virtually 100% (%99.8) of our 1999-2000 program completers hired in the districts during 2000-2001 were eligible for rehire in 2001-2002. Please see Chart 5 and Explanation for specifics. Part 111.13. Improvement Section (Plan): Academic Year 2002-2003

Attach a list of the improvements planned for the institution for the academic year 2002-2003, organized by the Continued Program Approval Standards.

Standard 1: All students who complete the program will demonstrate the 12 Accomplished Practices at the Preprofessional Benchmark as described in the institutional evaluation plan. The College has developed a strong system that verifies that students are not given a final grade until the Internship Office has received the appropriate documentation that demonstrates each student's competency pertaining to the Florida Accomplished Practices prior to graduation. No changes to

15

this system are anticipated, although the system will continue to be evaluated for its effectiveness and efficiency.

Standard 2: 90% of the students in the program will pass the CLAST, the Professional Education, and Subject Matter Subtests of the Florida Teacher Certification Examination.

No improvements anticipated. The results of the Title 11 report are outstanding with a 100% passing rate for the professional education and subject matter subtests of the FTCE. Standard 3, Indicator 1: State-mandated requirements related to candidate component of the program - including admissions, retention, and progression, and program completion - are implemented and monitored, with improvements made as appropriate. Admissions Requirements: The system of granting waivers across undergraduate and graduate programs in the College of Education will continue to be evaluated and monitored. Retention and Progression: Students who receive 10% waivers will continue to be informed of the remedial services available to them so that they can work toward meeting competencies

Program Completion - FTCE: Students in all teacher preparation programs at USF are required to pass the FTCE prior to graduation. They also must complete the state-mandated ESOL courses according to the requirements stipulated for their program area. There will be no change in this policy. Standard 3, Indicator 2 - Curriculum Design: State-mandated requirements related to the curriculum design component of the program - including both content and delivery - are implemented and monitored, with improvements made as appropriate.

• Two new Master's in the Arts of Teaching programs have been submitted for approval by the Florida

Department of Education: the MAT in Mathematics Education and the MAT in Science Education. They are designed to meet the increasing need for alternative certification.

• A new Master's in Career and Technical Education is in the process of being approved and,

beginning in Fall 2003, will be delivered virtually 100% via distance learning. • The College Technology Committee has been actively discussing the implementation of a "Laptop

initiative" which would require a laptop for new students with the goal of getting them more involved in the educational process. The Committee has made a variety of contacts, including Apple, and is in the process of gathering more information. Meetings will be ongoing throughout the 2002-2003 year to explore the possibility of implementing this initiative.

• The Childhood Education department is planning on developing new certification for K-6.

• The College plans to bring the Science Ed Program in line with the state mandate. Currently,

separate programs are being offered in Biology, Chemistry and Physics.

16

The COE is proud to be the first ESOL Endorsement through Infusion program in the state of Florida that has received approval for the five areas of Early Childhood, Elementary Education, English Education, Foreign Language Education, and Special Education. Graduate students admitted to these programs in Fall 2002 or later will be required to take three ESOL courses which have a heavy emphasis on diversity and appreciation of ethnicity. Students will also be required to have an early and a late field experience and take a comprehensive ESOL Education examination. Faculty have begun planning the development of new graduate level courses to prepare Master's students to meet the state-mandated ESOL requirements. An Introduction to Teaching course, required for all teacher education majors, is being co-planned with faculty across departments to add perspectives from various disciplines. Surveys: A pilot test of the newly revised Graduating Seniors' Exit Survey is currently planned for fall 2003. Revisions are also planned for the current Intern Evaluation Form. It is our intent to refine this tool so that it more adequately represents the accomplishment of state standards (i.e., 12 Accomplished Practices), and is more reflective of the reality of today's classrooms. Future plans include the development of a Master's Program Alumni Survey and an Advanced Graduate Mentor Survey. Technology - A PT3 (Preparing Tomorrow's Teachers to Use Technology - see Standard 3, Indicator 2 in the "Improvement Report" section for description of grant, p. 7-8) state-wide conference is planned for April 2003 for state-approved teacher preparation programs to come together to focus on, discuss and experience resources for teaching pre-service teachers to use technology in their instruction. Several national experts will report what is happening nationally with technology in teacher preparation and the directions institutions are going regarding teacher preparation with technology. The PT3 project evaluators will collect data in Fall 03 that will include a college-wide review of course syllabi, including cohort and non-cohort member faculty, as well as a review of changes in the cohort faculty course syllabi. The PT3 evaluators developed and administered an ISTE-aligned faculty survey designed to gather information about faculty proficiency with and use of technology, with particular emphasis on how they use technology within the courses they teach. The survey will be readministered to all participating faculty members in early Fall 03, and again in the Spring of 04, and a broad cross-section of non-participating college faculty will be asked to take the survey in early Fall 03. Results from the surveys will be used to plan future faculty development projects. Student surveys aligned to the ISTE standards for pre-service teachers were developed and administered in Year Two of the PT3 grant (see Standard 3, Indicator 1 in the "Improvement Report" section, p. 8, for results). Evaluators obtained permission from the USF Institutional Research Board to gather email addresses and other information that will allow us to follow students for multiple administrations of the survey, as well as into their first year of teaching, in order to chart progress against the standards. In January 2003 a survey will be distributed to College of Education students to gather information about students' proficiency and training with

17

technology as a teaching tool and to inform our college-wide planning and preparation for NCATE and other accrediting agencies.

Standard 3, Indicator 3 - Clinical and Field-Based Component: State-mandated requirements related to the clinical and field-based component of the program are implemented and monitored, with improvements made as appropriate.

• The College is exploring ways to address the needs of in-service teachers seeking certification

through paid in-service internships. These paid internships apply only to graduate students and to individuals who are practicing teachers. Candidates for in-service internships are normally current teachers who are successfully performing the tasks of teaching and integrating what they learn in classes into their settings. These teachers are not willing to quit their jobs to take an unpaid pre-service internship in another school in order to complete degree requirements. Paid in-service internships will give teachers who have already been hired the opportunity to meet certification requirements through a final internship in their own classroom. Meetings are being planned to develop procedures for implementation of the paid internship policy. The College is also exploring the possibility of additional mentoring for these in-service teachers during their paid internships to increase the probability of teacher retention.

• A required practicum for field experience is being incorporated into the curriculum for programs that

do not currently require one and is expected to be implemented by the end of the academic year 2002-2003.

• The USF SunCoast Area Teacher Training Honors (SCATT) program offers a 3 graduate credit hour

course each summer to train experienced, excellent teachers to mentor interns and beginning teachers. Embedded in that class is the State of Florida clinical teacher training. Through a collaborative effort with Hillsborough county to be implemented by summer 2003, that course will include a cadre of 22 teachers who will address issues related to diversity in their classrooms/schools through service learning. Upon completion of the clinical course the cadre will receive two full days of additional training by the district to implement service learning in urban classroom settings. USF interns can be assigned to those classrooms to support those efforts. The district is exploring the impact of service learning on urban student populations.

This collaborative plan will provide teachers with 3 graduate credit hours of clinical training, prepare them to work with interns, and help them implement a teaching strategy which is predicted to better address engaging students in learning.

Standard 3, Indicator 4 - Annual Review and Continuous Improvement: State-mandated requirements related to annual review and continuous improvement of the program are implemented and monitored. Institutional Program Evaluation Plan: The institution has an Institutional Program Evaluation Plan that outlines the manner in which the institution will address the continued program approval standards. This plan is approved by the Department of Education and is used annually to review programs.

18

Annual Review: Due to administrative and personnel changes in the Dean's Office, the USF 2001 IPEP was not submitted to the Florida Department of Education until December 2002. We have not yet received the results of our 2001 IPEP review.

Continuous Improvement: The results of the 2001-2002 Graduate Senior Exit and Alumni Surveys indicated that graduating seniors and alumni felt they were not sufficiently prepared to meet Accomplished Practice 12 (Technology). Instructors in the College are not necessarily equally committed to the use of technology in their own classrooms. Some incorporate technology on a regular basis and require the same of their students; some do not. Consequently, instructors will be informed on a regular basis of the technology workshops offered in the College and the university and will be encouraged to take advantage of the technology resource persons in the College, so that they can become more comfortable with integrating technology in the teaching and learning process and, thereby, assist students in enhancing their skills in this area. In addition, USF's federally funded PT3 TECH Initiative will continue to provide faculty with opportunities to become technologyproficient (see Standard 3, Indicator 2, p.17, for a description of one of the planned events).

Standard 4: The diversity of student population enrolled in each program will, over each fiveyear period, increase in both the number of minority students and the number of underrepresented classes appropriate to an institution's mission. The College continues to make efforts to increase its diversity on the faculty so that the demographics of the faculty more closely reflect the demographics of the students. The College's recruitment plan for 26 vacant positions was approved. Strategies outlined in the recruitment plan to identify and recruit outstanding minority candidates will be implemented, and every effort will be made to attract minority applicants. An increase in minority faculty should in turn attract more minority students to the College.

Ongoing diversity initiatives funded by grants are expected to increase the presence of students of color in our programs and to contribute to the diversity of the teaching pool in our surrounding districts.

Standard 5: Feedback from Florida employing districts will verify that 90% of the program graduates who complete their first year of teaching will be rehired, or in the case of "downsizing," will be eligible for rehiring.

Improvement Plan for 2002-2003: The major improvement planned is to send out the letters requesting feedback from the school districts earlier than we have in the past.

19

Appendix: Required Charts

Chart 1 Standard 2-FTCE Data

FTCE Data (from 1998-99 Standard Report)

DOE Prog. Program Title # of 1998-99 # Taking # Passing % Passing Code # Program Exam Exam Exam Com leters 114 Art 6 6 6 100 131 Elementary Ed 413 405 405 100 202 Music K-12 11 10 10 100 221 SLD K-12 69 67 67 100 232 Vary Exc K-12 51 47 47 100 277 PE K-8 8 7 7 100 278 PE 6-12 13 11 11 100 286 English 6-12 52 47 47 100 287 Math 6-12 17 17 17 100 288 Biology 6-12 10 9 9 100 289 Chemistry 6-12 2 2 2 100 291 Physics 6-12 1 1 1 100 293 Soc Sci 6- 12 60 56 56 100 298 Business Ed 6-12 6 5 5 100 Guid/Counseling 13 10 10 100 304 PK-12 309 EH K-12 44 43 43 100 311 Spanish K-12 4 2 2 100 Industrial Arts- 4 3 3 100 323 Tech 6-12 325 MH K-12 19 19 19 100 330 School Psy PK-12 9 5 5 100 ESOL 3 338 Endorsement Pre-K/Primary Ed 91 90 90 100 371

PK 3

20

Chart 2 Standard 3, Indicator 4

Program Completers Obtaining Full-time Teaching Employment within First Year of Graduation

# of 2000-2001 Program # of 2000-2001 Program % of 2000-2001 Completers Completers Hired in 2001- Program Completers 2002 per DOE Report Hired in 2001-2002 863 514 60%

Chart 3 Standard 3, Indicator 4

Average Length of Stay of 1998-99 Program Completers in their Full-Time Teaching Positions for a Three-Year Period-(School Years 1999-2001)

# of 1998-99 # Employed in # Employed for # Employed for Average Length of Program 1999-2000 only only Two Years Three Years (1999- Stay Completers (1999-2000 and one 2000, 2000-2001, other ear) and 2001-2002 979 76 134 474 2.58

Chart 4 Standard 3, Indicator 4

Satisfaction of Employers with Program Completers Performance

# of 2000-2001 Program # of Employers # of Employers Satisfied % of Employers Completers Responding to Employer with Program Satisfied with Program Satisfaction Survey Completers' Completers' Performance Performance _ 863 N/A N/A N/A

The information requested above for 2000-2001 program completers is not yet available. Employer surveys are sent out two years after completers' graduation (toward the end of their second year of employment). Employer survey administration for the 2000-2002 graduates is currently planned for the Spring of 2003. We can, however, provide this information for the 1999-2000 program completers. Please refer to the data below

# of 1999-2000 Program # of Employers # of Employers Satisfied % of Employers Completers Hired in Responding to Employer with Program Satisfied with Program Florida School Districts Satisfaction Survey Completers' Completers' (S rin 2002 Report) Performance Performance 610 323 313 97%

21

Chart 5 Standard 5 Data on Rehirability

DOE Program # 1999-00 #1999-00

# 1999-00 # 1999-00 % 1999-00

Prog. Program Program Program Program Program Code Completer Completers Completers Not Completers Completers# s Hired in Rehired in

2001- Rehired But Not Elig. for Eligible for

2000-01 2002 Elig. for Rehire Rehire in Rehire per DOE Per Add'] in 2001-02 2001-02 Report DOE Per Rpt. District

s

114 Art 3 3 100%117 Business Ed 3 3 100%131 Elementa Ed 225 201 12 12 100%202 Music 12 10 1 1 100%221 Specific Lrng 47 43 1 3 100% Disabilities 232 Varying 35 31 1 3 100% Exce tionalities 277 PE K-8 6 5 1 100%278 PE 6-12 7 7 100%286 English 31 29 1 1 100%287 Mathematics 10 8 1 1 100%288 Biolo 6 6 100%289 Chemist 2 0 1 1 100%290 Drama 1 1 100%293 Social Science 28 24 2 2 100%298 Business Ed 2 1 1 100%304 Guidance/ 15 14 1 100% Counselin 309 Emotionally 18 18 1 100% Handicapped 311 Spanish 2 1 1 100%323 Industrial Arts/ 3 3 100% Technical Education 324 Educational Media 24 22 2 100% Specialist 325 Mentally 7 5 1 1 86% Handicapped 330 School 9 6 3 100% Psychologist 339 Gifted 3 3 100%340 Middle Grades 1 1 100°l0352 SLD/MH 8 6 1 1 100%359 EH/SLD 37 30 5 2 100%371 Pre-K/Primary 63 59 3 1 100% Education 380 EH/MH 3 3 100% TOTAL 611 543 34 33 1

22

Review Process Used: Using the employment reports provided by the Florida Department of Education, the list of 1999-2000 graduates hired in 2000-2001 was compared with those employed in 2001-2002. The 2001-2002 list was missing 68 graduates who had appeared on the 2000-2001 list. Letters were sent to the employing districts along with a table indicating the graduates who were not rehired in the second year. Districts were asked whether or not graduates were in fact hired and, if not, if they were eligible for rehire. All districts responded. Of the 68 graduates missing from the DOE second-year employment list, 32 were in fact rehired and 35 were eligible for rehire. This indicates that 575 (94%) of the 1999-2000 graduates hired in Florida school districts in 2000-2001 were rehired during 2001-2002, their second year of employment. One graduate indicated as having completed the Mentally Handicapped program area was ineligible for rehire. Consequently, the eligibility rate for Mentally Handicapped is 86%. Please note, however, that this individual was actually certified in a different program area - Mental Retardation. The list of graduates included in this study is included as Attachment 1. Programs in Compliance and Programs Not in Compliance: Although the eligibility rate for the Mentally Handicapped Program is 86%, technically all programs are 100% in compliance. The graduate who was not rehired was actually certified in Mental Retardation, not Mentally Handicapped, as indicated on the DOE list. Mental Retardation is a low incidence program.

Explanation for Programs Not Meeting the 90% Criterion: Not applicable.

23