Field Test of Psychologist/ Psychometrist Evaluation System July 2000 Alabama Professional Education Personnel Evaluation Program

Field Test of Psychologist/Psychometrist

Evaluation System

July 2000

AlabamaProfessionalEducationPersonnelEvaluationProgram

Provides feedback from those who are being evaluated and those who are evaluators

Provides data for setting standards

Alabama Psychologist/Psychometrist Evaluation SystemField Test

Provides a sense of how well the instruments and procedures are working

Provides information on what areas of the evaluation system need changing

Alabama Department of Education Psychologist/Psychometrist Field Test Evaluation

Eight Competencies: Structure parallels other evaluation systems

1.0 Preparation for Delivery of Services

2.0 Delivery of Psychological/Psychometric Services

3.0 Evaluation of Psychological/Psychometric Services

4.0 Management of the Psychological/Psychometric Program

and Student Behavior

5.0 Positive Professional Climate

6.0 Communication

7.0 Professional Development and Leadership

8.0 Performance of Professional Responsibility

Competencies

Alabama Department of Education

Alabama Psychologist/Psychometrist Evaluation System

Psychologist/Psychometrist Field Test Evaluation

Data Collection Instruments/Forms


• Self-Assessment

• Interview (oral or written)

• Supervisor Review Form

• Professional Development Plan (generates Professional

Development/Leadership Competency score during next cycle)

• Evaluation Summary Report

• Portfolio• Surveys



Generates data for all competency areas

Optionally used by the psychologist/psychometrist and shared with evaluator only if psychologist/psychometrist agrees

Used by the psychologist/psychometrist to:

1) Identify areas that need improvement

2) Compare the psychologist/psychometrist ’s perceptions of performance with the results of the evaluation

3) Determine areas for professional growthactivities to be included in aprofessional development plan

Self Assessment




Two types of interviews:

A Structured Interview with 6 questions commonly asked of all psychologists/psychometrists (oral and written options)

A Clarifying Interview with probing questions focused on

clarifying or bringing understanding to the written interview and portfolio (questions differ from one psychologist/psychometrist to another)

Clarifying Interview conducted after the evaluator analyzes the written interview and portfolio

Modified scripting is used to record responses in oral option

The Interview




Generates data for competencies in Delivery of Psychological/Psychometric Services, Management,Communication, and Professional Responsibilities

Provides data where a supervisor should have objective, experiential information

Supervisor’s Review Form




Portfolio

There is no one specific form of packaging required for the portfolio, but there are organizational requirements

Each entry (piece of evidence) is accompanied by a Portfolio Entry Form

Total portfolio requires approximately 16 pieces of evidence

One piece of evidence might serve 2-3 purposes

With multiple purposes, a portfolio may have 9-11 entries (pieces of evidence)

Psychologist/Psychometrist is responsible for providing some evidence of his/her knowledge and skills

Generates data for 4 Competencies and 6 Indicators




Portfolio

Explicit guidelines for developing evidence in each area are provided in the evaluation/orientation manual

"Extra" entries will not be considered by the evaluator

Evaluated on the quality of materials rather than the quantity of materials in the portfolio

Portfolio



Indicator Evidence

2.1 Assessment and Evaluation Survey results2 artifacts

2.2 Intervention, Prevention Consultation

Survey results3 artifacts (1 in each area)

2.3 Education/Research Survey results2 artifact

3.1 Evaluates Program 1 artifact around evaluation1 artifact around use of results

6.2 Written Communication 2 artifacts around written work

8.4 Cooperation with School/Comm Survey results 1 artifact

PortfolioAlabama Psychologist/Psychometrist Evaluation System


Surveys are being field tested to determine whether they can be used in the Portfolio and Supervisor’s Review Form as part of the holistic scoring process

Surveys are provided for teachers, students in grades 4-6 and 7-12, and parents

Survey questions address the Competencies and Indicators

The Survey

The psychologist/psychometrist will distribute, collect, and analyze the surveys

Respondents are to remain anonymous and represent a cross-section within the group



Used to present the scores developed from each of the instruments

To complete the ESR, the evaluator:

1. Completes demographic information requested

2. Records the scores from each instrument

3. Writes comments, explanations, or justifications in the space provided that support the score for the specific area

4. Identifies 1 - 2 areas for focus in planning professional development for the coming year(s)

Evaluation Summary ReportAlabama Psychologist/Psychometrist Evaluation System


Activities focus on areas of improvement as identified in the Evaluation Summary Report

Activities focus on the improvement of student achievement

Professional Development Plan

Used to evaluate Competency 7: Professional Development and Leadership after first year of evaluation

Professional

Development

Plan



Questions?


SYSTEM COMPARISONS

Teachers Special Ed. II Speech/Lang. Counselors Librarians

Observation Observation Observation InstructionalObservation

ManagementObservationInstructionalObservation

Interview 1 added question 1 added question slight differences slight differences

SRF SRF SRF SRF SRF

PDP(Competency 7)

PDP(Competency 7)

PDP(Competency 7)

PDP(Competency 7)

PDP(Competency 7)

Portfolio(7 indicators)

Surveys(Students,

Parents, Faculty/Staff)

Surveys(Students,

Faculty/Staff)

ESR ESR ESR ESR ESR

Teacher System Psychologist/Psychometrist System

Eight Competencies Eight Competencies

COMPETENCIES

PSYCHOLOGIST/PSYCHOMETRIST

Preparation for Instruction

Presentation of Instruction

Assessment of Student Performance

Classroom Management

Positive Climate

Communication

Prof. Development/Leadership

Professional Responsibilities

Preparation for Delivery of Services

Delivery of Psychological/PsychometricServices

Evaluation of Psychological/PsychometricServices

Management of Psychological/Psychometric Program and Student Behavior

Positive Professional Climate

Communication

Prof. Development/Leadership

Professional Responsibilities

Teacher System

Self Assessment Self Assessment(Same format)

Interview Interview

Observation

(3 questions, 5 indicators.Focus around instructionalpreparation, studentassessment)

(6 questions, 7 indicators. Focus around preparation of services, delivery of servicesand evaluation of those services

(17 indicators) No Equivalent

Pre/Post Observation No Equivalent

INSTRUMENTS/DATA SOURCESPSYCHOLOGIST/PSYCHOMETRIST

Psychologist/Psychometrist System

Teacher System

(Adds program parts of services, management, and climate. Rest sameas teacher system)

Supervisor’s Review Form Supervisor’s Review Form

Evaluation Summary Report ESR is same except for adding portfolioscoring and removing observation

PDP PDP has no changes

(Not scored in field test since no pastPDP has been developed)

Competency 7 Scoring Competency 7 Scoring is same

INSTRUMENTS/DATA SOURCESPSYCHOLOGIST/PSYCHOMETRIST

Portfolio and surveys (No Equivalent)

Psychologist/Psychometrist System

PORTFOLIOPSYCHOLOGIST/PSYCHOMETRIST

Indicator Evidence

2.1 Assessment and Evaluation 1 survey results2 artifacts

2.2 Intervention, Prevention, and Consultation

1 survey results3 artifacts around each of the three areas

2.3 Education and Research 1 survey results2 artifacts

3.1 Evaluates Program 1 artifact around evaluation1 artifact around use of results

6.2 Written Communication 2 artifacts around written work

8.4 Cooperation with School/Comm 1 survey results1 artifacts

Documents

Field Test of Psychologist/ Psychometrist Evaluation System July 2000 Alabama Professional Education Personnel Evaluation Program