The SELF

STUDY

REPORTUAGM-CUPEYNovember 21, 2019

Dr. Ana Maria Schuhmann

Session 4

Objectives

What is the Self-study Process

Getting Started

Seton Hall Approach & Findings

Writing the Self-study Narrative & Exhibits

Issues with SSRs

UAGM-Cupey Site Visit- December 2023

- May-June 2022-Request a date for the visit

-June 2022- Template for Self-study report (SSR) and evidence room are opened in AIMS-18 months prior to a site visit

SELF-STUDY REPORT (SSR)

March 2023-SSR due nine (9) months

prior to site visit

- Narrative

- Evidence

From CAEP’s

Accreditation

Handbook

The self-study report is the collection of evidence and

supporting narrative which forms the basis of the

accreditation review and is the first source of information

for the site team.

SELF-STUDY REPORT (SSR)

Written by the EPP to make the case of meeting the

CAEP standards

The Evidence Room is an electronic space where

EPPs upload their “evidence” demonstrating how

they meet the CAEP standards.

SELF-STUDY REPORT (SSR)

If the EPP has initial and advanced level programs,

one self-study report is submitted.

The Accreditation Council reviews the accreditation

documents for each EPP and makes accreditation

decisions for the EPP. The Accreditation Council

makes two separate decisions. There is one decision

for initial licensure and one for the advanced-level,

with areas for improvement and stipulations assigned

(as appropriate) for each.

The Self Study Process

EPP leaders and faculty should engage in collaboration prior to developing a self-study narrative, then outline and compile evidence in support of their case for accreditation

The self-study process is the mechanism through which an EPP evaluates its programs and prepares its case for accreditation.

The process allows for focused analysis and reflection, includes steps for improvement, and serves as a means of accountability to the EPP’s stakeholders.

A self-study report documents the results from that process and demonstrates how the EPP is meeting each of the five Initial Licensure and five Advanced-Level CAEP Standards, along with diversity and technology themes.

The Process-How to Approach it

Form committees by standards?

Have a CAEP steering committee?

Who chairs the committee (s)?

Do you have a CAEP coordinator?

Do you have an assessment coordinator?

Do you have an institutional assessment

coordinator?

How do we

get started?

Form a team/committee(s)Take inventory by standard-What evidence/data do we have?

What is missing? What assessments need to be developed?

What assessments need to be modified to meet CAEP standard for EPP-developed assessments?-Validity? Reliability?

What documents need to be developed? Recruitment Plan? Quality Assurance System?

Taking Inventory/What Seton Hall used

to get going: CAEP Resources:

CAEP Standards

CAEP Rubric for EPP-developed Assessments

CAEP Rubric for Team Visitors

CAEP Handbooks

CAEPcon attendance & powerpoints

Seton Hall Resources

SSR/IR from previous accreditation

Assessment Grid-Handout

Timeline to the SSR with what needed to be done, dates, person in charge and an accomplished column

Other Resources:

InTASC Standards

Research on Focus Groups

INVENTORY: Standard 1 Holes

Field Experience Evaluation: Needed rubric

Assessments in place needed to be aligned with

InTASC standards

Lesson Plan rubric needed revision

EPP-created assessments needed validity & reliability

Standard 2 Issues

No input from stakeholders on a regular basis

No Cooperating Teacher evaluation of University Supervisor and vice-versa

Weakness in clinical work for post-bac candidates

No definite policy on diversity placement & little tracking

Standard 3 Issues

No Recruitment & Retention Plan

No assessment of local areas of

shortages

Determining cohorts

Collecting data on cohorts

Standard 4

Needed to send employer survey

Needed to send alumni survey

Needed to conduct focus groups for “multiple

measures”

Standard 5 Issues

Assessment System document needed revision

Data needed to be shared, analyzed, and utilized on a regular basis

Process/changes needed to be documented

Validity & Reliability

Writing the SSR

Start a draft as soon as possible

Write the draft in word document, not in

AIMS directly

Provide Context for Reviewers – programs

offered and structure of EPP

Example from SHU for 1.5 Technology

The EPP ensures that candidates model and apply technology standards as they design, implement and assess experiences to improve learning, by infusing technology in all courses and clinical work (Technology Crosswalk, E 1.5.b). Exhibit 1.5.a triangulates data from seven assessments to provide evidence for 1.5. All candidates are required to complete EDST 3700: Integrating Technology in Education. In this course, candidates create an instructional plan that supports effective digital-age learning environments to maximize the learning of all students and fulfills ISTE Standards 2 and 3.

Candidates' ability to model and apply technology to improve P-12 learning is assessed in EDST 3700, and during clinical experiences and practice through the O&C and CCI. Three indicators/competencies in the CCI assess candidates' technology skills in the classroom. Candidates do well in all of them with EPP means ranging from 3.19 to 3.73 (out of 4) in the three administrations of the assessment. Secondary candidates have better means than their elementary education peers in "provides evidence of maintaining and analyzing accurate student records".

Example-Continued

The O&C assessment has one criterion that evaluates candidates' understanding and use of technology to develop content and skills. The data presented are for the fourth clinical experience prior to student teaching/clinical practice as assessed by the clinical supervisor. Means range from 2.92 to 3.28 (out of 4-passing is 2.75) with elementary candidates performing slightly better than secondary candidates.

The revised lesson plan piloted in fall 2016 with elementary education candidates assesses the integration of technology. Candidates had 2.50 (out of 3.0) means in the pilot study. The Exit/completer survey asks candidates if they are able to effectively integrate technology into teaching. Means ranged from 3.31 to 3.76 (out of 4) indicating that candidates feel confident implementing this skill.

Example-Continued

The Alumni survey questions graduates about preparation received in

technology. Ratings on the statement "the program provided me

opportunities to integrate technology into teaching and learning" have

been increasing since 2012 when 41% "strongly agreed", to 55% "strongly

agreed" in 2014, to 61% "strongly agree" in 2016.

The Employer survey provides further evidence on EPP graduates ability

to model and apply technology. The biggest improvement between the

2012 and 2016 administrations of the survey is in the indicator:

"Effectively integrates technology into teaching and learning". In 2012,

91% of employers rated graduates in the two top categories of the

survey, while in 2016, 100% did.

Questions

What is the argument presented?

What are the data sources?

Is there any action/follow up recommended? Any

implications?

Writing the Narrative

Frame the argument to be made for a standard

Describe the data sources (representativeness, relevance, and credibility for the standard)

Draw a conclusion about the extent the data support the standard (triangulation and convergence of evidence)

Discuss implications of the findings for subsequent action

Use a summary statement to make a case for meeting the standard based on evidence

Evidence/Exhibits

Label them carefully, give them a title

Indicate for what standard it provides evidence

Number them by standard and component

Make sure the numbers and titles match the

narrative

Number the pages on each exhibit

Exhibit number, title, and standards

Exhibit 1.5a M o d e l a n d A p p l y T e c h n o l o g y CAEP Standards:1.5, 2.3,TCCT

EVIDENCE for CAEP 1.5 Candidates Model and Apply Technology Standards

CAEP Standard/Component: 1.5, 2.3, TCCT

InTASC Standards: 7, 8, and 9, NJPTS: 7, 8, and 9

EPP Conceptual Framework/Goals: Competence

Context and Data: The exhibit presents data from seven different assessments to provide evidence that candidates model and apply technology standards as they design, implement, and assess learning experiences.

The first set of data comes from the course that all candidates must successfully complete: EDST 3700: Integrating Technology in Education. Data is presented from three administrations of the assessment: fall 2015, spring 2016, and fall 2016. In this course, candidates’ ability to plan utilizing technology tools is demonstrated by the development of an instructional lesson plan. The lesson plan incorporates accommodations and modifications for two students with IEP’s- one with a designated learning disability and one with a designated physical disability. Incorporating best practices in instructional design, candidates create an instructional plan that supports effective digital-age learning environments to maximize the learning of all students.

SSR Issues: Narrative

Not clear what programs (licensure areas) are part of

the CAEP review

Not clear on the option for program review

Not addressing components in standards

Not addressing the InTASC Standards or InTASC

Categories

Technology not addressed or addressed superficially

SSR Issues: Narrative

Mutually beneficial partnerships

No stakeholder input

No online materials/training for CTs or supervisors

Minimal clinical experiences

SSR Issues: Narrative

No systematic process for data sharing and

decision-making

Using Exit surveys as part of Standard 4

No response rates on surveys (CAEP expectation:

20%)

SSR Issues: Narrative

No recruitment plan OR plans for the

University, OR plans for the system

No assessment of local/state needs

Dispositions issues

No Gateways

SSR Issues: Narrative

Language not consistent

Numbers not consistent

Organization

Errors, spelling, grammar

SSR Issues: Data

Fewer than 3 cycles of data for all assessments

Check-lists/checkboxes vs. rubric or evaluation

Data not disaggregated by program, and aggregated for the EPP

Data not disaggregated by InTASC standard or category (e.g. no data for Learner & Learning, Category 1)

SSR Issues: Data

Rubrics not actionable

No validity and reliability for EPP-created assessments

No changes based on data

No analysis, trends, comparisons, differences

SSR Issues: Evidences

Technical Issues: Files do not open, file corruption, missing files

Puerto Rico: Exhibits in Spanish

Loading evidences into AIMS

Exhibits without titles or numbers or standards or page numbers

REGUNTAS?

QUESTIONS?

THANK YOU !

!GRACIAS!