1 Evaluator Workshop Spring Visits 2011 Wednesday, December 8, 2010 Please join the audio portion of this training: 866-740-1260, Access Code: 7489001 ReadyTalk Help Desk: 800.843.9166 International Help: 303.209.1600
1. Evaluator Workshop Spring Visits 2011 Wednesday, December 8,
2010 Please join the audio portion of this training: 866-740-1260,
Access Code: 7489001 ReadyTalk Help Desk: 800.843.9166
International Help: 303.209.1600
2. Announcements This presentation and the accompanying
materials are available for download from:
http://www.wascsenior.org/spring2011 For assistance with Voice and
Web connections please contact: ReadyTalk Help Desk, 800.843.9166
International Help: 303.209.1600 Please mute your microphone if you
are not speaking. If you have questions, please enter them into the
Chat window.
3. WASC Evaluator Workshop Spring Visits 2011
4. Workshop Outcomes
Know how to prepare for and conduct an effective visit
Be prepared to produce a useful, high-quality team report
Be ready to make sound judgments about institutions under the
Standards
Be familiar with resources that support your work on a
team
5. Agenda
Context for the Visit/Accreditation
Preparing for the Visit
Conducting the Visit
Developing Team Recommendations
Writing the Team Report
6. Context for Accreditation and Visits
The continuing evolution of the WASC process and Standards
The accountability movement
Moving from assessment for improvement to assessment for
accountability
New areas of emphasis
The Handbook revision process
7. Recent Changes in the Institutional Review Process and
Standards
Changes to Institutional Review Process re: Student Success,
Program Review and EE Sustainability
Changes to CFRs
Clarifying the scope of the CPR visit to review the
infrastructure for assessment of student learning
Program Review and Program-Level Student Learning in a
systematic way
Tool: Table A & B (EVG pg 65 & 71, SVG pg. 63 &
69)
8. Covering the Impact of the Financial Recession on
Institutions
Questions to ask the institution:
How has the financial recession affected your institution?
How has your institution responded?
What plans are in place in case the current state of affairs
becomes permanent?
9. Q&A
Please feel free to type in your questions using the chat
window or just chime in.
10. The Three-Stage Review Process Institutional Proposal
Capacity & Preparatory Review Educational Effectiveness Review
1. 2. 3.
11. The Three-Stage Review Process
Institutional Proposal
Capacity and Preparatory Review
Educational Effectiveness Review
12. Timeline for Three-Stage Review Process Submitted 2 Years
before CPR review Capacity And Preparatory Review 18-24 months to
prepare for EER (or less for Candidacy or Initial Accreditation)
Educational Effectiveness Review Extended period of time to sustain
Initiatives (7-10 years) Proposal
13. Institutional Self-Review
The heart of accreditation
Built upon an effective internal process of
Evaluation
Self-reflection
Recommendations from previous reviews
Plans for action
14. Stage 1: The Institutional Proposal
15. The Institutional Proposal
Guides the entire accreditation review process
Connects institutions context and priorities with the Standards
of Accreditation
Provides primary basis for both institution self-review and
team evaluation
Allows alignment of accreditation activities to institutional
strategic plan and key areas chosen for improvement
16. The Letter of Intent
Submitted by institutions seeking Candidacy or Initial
Accreditation, the LOI serves the same purpose as the proposal
Includes suggestions from Eligibility approval letter
Submitted to assigned WASC liaison, one year in advance of CPR
Review
Instructions are in How to Become Accredited on WASC
website
17. Stage 2: The Capacity and Preparatory Review
18. Purpose of the CPR
Review and verify the information in the institutional
presentation (report and data)
Evaluate key institutional resources, structures, processes in
light of Standards
Evaluate institutions infrastructure to support and assess
student learning
Assess institutions preparedness to undertake the Educational
Effectiveness Review
19. Stage 3: The Educational Effectiveness Review
20. Purpose of the EER
Invite sustained engagement by the institution on the extent to
which it fulfills its educational objectives
Enable the Commission to make a judgment about extent to which
institution fills its Core Commitment to Educational
Effectiveness
21. Q&A
Please feel free to type in your questions using the chat
window or just chime in.
22. The Two Core Commitments
23. Core Commitment 1
The institution functions with clear purposes , high levels of
institutional integrity , fiscal stability , and organizational
structures to fulfill its purposes.
24. Core Commitment 2
The institution evidences clear and appropriate educational
objectives and design at the institutional and program level. The
institution employs processes of review , including the collection
and use of data , that ensure delivery of program and learner
accomplishments at a level of performance appropriate for the
degree or certificate awarded.
25. The Four Standards Tool: Standards of Accreditation (EVG
pg. 49, SVG pg. 47) Standards at a Glance (EVG pg. 61, SVG pg.
59)
26. Standard 1: Defining Institutional Purposes and Ensuring
Educational Objectives
Institutional Purposes
Integrity
27. Standard 2: Achieving Educational Objectives Through Core
Functions
Teaching and Learning
Scholarship and Creative Activity
Support for Student Learning
28. Standard 3: Developing and Applying Resources and
Organizational Structures to Ensure Sustainability
Faculty and Staff
Fiscal, Physical, Information Resources
Organizational Structures & Decision Making Processes
29. S tandard 4: Creating an Organization Committed to Learning
and Improvement
Strategic Thinking and Planning
Commitment to Learning and Improvement
30. Expectations for Two Reviews Tool: Expectations for Two
Reviews (EVG pg. 44, SVG pg. 42)
Educational Effectiveness
Demonstrating student learning
Demonstrating institutional learning
Demonstrating evidence-based decision-making
Capacity and Preparatory
Preparatory = readiness for the Educational Effectiveness
Review
Please feel free to type in your questions using the chat
window or just chime in.
32. Preparing for the Visit
33. Roles and Responsibilities of Team Members and Staff
Role of Team Chair (EVG pg. 259, SVG pg. 193)
Role of Team Assistant Chair (EVG pg. 261, SVG pg. 195)
Role of assigned WASC staff liaison (EVG pg. 9)
Team assignments
Tool:
The Role of the Team Chair & Assistant Chair/Team Editor
(EVG pg. 8)
Section 10 (Tips, Roles and Advice)
34. Timeline For CPR/EER Reviews 12 weeks 2 months Institution
mails report to team and WASC Team holds conference call Site visit
held and team report written Institution responds to errors of fact
in team report Institution responds to final team report Commission
acts at February or June meeting Tool: CPR or EER Timeline (VG, pg.
75, SVG pg. 73)
35. Pre-visit Preparation
Read all the documents from WASC
Standards, CFRs, policies, visit guide, rubrics
Background documents re: institution and purpose of the visit,
including Proposal and/or last action letter/team report
Read the institutional report
Review the data portfolio and exhibits
What to look for and highlight?
36. Reviewing the Exhibits
Enrollment data
Headcounts and FTE
Graduation data
Faculty data
Key financial indicators
Inventory of Educational Effectiveness Indicators
Inventory of Concurrent Accreditation and Key Performance
Indicators
Tool: How to Review WASC Data Exhibits (EVG pg. 97, SVG pg.
90)
37. Reading the Report
Has the institution done what it said it would do in its
Proposal?
Has it collected and analyzed data effectively?
Are its conclusions supported by evidence?
Are there serious problems or potential areas of
noncompliance?
Does the report contain recommendations for further
institutional action?
38. Worksheet for Team Pre-Visit Conference Call
Organizes teams responses to institutional materials
Helps team make preliminary evaluation under the Standards
Provides basis for team to work toward consensus
Should be submitted in advance of call
Tool: CPR Conference Call Worksheet (EVG pg. 84)
EER Conference Call Worksheet (EVG pg. 88)
SV Conference Call Worksheet (SVG pg. 82)
39.
Evaluates quality of institutional report and alignment with
Proposal and previous action letter(s)
Identifies areas of good practice, improvement, and further
inquiry
Identifies issues, strategies, evidence needed
Identifies persons and entities to be interviewed
Makes or refines team assignments
Plans visit logistics
Team Pre-Visit Conference Call
40. Off-Campus Sites and Distance Education Programs (special
requirement for some visits)
Prior to Visit: Sites/online programs will be identified and
assignments made
Review substantive change action letters to determine if issues
have been identified
Develop plan for the review of the programs and/or sites
During Visit
Interview faculty, administrators and students
Evaluate facilities OR online infrastructure
Observe classes (can be done ahead for online)
Document visit and findings in the appendix
Discuss important findings with team for inclusion in report,
as appropriate
What evidence is provided to show capacity and readiness for
EE?
What are the strengths and weaknesses of the evidence?
What other evidence do you want to review to evaluate capacity
and preparation for EE?
Do any issues arise with regard to the Standards?
Meetings: format/methodologies
43. Determining Strategy for EER Visit
What evidence is provided to show EE?
Why was it chosen?
What are the strengths and weaknesses of the evidence?
What other evidence do you want to see to evaluate
effectiveness?
Do any issues arise with regard to the Standards?
Meetings: format/methodologies
44. Drafting in Advance of the Visit
Assistant Chairs draft outline of team report and Section
I
Team members draft outline or text for which they are
responsible, using institutions report and data portfolio, with
space for additional evidence, analysis and conclusions
Tool: Guidelines for Drafting Preliminary Report Narratives
Prior to the Visit (EVG pg. 92, SVG pg. 86)
45. Q&A
Please feel free to type in your questions using the chat
window or just chime in.
46. Conducting the Visit
47. Launching the Visit: Team Executive Session
Discuss preliminary findings
Identify major issues for exploration
Refine lines of inquiry for each meeting
Confirm team assignments
Discuss use of tools and rubrics
Review preliminary outline/draft team report
Discuss options for confidential team recommendation
Review schedule
48. Visit Schedule
Executive sessions and debriefings with team only
Meetings and interviews with key individuals and groups
Open meetings with students, faculty and staff
Document review
Time for drafting report sections
Final exit meeting
49. Confidential Email Account
Set up by WASC as extension of open meetings
Checked by Assistant Chair during visit
Important emails shared with team and investigated
Comments included in team report only if the institution has a
chance to address them
Ask questions that elicit information, stimulate discussion, or
require judgment
Avoid interrogation, leading questions, or loaded language
Avoid consultation, giving solutions, or talking about your
institution
Let them do the talking
52. Addressing Student Success
All CPR and EER reports should address this topic
Teams should designate someone to study and write about student
success
Discussion should include summary of data, identification of
issues, and plans for improvement
53. Evaluating Program Review and Student Learning on EER
Visits Tool: EE Toolkit (available online) Suggested Approaches for
Evaluating Program Review (EVG pg. 174)
54. EER Toolkit
55.
56.
57.
58. Rubrics: Assessment of Student Learning
Academic Program Learning Outcomes
Use of Portfolios in Assessing Program Outcomes
Use of Capstones in Assessing Program Outcomes
Integration of Student Learning Assessment into Program
Review
General Education Assessment Process
Tool: Rubrics for Evaluating the Effectiveness of Assessment
Practices (EVG pg. 180)
59. Educational Effectiveness Framework
Use with team to evaluate institutions place
Use language of rubric to describe the institution in the
report
Ask the institution to evaluate itself and discuss
Confer with team toward end of visit to mark a copy of the
EEF
Team chair communicates commendations and key recommendations
that will be included in report
Chair may ask team members to participate
The meeting is not a dialog, discussion or debate
61. Q&A
Please feel free to type in your questions using the chat
window or just chime in.
62. Developing Team Recommendations
63. Two Kinds of Recommendations
Team recommendations at the end of team report, delivered at
the exit meeting
Confidential Team Recommendation to the Commission for
action
Tools:
Commission Decisions on Institutions
(EVG pg. 233, SVG pg. 167)
Commission and Team Decision Indicators
(EVG pg. 243, SVG pg. 177)
64. Team Report Recommendations
Should be:
Overarching and important
Supported by evidence
Linked clearly to Standards and CFRs
Supported by text in the report
Distinguish recommendations from suggestions and observations
embedded in the report
Tool: Educational Effectiveness Framework
(EVG pg. 178)
65. Producing Effective Team Reports
66. Report Preparation Logistics
Follow report template
Start writing before the visit
Complete your sections on site and give to Assistant Chair for
editing together
Tool: Section 7 (Producing Effective Team Reports)
67. Using Evidence in Team Reports
Use qualitative and quantitative evidence
Select evidence carefully and purposefully
Connect evidence to an assertion or question
Analyze information; do not just set forth data
Let evidence suggest improvements
Use evidence that speaks to the institutions themes and the
team's questions
68. Team Use of the Standards and CFRs
Team judgments must be linked to specific Standards and
CFRs
CFRs must be cited in reports
Standards and CFRs form the basis for Commission decisions
Standards and CFRs provide a context for continuous quality
improvement
69. What is an effective team report?
Reflects a thorough assessment of the institutions capacity,
preparation, and/or effectiveness
Is evidence based
Cites the Standards and CFRs
Provides the basis for a sound and supportable Commission
decision
Identifies important areas for institution to address
70. Tips for Writing Team Reports
Consider multiple audiences: institution, Commission, and next
team
Know your areas of responsibility, including length and depth
of your section
Start writing before you arrive on campus
Address priorities and goals set by the institution
Address Commissions concerns (last action letter)
Make commendations, but dont overdo it
Use praise that doesnt send wrong or mixed signal
71. More Tips on Team Reports.
Be sure to check facts
Support findings and recommendations with evidence --and tie
them to CFRs
Ensure evidence is sound and valid
Distinguish recommendations from suggestions or
observations
Use formal language and tone (e.g., not we/they)
Dont mention personnel by name
Dont prescribe solutions
72. Q&A
Please feel free to type in your questions using the chat
window or just chime in.
73. After the Visit
74. What happens next?
Assistant Chair prepares draft for Chair, team and staff
review; changes as needed
Chair sends to institution for corrections of fact
Chair finalizes draft and submits to WASC
Chair sends Confidential Team Recommendation and completed EEF
to WASC
WASC sends report to institution
75. Then
Staff prepares draft action letter, which is reviewed by team
Chair
Commission Panel reads report and documentation including
institutions written response, meets with institutional
representatives at Commission meeting
Panel makes recommendation to Commission, and Commission
acts
Staff finalizes draft action letter on behalf of
Commission
76. Also after the visit.
Team members send reimbursement forms to WASC within 30
days
Hotel arranged and paid directly by institution
Travel / food reimbursed
Rental car must be approved in advance by WASC staff
Spouse or assistant costs not covered
See policy for more details
Team members should not have any contact with the
institution
About the visit OR
Consult with the institution for one year
77. The Teams Impact
Peer review is the foundation of accreditation.
The team report forms the basis for the Commission action and
its letter.
The team report and action letter inform the work of the
institution for years to come.
Why were you chosen for a team?
78. Resources for Teams
Appendices of Visit Guide
Team Materials and Institutional Report mailed 10-12 weeks in
advance of visit
WASC Website: www.wascsenior.org
WASC Email Advisory (sent prior to visit)
WASC Staff
79. Q&A
Please feel free to type in your questions using the chat
window or just chime in.
80. Thank you for your service to the region
81. Announcements
The materials presented during this webinar and a recording of
this session will be posted at: